Realtime Datastream Vicon Nexus and Unity

@konchatzinikolaou
You can use Vicon's "Datastream SDK" to stream Nexus marker (and other) data into Unity. Vicon offers a third party plugin (Unity Plugin 1.3) which contains this "Datastream SDK" and shows some examples on how to stream segment data to Unity (can be easily adapted to marker data). Find the step-by-step instructions on how to setup the third party plugin here (Unity Plugin 1.3 Instructions). Ignore or remove the code for the head-mounted displays.

Follow these steps to aquire your marker data. First setup a datastream client and connect to the Vicon server. Then you enable streaming of marker data before querying Vicon frames in the update function. After getting the current frame you can obtain a marker's global translation by providing its name:

 ViconDataStreamSDK.DotNET.Client MyClient = new ViconDataStreamSDK.DotNET.Client();
 MyClient.Connect( "localhost"); //or the "hostname:port" e.g. 192.168.0.60:801
 MyClient.EnableMarkerData();
 MyClient.GetFrame(); //for repeated querying put in update function
 Output_GetMarkerGlobalTranslation Output = MyClient.GetMarkerGlobalTranslation( "Alice", "LASI" );
 //for subject "Alice" and pelvis marker "LASI"
 float xPos = (float)Output.Translation[0]/1000f; //marker global x-position in meters
 float yPos = (float)Output.Translation[1]/1000f; //marker global y-position in meters
 float zPos = (float)Output.Translation[2]/1000f; //marker global z-position in meters
 

Google for the Vicon DataStream SDK Developer’s Guide to have all available functions for interacting with Vicon data from Unity (look at the .NET example code).

Dear members, my question is a follow up on the current one, and I couldn't post it as a separate question for some reason. Specifically, the question is regarding how can I properly map the segments in Vicon and the bones in my fbx model in Unity. Apologies if it is out of place here.

Hi, I am trying to work with data streaming in real time from Vicon Nexus to Unity. My ultimate goal is to display a hand model in Unity which would move according to the real-time motion of a real hand tracked by Vicon. I am using the Vicon Unity Plugin which includes the Vicon DataStream SDK.

So far I managed to connect to the Vicon server from Unity and I am able to get in Unity real-time coordinates of specific markers or segments. However, I have trouble when it comes to the hand model. I created in Blender a hand .fbx model with armature. For simplicity, the armature consists of two sets of bones constituted by 6 markers: one set has two bones for the index finger, and another set has two bones for the thumb. I imported the model in Unity.

In the meantime I recorder a trial in Vicon Nexus software of a moving hand. Specifically, I created a skeleton template consisting of two separate linked segments: one for the index finger and one for the thumb. I attached this template to my subject in Vicon. Each segment has the same amount of markers as the number of vertices in the .fbx model (3 markers each segment to make two bones for each finger). So, I have a Subject with two segments named "index" and "thumb".

To "make the hand move" in Unity, I attached the SubjectsScript.cs from the Unity plugin and set the respective Subject name. My trouble now is to properly match the segments in Vicon to the bones of the model in the Unity scene. In particular, in Unity I have a GameObject containing the hand mesh and the Armature. Further, the Armature has as children different bones of the index finger, as well as different bones of the thumb. But a bone of the model is defined as a "line" between two vertices whereas a segment in Vicon is a set of bones for the whole index finger of the thumb. Could anyone give me a tip, how exactly I can properly map the Vicon segments and bones of my fbx model? Do I need to modify Motion tracking in Unity using Vicon fbx model / skeletal configuration in Vicon? Or am I missing something basic? Thank you very much in advance!