koobas.hobune.stream
How to reliably send Kinect v2 data over MQTT?
I am trying to send Microsoft's Kinect v2 body data over MQTT to effectively map skeletal data without direct connection to the Kinect, but I can't seem to deserialize a Body[] correctly. Essentially, the Kinect is connected to Computer A and processes the data and publishes it over MQTT, where Computer B listens and retrieves the data, then maps joints to its screen. I am just having trouble sending the Body[] over MQTT. I am publishing the Body[] every frame in Update().
My current setup is using Newtonsoft's JSON.Net to serialize a List taken from Body[] and publish it to MQTT (using https://github.com/vovacooper/Unity3d_MQTT). I used this since the Body class isn't serializable (so I can't use JSONUtility?). My current set up:
Computer A
void Update() {
...
//trackedBodies is a List<Body> that contains the tracked Bodys
//client is MQTTClient that is connected
string bodyData = JsonConvert.SerializeObject(trackedBodies);
client.Publish("test", System.Text.Encoding.UTF8.GetBytes(bodyData));
...
}
Computer B
void client_MqttMsgPublishReceived(object sender, MqttMsgPublishEventArgs e) {
//Check MQTT for data, then deserialize
List<Body> bodyData = JsonConvert.DeserializeObject<List<Body>>(System.Text.Encoding.UTF8.GetString(e.Message));
Debug.Log(bodyData);
}
When there are no bodies being tracked, I get an empty Body[].
System.Collections.Generic.List`1[Windows.Kinect.Body]
When there are bodies being tracked, MQTT seems to halt and stop working on the JsonConvert.DeserializeObject() line. I am not getting errors, and Update() is still being called.
My limitations are that I need to use Unity and MQTT.
I know that was a lot but I would appreciate it if anybody knows how to help with what I currently have, or suggest a better alternative to my problem.
I see that some people also may have the same question. I have finally gotten a working solution for myself. I personally just exposed the Body object by creating a custom 'pseudo' class to represent the body and hold the Joint and JointOrientation arrays. I was able to send a list of these custom objects through MQTT by serializing them with Json.NET
Cheers.
