Streaming data (point cloud ) to Unity?

Question

Hello,

I am trying to tackle 2 problems at the moment and I found no information for either:

1) How can I stream data to Unity? In this case I want to stream Kinect v2 data, but I am doing some preprocessing on the images that I m getting first, so the question is how do I connect the output of my application to Unity?

2) The data output type is point clouds. I 've seen some plugins on the asset store, but they are for loading static point clouds (and I don't get to write it/build it myself that way).

Any tutorials, reading resources, existing projects are extremely welcome. I assume I might have to write a plugin or something, which is way deeper water than everything I 've done so far (simple C# logic games in Unity).

Answer 1

Answer by triff · Jun 09, 2017 at 08:59 PM

Hey @Merxim - I'm poking about in similar areas at the moment and interested in using Kinect data within Unity for a visualisation/VFX project (although my background is in games).

This is a fair way over my techy/artist head, but there might be some info over here you can kick off with. In short, I think you want to look at the Kinect v2 SDK/Unity examples and fragment shaders.

https://social.msdn.microsoft.com/Forums/en-US/0a2de8ea-3437-4a31-be39-b14654160b82/kinectforwindows-unity-plugin-documentation-and-samples?forum=kinectv2sdk

Check this out too, I can't find StreamMeUp.unitypackage anywhere, RGBDToolkit seems to have metamorphosed into DepthKit and all the links are broken :( https://vimeo.com/90798783

There are some other pretty cool Unity examples on the RGBDToolkit Vimeo playlist: https://vimeo.com/album/2024049

Add comment · Share

Answer 2

Answer by steve-o616 · Feb 05, 2018 at 05:56 PM

Hey I just stumbled on this thread. I'm attempting something similar but search results are still coming up with very little for this workflow. Anyone know of any progress made in this area? Specifically the network live streaming of point-cloud data. Is this possible?

Add comment · Share

Answer 3

Answer by bmegli · Feb 11, 2020 at 04:05 PM

See example with Realsense camera.

Grabbing Realsense data (native c++)
HEVC Main10 hardware encoding (native c++)
streaming (native c++)
receiving (Unity native plugin)
hardware decoding (Unity native plugin)
unprojection (Unity native plugin)
wrapping point cloud in native buffer (Unity, no data copied)
rendering (Unity shader)

The video, the unity side code and the encoder side code.

There are more relevant links in video description.

@steve-o616, @triff, @Merxim

Add comment · Share

Answer 4

Answer by triff · Feb 11, 2020 at 04:43 PM

Finished my proj eventually, I did use Kinekt but with Brekel Point Cloud and 3ds Max/Redshift instead of the Unity/UE approaches I was exploring back in 2017. https://www.youtube.com/watch?v=2srkD_BTVkg

Add comment · Share

Streaming data (point cloud ) to Unity?

4 Replies

Your answer

Follow this Question

Related Questions