Kinect Map Depth into Color

Hello!

Im currently working on a project where I need to display a pointcloud using Kinect, which I already achieved. The result can be seen at the picture bellow. For this Im using Microsoft Kinect SDK v1.8.0 with a Kinect for XBOX hardware at Windows 7 and Unity Pro.

The problem is that the RGB camera and the Depth camera are not calibrated, so the pixels from each camera arent corresponding to each other at the moment.

After searching at the Kinect SDK documentation, I saw that there is this method called CoordinateMapper.MapDepthFrameToColorFrame, which does exactly what I need to match both streams.

But then again I cant seem to find this method within Unity Kinect Plugin. To be honest I even opened the kinect10.dll and search for it, but couldnt find it. So I decided to implement it myself. Here is my try:

At KinectInterop.cs, within NativeMethods class, I did this:

 [DllImportAttribute(@"C:\Windows\System32\Kinect10.dll", 
            EntryPoint = "MapDepthFrameToColorFrame")]
                            
     public static extern int MapDepthFrameToColorFrame(
            DepthImageFormat depthImageFormat,  
            ref DepthImagePixel[] depthPixels,  
            ColorImageFormat colorImageFormat,  
            ref ColorImagePoint[] colorPoints);

I also tried using the following instead with no success:

 [DllImportAttribute(@"C:\Windows\System32\Kinect10.dll", 
            EntryPoint = "CoordinateMapper.MapDepthFrameToColorFrame")]

The structs are defined before NativeMethods class and goes as follow:

 [StructLayout(LayoutKind.Sequential)]
     public struct ColorImagePoint
     {
         public int X;
         public int Y;
     }
  
     [StructLayout(LayoutKind.Sequential)]
     public struct DepthImagePixel
     {
         public Int16 Depth;
         public bool IsKnownDepth;
         public Int16 PlayerIndex;
  
     }
  
     public enum ColorImageFormat : int
     {
         RGB = 0,
         YUV = 1,
         Bayer = 2,
         Undefined = 3,
     }
  
     public enum DepthImageFormat : int
     {
         Resolution320x240Fps30 = 0,
         Resolution640x480Fps30 = 1,
         Resolution80x60Fps30 = 2,
         Undefined = 3
     }

Now at the main file:

     private Kinect.ColorImagePoint[] mappedDepthLocations;
         private Kinect.DepthImagePixel[] depthStreamPixel;
     
         void Start () {
         mappedDepthLocations = 
 new Kinect.ColorImagePoint[depthImageHeight*depthImageWidth];
         depthStreamPixel = 
 new Kinect.DepthImagePixel[depthImageHeight*depthImageWidth];
         }
     
         void Update () {
     
                 if (depthStream.pollDepth () && colorStream.pollColor()) {
     
     
                 for ( int i = 0; i < depthImageWidth; i++ )
                     {
                         for ( int j = 0; j < depthImageHeight; j++ )
                         {
                             int index = i+j*depthImageWidth;
                             depthStreamPixel[index].Depth = depthStream.depthImg[index];
                             depthStreamPixel[index].IsKnownDepth = true;
                             depthStreamPixel[index].PlayerIndex = 0;
                         }
                     }
     
                     Kinect.NativeMethods. MapDepthFrameToColorFrame ( 
                         Kinect.DepthImageFormat.Resolution320x240Fps30, 
                         ref depthStreamPixel,
                         Kinect.ColorImageFormat.RGB, 
                         ref mappedDepthLocations );
                 }
         }

This is giving me the following error: EntryPointNotFoundException.

After double checking the Kinect SDK reference ( http://msdn.microsoft.com/en-us/library/jj883690.aspx ), Ive seen that the class defined is within the CoordinateMapper class, which is not defined in my project, but should it really be?

If thats not the problem, what could it be? Im kinda lost here, is the first time Im working with these DllImportAttribute methods.

Besides trying the above, I can also calibrate it myself by multiplying Projection and Modelview Matrixes, but that would be a bummer.

Is there any other way to achieve the same result?

Thank you all for the help!!