The question is answered, right answer was accepted
Obtaining a depth Point Cloud
Hi,
I've been trying to extract a point cloud from Unity which has depth information for a while now, but have had some trouble. I found some links online to help me get started and I feel as if I'm almost there, but there is slight problem - what should be straight lines appear to be curved. I was hoping someone could help me locate the issue / point me in the right direction.
To start off, to obtain the depth information, I used this approach: http://answers.unity3d.com/questions/877170/render-scene-depth-to-a-texture.html
TLDR for the link: The shader and the code produce a greyscale image which represents the depth in the scene, where black is closest to the camera, and white is the farthest. The code and shader used is the same as the ones in the reply of the user who posted this.
Running the code produces this image in Unity which looks proper:
From there, I obtain the render texture in the same way as the link above and store the pixel information in an array. Here is a quick reference:
int resolutionX = Screen.width;
int resolutionY = Screen.height;
RenderTexture tempRt = new RenderTexture(resolutionX, resolutionY, 0);
camera.targetTexture = tempRt;
camera.Render();
RenderTexture.active = tempRt;
Texture2D tex2d = new Texture2D(resolutionX, resolutionY, TextureFormat.ARGB32, false);
tex2d.ReadPixels(new Rect(0, 0, resolutionX, resolutionY), 0, 0);
string[] output = new string[resolutionX * resolutionY];
Color[] pixelInfo = tex2d.GetPixels();
From here, I run the following code on each pixel to attempt to translate it into real x, y, and z coordinates and output it into a file which is to be my point cloud.
float r, g, b;
double x, y, z, fl, range;
double realX, realY, realZ;
// Obtain X and Y Pixel Coordinates
double pixelX = i % resolutionX;
double pixelY = i / resolutionX;
range = pixelInfo[i].r;
x = (pixelX / resolutionX) - 0.5;
y = (-(pixelY - resolutionY) / resolutionY) - 0.5;
fl = -0.5 / (Math.Tan((Fov / 2) * Math.PI / 180));
z = -fl;
double vecLength = Math.Sqrt((x * x) + (y * y) + (z * z));
// r = g = b because we are getting the value from the depth grayscale image
r = (int)(pixelInfo[i].r * 255);
g = (int)(pixelInfo[i].g * 255);
b = (int)(pixelInfo[i].b * 255);
// unitize the vector
x /= vecLength;
y /= vecLength;
z /= vecLength;
// multiply vector components by range to obtain real x, y, z
realX = x * range;
realY = y * range * -1;
realZ = z * range;
When reading the file into Meshlab, all looks more or less okay, EXCEPT my straight lines are not as straight as they should be.
Image of the scene in Meshlab, rotated to show curvature.
As you can see, the rectangle which is a cube in the Unity world is curved when it should be straight and I have no idea why. Any help is appreciated. If anything is unclear I'll gladly clarify. Thanks!
EDIT: I got the answer by posting in the forums. All I had to do was normalize the z component of vector to 1.
I've had a similar issue, any insight will be valuable, thanks.
Follow this Question
Related Questions
writing _CameraDepthTexture values to file 0 Answers
Get UV warped texture as another texture 0 Answers
Receive Shadow Mesh Renderer - Weird Effect 0 Answers
Tilemap Shader Glitch 0 Answers
HDRP and UMMORPG server 1 Answer