koobas.hobune.stream
Matrix Transformation from Local Space to Screen Space
I am creating a drag-selection box that selects an object once all of its vertices are within the selection box (just like in the Unity Scene Editor).
Currently, I am converting the vertices of the mesh to world space, then to camera space.
Vector3 worldVertex = mapObject.transform.TransformPoint(localVertex);
Vector2 screenVertex = Camera.main.WorldToScreenPoint(worldVertex);
But this is too slow when there are many objects on the screen.
I was hoping to create a transformation that can map local space coordinates to screen space directly.
My attempt:
//Calculate the transformation from local space to screen space
Matrix4x4 localToWorld = mapObject.transform.localToWorldMatrix;
Matrix4x4 worldToScreen = Camera.main.worldToCameraMatrix;
Matrix4x4 localToScreen = worldToScreen * localToWorld;
//Apply the transformation to the vertex
Vector2 screenVertex = localToScreen.MultiplyPoint(localVertex);
But from what I read, this doesn't work due to how the worldToCamera matrix is implemented.
Is there a method of converting from local space to screen space more efficiently?
How can the camera perform these calculations so quickly when rendering?
You're almost there with the matrices. There is one additional step when it comes to rendering, and that is the projection matrix (which will be either perspective or orthographic). This is what actually maps things to screen, and so if we multiply by that we will end up with coordinates in viewport space. Then, we can remap those to screen pixel coordinates;
//Cache this so we don't need to find the main camera every time
Camera cam = Camera.main;
Matrix4x4 localToWorld = mapObject.transform.localToWorldMatrix;
Matrix4x4 worldToView = cam.worldToCameraMatrix;
Matrix4x4 projection = cam.projectionMatrix;
//This transforms from object space to clip space, also known as the MVP matrix
Matrix4x4 localToViewport = projection * worldToView * localToWorld;
//This remaps from a range of [-1,1] to a range of [0,resolution] (i.e. screen space)
Matrix4x4 remap = Matrix4x4.TRS (new Vector3 (0.5f * cam.pixelWidth, 0.5f * cam.pixelHeight, 0f), Quaternion.identity, new Vector3 (0.5f * cam.pixelWidth, 0.5f * cam.pixelHeight, 1f));
//Finally, combine the matrices and transform the vertex
Matrix4x4 localToScreen = remap * localToViewport;
Vector2 screenVertex = localToScreen.MultiplyPoint (localVertex);
If you want this to be even faster, then rather than checking each vertex on the mesh, you can instead use the corners of it's bounding box;
//Get the (local space) mesh bounds and its minimum and maximum extents
Bounds bounds = mesh.bounds;
Vector3 max = bounds.max;
Vector3 min = bounds.min;
//The corners are simply all combinations of min and max
Vector3[] localCorners = new Vector3[8];
localCorners[0] = new Vector3 (max.x, max.y, max.z);
localCorners[1] = new Vector3 (min.x, max.y, max.z);
localCorners[2] = new Vector3 (max.x, min.y, max.z);
localCorners[3] = new Vector3 (min.x, min.y, max.z);
localCorners[4] = new Vector3 (max.x, max.y, min.z);
localCorners[5] = new Vector3 (min.x, max.y, min.z);
localCorners[6] = new Vector3 (max.x, min.y, min.z);
localCorners[7] = new Vector3 (min.x, min.y, min.z);
//Then do the same transformation, but with the corners instead
Vector2 screenVectex0 = localToScreen.MultiplyPoint (localCorners[0]);
//etc...
This works perfectly! And its much faster than what I had originally. Thank you. I like the idea of using the 3D bounding box of the object since its orders of magnitude faster. But the 2D bounding box it produces is too inaccurate for my purposes. Ins$$anonymous$$d, I think I will construct my own bounding box from the transformed screen points. If I still have performance issues, I'll try compute shaders.
