Calculation behind camera.WorldToScreenPoint

Question

Hello all,

Is there any way to map world point to screen point using matrix, which output value same as one get when using Camera.main.WorldToScreenPoint function?

Answer 1

Best Answer

Answer by zach-r-d · Jul 25, 2015 at 12:30 PM

Unfortunately, this is mathematically impossible. One of the operations involved in computing the screen point of a world coordinate (dehomogenizing a homogeneous coordinate) cannot be represented as a linear transformation, so there cannot exist a matrix (which is really just a convenient encoding of a linear map) that can represent it.

If you're curious how the function actually works, it probably looks something like this:

 Vector3 manualWorldToScreenPoint(Vector3 wp) {
         // calculate view-projection matrix
         Matrix4x4 mat = cam.projectionMatrix * cam.worldToCameraMatrix;
 
         // multiply world point by VP matrix
         Vector4 temp = mat * new Vector4(wp.x, wp.y, wp.z, 1f);
 
         if (temp.w == 0f) {
             // point is exactly on camera focus point, screen point is undefined
             // unity handles this by returning 0,0,0
             return Vector3.zero;
         } else {
             // convert x and y from clip space to window coordinates
             temp.x = (temp.x/temp.w + 1f)*.5f * cam.pixelWidth;
             temp.y = (temp.y/temp.w + 1f)*.5f * cam.pixelHeight;
             return new Vector3(temp.x, temp.y, wp.z);
         }
     }

Add comment · Show 12 · Share

amit_nath30 · Jul 25, 2015 at 12:36 PM 1

Share

This is exactly what i was looking for. thank you

zach-r-d · Jul 25, 2015 at 01:42 PM 0

Share

Glad it helped! Remember to mark an answer as accepted if it answers your question. :)

Phylliida · May 07, 2017 at 10:21 PM 0

Share

How do you invert this? (i.e., what is manualScreenToWorldPoint)

JanSenseVR · Nov 13, 2019 at 12:41 PM 0

Share

Inverted function - $$anonymous$$anualScreenToWorldPoint

To get the required matricies you can pass in camera.cameraToWorld$$anonymous$$atrix and camera.projection$$anonymous$$atrix.inverse.

Hope this helps someone someday :)

   public static Vector3 $$anonymous$$anualScreenToWorldPoint(Vector2 screenPoint, float distance, $$anonymous$$atrix4x4 cameraToWorld$$anonymous$$atrix, $$anonymous$$atrix4x4 projection$$anonymous$$atrixInverse) {
     // here we are converting screen point in screen space to camera space point placed on a plane "distance" away from the camera
     // screen point is in range [(0,0) - (Screen.Width, Screen.Height)]
     Vector2 pointViewportSpace = screenPoint / new Vector2(Screen.width, Screen.height); // convert space [(0,0) - (Screen.Width, Screen.Height)] to [(0,0) - (1,1)]
     Vector2 pointCameraSpaceNormalized = (pointViewportSpace * 2.0f) - Vector2.one; // convert space [(0,0) - (1,1)] to [(-1,-1) - (1,1)]
     Vector2 pointCameraSpace = pointCameraSpaceNormalized * distance; // convert space [(-1,-1) - (1,1)] to [(-dist,-dist) - (dist, dist)]
     Vector4 planePoint = new Vector4(pointCameraSpace.x, pointCameraSpace.y, distance, distance); // define the point (don't know why z and w components need to be set to distance)
 
     // calculate convertion matrix from camera space to world space
     $$anonymous$$atrix4x4 matrix = cameraToWorld$$anonymous$$atrix * projection$$anonymous$$atrixInverse;
     // multiply world point by VP matrix
     Vector4 worldPoint = matrix * planePoint;
 
     return worldPoint;
   }

cyz2727327 JanSenseVR · Sep 22, 2021 at 06:50 PM 0

Share

Thank you so much JanSenseVR, this works!!! I have been trying to find this for so LONG

JanSenseVR · Nov 13, 2019 at 12:53 PM 0

Share

The convertion other way around could look something like this. Tested and works without issues.

$$anonymous$$anualScreenPointToWorldPoint function

   public static Vector3 $$anonymous$$anualScreenPointToWorldPoint(Vector2 screenPoint, float distance, $$anonymous$$atrix4x4 cameraToWorld$$anonymous$$atrix, $$anonymous$$atrix4x4 projection$$anonymous$$atrixInverse) {
     // here we are converting screen point in screen space to camera space point placed on a plane "distance" away from the camera
     // screen point is in range [(0,0) - (Screen.Width, Screen.Height)]
     Vector2 pointViewportSpace = screenPoint / new Vector2(Screen.width, Screen.height); // convert space [(0,0) - (Screen.Width, Screen.Height)] to [(0,0) - (1,1)]
     Vector2 pointCameraSpaceNormalized = (pointViewportSpace * 2.0f) - Vector2.one; // convert space [(0,0) - (1,1)] to [(-1,-1) - (1,1)]
     Vector2 pointCameraSpace = pointCameraSpaceNormalized * distance; // convert space [(-1,-1) - (1,1)] to [(-dist,-dist) - (dist, dist)]
     Vector4 planePoint = new Vector4(pointCameraSpace.x, pointCameraSpace.y, distance, distance); // define the point (don't know why z and w components need to be set to distance)
 
     // calculate convertion matrix from camera space to world space
     $$anonymous$$atrix4x4 matrix = cameraToWorld$$anonymous$$atrix * projection$$anonymous$$atrixInverse;
     // multiply world point by VP matrix
     Vector4 worldPoint = matrix * planePoint;
 
     return worldPoint;
   }

Example usage (place on the camera gameobject and play around with screen point and distance fields)

 using UnityEngine;
 
 public class Draw$$anonymous$$anualScreenPointToWorldPoint : $$anonymous$$onoBehaviour {
   Camera cam;
 
   [SerializeField]
   float distance = 1000;
   [SerializeField]
   Vector2 screenPoint = Vector2.zero;
 
   void Start() {
     cam = GetComponent<Camera>();
   }
 
   void Update() {
     var point = $$anonymous$$yUtilityClass.$$anonymous$$anualScreenPointToWorldPoint(
       screenPoint,
       distance,
       cam.cameraToWorld$$anonymous$$atrix,
       cam.projection$$anonymous$$atrix.inverse);
 
     Debug.DrawLine(cam.transform.position, point, Color.green);
   }
 }

Show more comments

Answer 2

Answer by cyz2727327 · Sep 22, 2021 at 06:49 PM

I can confirm JanSenseVR's code works, thank you so much I have been searching for this for so LONG !!! @JanSenseVR

Add comment · Share

Calculation behind camera.WorldToScreenPoint

2 Replies

Your answer

Follow this Question

Related Questions