koobas.hobune.stream
RenderTexture from camera viewpoint using a plane as a "window"?
Hi all,
I'm creating a Kinect simulation that needs to adjust what's on screen based on the user's head position/rotation. I've got the Kinect part worked out, and a camera is moving and rotating correctly. The problem is, I want the screen to show a fixed position only changing the perspective based on the camera position.
Basically, I want the user to be looking into a box, with an opening exactly the size of the monitor resolution. When the user moves their head to look around, the perspective of the screen image changes to match, but the view does not ever show the outside of the box, only the inside. This will create an effect that the monitor is the box opening and will seem as though the monitor has depth.
My idea is to use the existing camera that is tracking head movement/rotation correctly as a RenderTexture source, along with a plane that is the exact size of the window resolution. The plane would serve as a "window" that the camera is looking through and that window view would replace the screen draw buffer.
Unfortunately, I'm only familiar with the basics of RenderTextures and thus I cannot figure out how to implement something like this. Any ideas?
![diagram][1] [1]: /storage/temp/144722-2019-08-16-15-35-08.png
Solved!
Took this code and applied it to my setup. Works perfectly.
void Update() {
cameraController.SetCameraGroupRotation(targetWindow.rotation);
if (dynamicWidth) {
float newWidth = targetWindow.localScale.y * cameraController.GetAspectRatio();
targetWindow.localScale = new Vector3(newWidth, targetWindow.localScale.y, targetWindow.localScale.z);
}
}
void LateUpdate() {
Vector3 pa, pb, pc, pd;
pa = targetWindow.TransformPoint(new Vector3(-.5f, -.5f, 0)); //Bottom-Left
pb = targetWindow.TransformPoint(new Vector3( .5f, -.5f, 0)); //Bottom-Right
pc = targetWindow.TransformPoint(new Vector3(-.5f, .5f, 0)); //Top-Left
pd = targetWindow.TransformPoint(new Vector3( .5f, .5f, 0)); //Top-Right
Vector3 pe = cameraController.GetCameraGroup().position; // eye position
Vector3 vr = (pb - pa).normalized; // right axis of screen
Vector3 vu = (pc - pa).normalized; // up axis of screen
Vector3 vn = Vector3.Cross(vr, vu).normalized; // normal vector of screen
Vector3 va = pa - pe; // from pe to pa
Vector3 vb = pb - pe; // from pe to pb
Vector3 vc = pc - pe; // from pe to pc
Vector3 vd = pd - pe; // from pe to pd
float n = cameraController.GetCameraGroup().InverseTransformPoint(targetWindow.position).z; // distance to the near clip plane (screen)
float f = cameraController.GetFarClipPlane(); // distance of far clipping plane
float d = Vector3.Dot(va, vn); // distance from eye to screen
float l = Vector3.Dot(vr, va) * n / d; // distance to left screen edge from the 'center'
float r = Vector3.Dot(vr, vb) * n / d; // distance to right screen edge from 'center'
float b = Vector3.Dot(vu, va) * n / d; // distance to bottom screen edge from 'center'
float t = Vector3.Dot(vu, vc) * n / d; // distance to top screen edge from 'center'
Matrix4x4 p = new Matrix4x4(); // Projection matrix
p[0, 0] = 2.0f * n / (r - l);
p[0, 2] = (r + l) / (r - l);
p[1, 1] = 2.0f * n / (t - b);
p[1, 2] = (t + b) / (t - b);
p[2, 2] = (f + n) / (n - f);
p[2, 3] = 2.0f * f * n / (n - f);
p[3, 2] = -1.0f;
cameraController.SetProjectionMatrix(p); // Assign matrix to camera
if (drawFrustrumToWindow) { //Draw lines from the camera to the corners f the screen
Debug.DrawRay(pe, va, Color.blue);
Debug.DrawRay(pe, vb, Color.blue);
Debug.DrawRay(pe, vc, Color.blue);
Debug.DrawRay(pe, vd, Color.blue);
}
}
cameraController is my own custom class that manages multiple cameras (for post effects, etc).
targetWindow is a generic quad object that serves as the physical monitor. Place the quad where you want your "window" view.
I have the cameraController position being updated by the Kinect in a different class, but you don't need that to make this work.
