- Home /
Projecting a rectangle into screen space
I am trying to implement Doom's Visportal Occlusion system in Unity: https://www.iddevnet.com/doom3/visportals.php
As the link states, the goal is to perform "unions of the screen space rectangles between the areas". However, since the rectangles, or visportals, exist in the 3D world, a method of projecting them into the screen space is necessary, and Camera.WorldToScreenPoint method seemed perfect for it.
So first, I was able to achieve this:
The lines are drawn with GL_LINES, in the screen space by using the WorldToScreenPoint method in each of the visportal's corners.
However, due to how the method works, looking at the portal in different angles makes some corners go behind the camera, producing unwanted results:
In this example, the left bottom corner of the visportal, because it is behind the camera, gets wrapped around, which makes the resulting shape unusable for further calculations.
So how can I bypass this? How can I turn a 3D "Rectangle" in the world and convert it into a 2D version, flattened into screen space?
One solution i can come up with is cutting the rectangle straight by the screen, achieving a shape like this:
http://i.imgur.com/Xn6xDRh.png
But I have no idea how to implement this, especially since rotating the camera makes the conversion tend to infinite....
Any idea?
After receiving such great responses, I decided to add a link for a sample project containing my current implementation. If anybody thinks he can provide with improvements and feedback, feel free!
Link: http://www.filedropper.com/visportal
Basically, each visportal knows both rooms it is linking, and each room knows every visportal it contains.
For now, each room has a trigger collider so that I know where the player is. I do not love this approach, especially since unity's callbacks aren't always reliable, but it will do for now.
Then, for each room where player is, I call a recursive method which continuously performs polygon clipping with the help of the Clipper library (http://www.angusj.com/delphi/clipper.php).
If there is a solution, the method shows the room in which the player is not (so if im in room A and visportal connects A and B, I want to show room B), and recalls itself for each portal of the new visible room.
If you approach a portal and rotate the camera, you will see the problem deriving from Camera.WorldToScreenPoint, where a corner will go to the other side, due to the wrapping.
I hope this can be of use to somebody :)
Answer by Bunny83 · Jul 25, 2016 at 02:17 PM
Vis portals is a quite advanced technology. As you mentioned Doom3 i took a look at the dev-article you've linked and the actual code (since the Doom3 source has been released under GPL).
It's a bit more complicated than the article might suggest ^^. The article is mainly written for mappers to understand what a visportal is.
The actual portals are defined in worldspace and stay in worldspace. It's true that they create a screenspace axis aligned rect around the actual polygon, but it's only used as scissor rect for entities. Since that rect is axis aligned (so it's an actual rectangle unlike the projected portal borders) the scissor clipping is also axis aligned. You can see this in the last picture in the article where the geometry isn't clipped by the actual portal lines but by the enclosing AA-Rect.
What actual happens is they use a class they call "idWinding" which is basically just a collection of vertices that form a convex shape. For each portal they create clipping planes for each edge of the winding. Additionally they clip the winding to the parent portal clipping planes which could increase / decrease the point count of the winding. If the point count is 0 the portal isn't visible.
In addition you should be aware of the fact that portals are "directed". They only work in one direction. So at an intersection of two areas you always need two portals, one that belongs to room A and leads to room B and one that faces the other direction and belongs to room B and leads to A. Each portal also has a "surface plane" so you can do an easy check if you look at the "right" side of a portal. Portals that face in the wrong direction are ignored.
The clipping code is quite simple and straight forward as the shape always has to be convex. cutting a convex polygon in half always leads to no, one or two intersections. There are two different clipping methods Clip and ClipInPlace. Clip creates a new Winding while ClipInPlace cuts down the current.
If you want to explore the source of Doom3 i recommend you install VisualStudio Express for C++ so you can easily lookup certain types / classes / ...
edit
So here we are, finally ^^:
And here's the whole project as ZIP
It's not yet in a state that could be used. There are still some minor epsilon errors (an area might be clipped too early if only a small piece of a corner is in view). Also one major problem is, at least in this example: Shadows!. Even an area can't be seen by the player through any portal, you can still see the shadow which might be casted into a visible area. So when an area is disabled you'll see the shadow is changing. Doom solves this completely different since the VisPortals are integrated into the whole renderer.
Also at the moment there are a few things that should be changed, depending on the requirements:
At the moment an area that isn't visible is simply "deactivated". So the whole section just vanishes. This can cause problems with other things like AI characters or other things. It would be better to just disable all renderers in an area instead erasing it from the scene temporarily.
Along with point 1 the next thing is the way how the current area is determined is a bit, well, "limited". I use one of my polygon shapes to test if we are inside an area. However that polygon is a 2d shape flat on the ground. So it doesn't matter at which height you are, you are always inside the area. The best fix would be to use box colliders to roughly enclose each area, do a Physics.OverlapSphere around the camera and then do the area check for those areas only. That allows you to have areas on top of each other (think about a multiple story building).
Oh, about the WebGL example, you can move around with WASD and you have to hold the right mouse button down to look around. The mouse hiding / locking is a bit odd in WebGL. After you allowed it, it doesn't propertly lock the cursor unless you press a key ^^. It automatically unlocks when you release the mouse button (pretty much like Unity's sceneview movement). You also might want to enable the debug draw button.
The mouse speed is really strange in WebGL. If the canvas is small (say 100x100 pixels) the mouse speed is slow. If you have a large canvas it moves extremly fast. At least in FireFox that's the behaviour i noticed. The actual speed is very low when testing in the editor.
(ps: you can exchange the two views if you like ^^)
Hello, thank you for your detailed answer.
I do not need a system so perfected and complex as doom's, i simply loved the logics behind it and Id love to implement a form of it, even if slightly worse.
Thank you for pointing out doom's source code, I might indeed explore it If I get desperate enough...
Interesting explanation though, I did not understand everything right now, in particular how clipping planes work in worldspace, but I will read with more carefully soon enough, tyvm.
Still, like I said, Id be satisfied with a "mediocre" implementation. By the way, I have attached a sample project containing my current implementation, it is very simple and if you have some time, Id love that you could provide with some feedback for improvements :)
Hey, even though you are already implementing the visportal solution I'd like to understand the theory behind it, when you say "a collection of vertices that form a convex shape"; are these vertices in the 3D world? Because you say it like it's a polygon, but the portal exists in the 3D world, so its not really a polygon, but a "convex shape" in the 3D world right?
So for each edge you define a "Clipping plane", being a plane defined by a normal vector and any point belonging to it (an infinite plane, therefore), and afterwards you perform clipping math to these clipping planes?
Right. So a vis portal itself is in fact a polygon in 3D. All points are on the same plane. That plane can be oriented the way you like. So a visportal basically is a plane (position and normal vector) and inside that plane i define 2d points which form the polygon, usually a rectangle in the beginning. Like always the order of the vertices matters. I followed the usualy left-hand rule which applies to Unity.
For each edge of the polygon (after transfor$$anonymous$$g the 2d shape into 3d) we create a new clipping plane based on the two corners of an edge and the view position (the camera position).
It's a recursive method that runs from area to area. Each portal has a certain destination area. In the beginning we have to deter$$anonymous$$e the area in which we are. If we aren't in any area we simply activate all areas (doom actually does the same. If you "noclip" out of bounds suddenly the whole level will be rendered). To start the recursion we iterate through all portals that are defined inside that area and clip the portal polygon agains the "current set" of clipping planes. For the first area we just use the camera clipping planes but without "near" and "far". So only left, right, top and bottom.
So if a polygon intersects any of the clipping planes it will be cut down by that plane. Everything behind the plane will be removed. So in your first image above nothing would be clipped and you still have the 4 edges of the polygon. When "going through" a portal we simply change the "area" to the area this portal leads to and create a new set of clipping planes, one for each edge of the polygon. It's like a new but smaller "view frustum".
When iterating through the portals of the next room, their polygons will be clipped agains the new clipping planes. So the second portal has to fall into the new frustum and if it intersects this one it gets cut down even further.
If a after clipping a polygon down there are less than 3 points over, the portal is not visible since everything lies outside the clipping planes. This is the case where we simply ignore that portal. Of course portals that doesn't face the viewer don't have to be checked at all. For that i use the "portal plane" and test on which side the viewer position is. That's all quite trivial operations.
The Polygon class has an automatic pool build into since we can't create classes on the stack like you can in C++. So to avoid garbage collection I simply pool them. I also use Lists for most things. Lists can be cleared and reused without any allocation / garbage being created.
Yesterday i fixed the final bugs ^^. It's already working reliable. I'm just cleaning up a bit and adding some debug tools.
@Glurth: Yes, exactly. As i said the article is a bit misleading as it doesn't describe the actual used technic behind the visportals but only explains the concept for map authors so they know how to use them and where to place them.
I'm pretty sure that Doom actually creates the vis areas automatically. However for that you need to find the enclosing geometry to figure out the boundary of the interior. The placed portals finally just splits the whole area into smaller ones.
In older games (like Quake2 / Halflife1) this process was done by one part of the map compiler (VIS.exe). Doom3 seems to do this on the fly as Doom3 directly uses ".map" files ins$$anonymous$$d of compiled ".bsp" files. Apart from the long lighting phase (rad.exe) finding "leaks" in the level was the most anoying part about mapping back then ^^.
Answer by Naphier · Jul 25, 2016 at 08:36 AM
I think what's going on here is your camera's near distance is cutting off the 3D object of the vizportal so it is literally behind the camera. Try changing the near distance down to 0.
Another way you might consider doing this is with an AABB plane for the camera's frustum. You can see an example of the AABB frustum checks here: https://www.3dbuzz.com/training/view/3rd-person-character-system
You could also use scripts on the objects with renderers and use Unity's OnBecameVisible/OnBecameInvisible event (https://docs.unity3d.com/ScriptReference/MonoBehaviour.OnBecameVisible.html) to enable/disable the objects.
But if you're doing this to just optimize occlusion then, Unity should already be handling this for you.
Hey,
Yeah I tried to do that (changing the near distance), it was one of the first things I tried, but it doesn't matter because eventually a point will still go behind the camera.
I did not understand very well what you mean with frustum check, I know I can use Unity's GeometryUtility class to check if an object's collider bounds are inside the view frustum, but I can't do much with just that...
Unity does have frustum culling, but that isn't enough. There is also occlusion culling, but it has to be baked, and my map is procedurally generated, therefore I can't follow conventional methods. Doom's occlusion system interests me because It would actually work very well with my map type (generated by connecting room templates), so I would really like to be able to implement it.
If you take a look at those 3D buzz tutorials you'll see what I mean with the frustum check. In their example they use it to see if the camera can no longer see the player. Basically they draw planes inside the frustum's area and do a few steps to check if something (anything) collides. Starts here at video 11 (https://www.3dbuzz.com/training/view/3rd-person-character-system/enhanced-character-system). In your case you could use this to see if the camera can/should see anything then enable the objects as needed and start predictive generation of the next rooms.
Another simple technique you could do is a have trigger areas set up at the portal mouths and then have a box collider at each side of the frustum (or probably better, just a little outside of the frustum). Then if one of those hits your portal mouth trigger you activate the portal. $$anonymous$$ight be kind of a pain to deactivate something that way though. I've done maps and scene loading with similar techniques where the player has a big box collider out in front of him that will activate new areas, and a big box collider out his back that will deactivate existing areas.
Answer by Glurth · Jul 25, 2016 at 02:57 PM
When using viewspace, it's important to keep in mind, it is NOT a smooth continuous coordinate system like world-space. For example a point at z=0, is technically undefined. (http://answers.unity3d.com/questions/1215981/camera-position-in-viewport-coordinates.html)
Another thing I've noticed is that: if a coordinate is in front of the camera, at view coord (x,y,+z), and is viewed in the top right side of the screen, then (x,y,-z) will be the LOWER_LEFT side of the screen. If you think about the view frustum, and draw a line along one of the corners, through the camera, and out the back, you can see WHY viewspace does this.
You are right, the code I posted before just wasn't working as I thought. I'm starting to suspect there may not be enough information in the Vector3 provided by WorldToViewportFunction, to do this with only 1 point (no "w").
The best I could get working was clipping the line segment at the edge of the screen, in VIEWspace, given two worldspace points.Though I'm not sure if this is actually what you want, or had working already...here is it anyway:
Inputs: P1World,P2World, Camera
Outputs: P1View,P2View (both will have a positive Z > 0)
Vector3 P1View = cam.WorldToViewportPoint(P1world); Vector3 P2View = cam.WorldToViewportPoint(P2world); Vector3 lineViewSpace = P2View - P1View; if (P1View.z < 0) { if (P2View.z < 0) { //don't draw, both out of view return false; } //line eq in parameter form //x = x0 + t(x1 - x0) //y = y0 + t(y1 - y0) //z = z0 + t(z1 - z0) //(a-c0)/cDiff=t (where c is x,y or z) float a = 0; // x-left or y-bottom coord float t; //param used to compute intersection points if (Mathf.Abs(lineViewSpace.x) > Mathf.Abs(lineViewSpace.y))//slope of line is such that it will pass off screen on the left/right side { if (lineViewSpace.x > 0) //if line passes off screen on RIGHT a = 1;// x-right coord t = (a - P1View.x) / lineViewSpace.x; } else//slope of line is such that it will pass off screen on the top/bottom side { if (lineViewSpace.y > 0)//if line passes off screen on TOP a = 1; //y-top coord t = (a - P1View.y) / lineViewSpace.y; } //now that we have t plug into param line form P1View = new Vector3(P1View.x + t * (lineViewSpace.x), P1View.y + t * (lineViewSpace.y), nearDist); } if (P2View.z < 0) { //line eq in parameter form //x = x0 + t(x1 - x0) //y = y0 + t(y1 - y0) //z = z0 + t(z1 - z0) //(a-c0)/cDiff=t (where c is x,y or z) lineViewSpace *= -1; float a = 0; float t; //param used to compute intersection points if (Mathf.Abs(lineViewSpace.x) > Mathf.Abs(lineViewSpace.y))//slope of line is such that it will pass off screen on the left/right side { if (lineViewSpace.x > 0) //if line passes off screen on RIGHT a = 1; t = (a - P2View.x) / lineViewSpace.x; } else//slope of line is such that it will pass off screen on the top/bottom side { if (lineViewSpace.y > 0)//if line passes off screen on TOP a = 1; t = (a - P2View.y) / lineViewSpace.y; } P2View = new Vector3(P2View.x + t * (lineViewSpace.x), P2View.y + t * (lineViewSpace.y), nearDist); }
Hey,
I tried that too, but I can't make it work...
I tried a new thing though. In the project sample I attached, on the methods:
Vector3 p1 = Camera.main.WorldToScreenPoint (visportal.p1);
etc, I first sent visportal.p1 through this method:
private Vector3 calculateWorldPosition(Vector3 position, Camera camera) {
//if the point is behind the camera then project it onto the camera plane
Vector3 camNormal = camera.transform.forward;
Vector3 vectorFromCam = position - camera.transform.position;
float camNormDot = Vector3.Dot (camNormal, vectorFromCam.normalized);
if (camNormDot <= 0f) {
//we are beind the camera, project the position on the camera plane
float camDot = Vector3.Dot (camNormal, vectorFromCam);
Vector3 proj = (camNormal * camDot * 1.01f); //small epsilon to keep the position infront of the camera
position = camera.transform.position + (vectorFromCam - proj);
}
return position;
}
Basically, If the object is behind the camera, we project it onto the camera's plane, with a small epsilon making it so that it is infront of the camera. Then, with Camera.WorldToScreenPoint, this works "better", but it is imperfect and still flawed though...
If you move the object directly along the camera facing direction, in world space, you will be skewing the image. The frustum should ins$$anonymous$$d expand(+x)/contract(-z) like a pyramid in world space.
Ah, on the camera's plane: (assu$$anonymous$$g you mean the near clipping plane) keep in $$anonymous$$d this implies that either your viewspace x or y coordinate will have a value of less than 0 or greater than 1, because it passes out of the fustrum. Or, do you want to get the clipping point right on the edge of the screen, where the x or y is exactly equal to 0 or 1?
I'll add some code to my answer showing what I'm talking about.
If i were to solve the problem by clipping the "first" polygon straight by the camera bounds, yes that would be the case, getting the clipping points on the edge of the screen. I tried your code but I can't seem to make it work tho, and by the way I assumed you made a typo in if (P1View.z > 0), you meant < right?
Your answer
Follow this Question
Related Questions
How to reposition camera so that given plane point is in given screen position? 1 Answer
Proper use of Camera WorldToScreenPoint 1 Answer
Raycasting forward from screen perspective instead of game position. 1 Answer
Indicate with markers the direction of Gameobjects which are outside of the camerafield. 0 Answers