koobas.hobune.stream
Get mouse click world coordinates "through" UI Raw Image with Render Texture from second camera
Greetings! I've seen a lot of similar questions posted for this, but I'm still struggling so thought I would post my own question.
I have a Render Texture that I am using as the Target Texture for an offscreen second camera. I am using a Canvas to render a UI menu, including a game object with a Raw Image using the aforementioned Render Texture to display the second camera's output in the UI.
I would like to be able to click on the UI Raw Image and get the mouse click coordinates in world space for where that click should be in relation to what was visible through the second camera.
I've tried using the UI EventSystem to get the event data for the mouse click, but I am not quite sure what to do with it next.
using UnityEngine;
using UnityEngine.EventSystems;
public class UICameraPanel : MonoBehaviour, IPointerClickHandler
{
public void OnPointerClick(PointerEventData eventData)
{
// need to do something with eventData
}
}
My main and secondary cameras are both orthographic, if that matters. Any and all suggestions very much welcome! I have been banging my head against the wall on this one for longer than I'd care to admit.
I was able to come up with a solution that gets the correct point within a few pixels, which is accurate enough for what I am trying to accomplish. Maybe someone else will find this useful in the future.
I start by getting the local coordinates of my render texture's rect
Vector2 localClick;
RectTransformUtility.ScreenPointToLocalPointInRectangle(textureRectTransform, eventData.position, Camera.main, out localClick);
I haven't looked into the reason yet, but it seems my render texture's Y axis is reversed (yMax is 0 and yMin is negative), so I un-reverse this effect on my local click coordinates
localClick.y = (textureRectTransform.rect.yMin * -1) - (localClick.y * -1);
Next, I manually normalize my local rect coordinates to 0-1 viewport coordinates (correcting for the reverse Y axis again)
Vector2 viewportClick = new Vector2(localClick.x / textureRectTransform.rect.xMax, localClick.y / (textureRectTransform.rect.yMin * -1));
Finally, I use the second camera to convert the viewport coordinates to world coordinates
Vector2 worldClick = designerCamera.ViewportToWorldPoint(viewportClick);
While testing this out, I was spawning a 1x1 pink pixel sprite prefab at the world coordinates to visualize how close it looked to the UI click. It's still not perfect, but close enough for my current requirements
GameObject clickPoint = Instantiate(pinkPixel);
clickPoint.transform.position = worldClick;
Was this for a 2D game? Or did you use Vector2 because of your orthographic cameras? How could you get the Vector3 coordinates?
Hey bro you can just use new Vector3(worldClick.x, 0, worldClick.y) if your map is flat or you want to order a Nav$$anonymous$$eshAgent
Hi bro. Could you share please the entire script ? Not really sure how are you getting textureRectTransform, eventData.position
I just came across this post and made my own version based on the answer that @SleepingRobot gave, so here it is:
using UnityEngine;
using UnityEngine.EventSystems;
public class CameraScreen : $$anonymous$$onoBehaviour, IPointerClickHandler {
public GameObject pinkPixel;
public Camera playerCamera;
private RectTransform _screenRectTransform;
private void Awake() {
_screenRectTransform = GetComponent<RectTransform>();
}
public void OnPointerClick(PointerEventData eventData) {
RectTransformUtility.ScreenPointToLocalPointInRectangle(_screenRectTransform, eventData.position, null, out Vector2 localClick);
localClick.y = (_screenRectTransform.rect.y$$anonymous$$in * -1) - (localClick.y * -1);
Vector2 viewportClick = new Vector2(localClick.x / _screenRectTransform.rect.x$$anonymous$$ax, localClick.y / (_screenRectTransform.rect.y$$anonymous$$in * -1));
Ray ray = playerCamera.ViewportPointToRay(new Vector3(viewportClick.x, viewportClick.y, 0));
if (Physics.Raycast(ray, out RaycastHit hit)) {
GameObject clickPoint = Instantiate(pinkPixel);
clickPoint.transform.position = hit.point;
}
}
}
$$anonymous$$y scenario was that I have my main camera rendering to a render texture that I display in a raw image in my UI and I wanted clicks on that raw image to tranfer into the game world. This seems to work perfectly from what I can tell.
