koobas.hobune.stream
How to convert scale and aspect ratio of cpu image to match screen size?
Hello everyone,
I use unity with the free opencv package to do some object recognition. For this I purpose I use TryAcquireLatestCpuImage(out XRCpuImage cpuImage). This gives me a 640x480 image which I use for my detection in connection with some opencvsharp code. The problem is, that the rectangle for my detected objects obviously corresponds to the 640x480 texture. So when I visualize the position of my detected object it doesn't line up with the position of what the AR camera shows on screen, which has a resolution of 2200x1080 (Galaxy S7 for instance). When the device is orientated ScreenOrientation.LandscapeRight e.g. I can simply multiply the position of the opencv rectangle with the ratio I get from Screen.width / image.width, which works fine. But in portrait mode the two images somehow differ too much. I found a thread with the same problem: Difference between AR view and cpu image. But there wasn't posted a real solution yet. I read that there is a projectionMatrix for the AR camera which gives a Matrix4x4 and wondered if I could use this to somehow transform the cpu image to get the same image that is displayed on screen? I'm relatively new to this whole stuff, so I hope the discription of my problem is not too confusing.
