koobas.hobune.stream
Raycasting with Lense Distortion
I'm having an issue with raycasting while using Unity's Lens Distortion image effect.
Raycasts lose accuracy because they use the camera's projection matrix without accounting for the distortion of the image effect. I tried to find the matrix or algorithm used for the distortion but I haven't found anything.
Does any know what algorithm is used and how to modify the raycasting logic to account for the distortion?
Documentation about the effect is here: https://docs.unity3d.com/Packages/com.unity.postprocessing@2.0/api/UnityEngine.Rendering.PostProcessing.LensDistortion.html
Hello,
I'm facing the same problem.
I have 2 questions:
Is settings a LensDistortion? And how it work? [SerializeField] private LensDistortion settings;
What is the variable half?
Thanks in advance,
Cheers.
oh sorry my bad. It's new Vector2(.5f, .5f) and yes, the variable settings is of type LensDistortion
You can use this to get LensDistortion object from a PostProcessVolume reference:
public PostProcessVolume postProcessVolumeForRaycast;
public LensDistortion lensDistortionSettings => postProcessVolumeForRaycast.sharedProfile.GetSetting<LensDistortion>();
Ok after some research I was able to solve my problem.
The source of the lens distortion shader is here
and its C# instructions are here
I ended up translating the shader code to C# so I could perform the same distortions on my mouse position. Here is my code:
public Ray RaycastWithDistortion(Vector2 screenPosition)
{
if (settings.active)
{
return cam.ViewportPointToRay(DistortAndNormalizeScreenPosition(screenPosition));
}
return cam.ScreenPointToRay(screenPosition);
}
Vector2 half = new Vector2(.5f, .5f);
public Vector2 DistortAndNormalizeScreenPosition(Vector2 screenPosition)
{
return DistortUV(cam.ScreenToViewportPoint(screenPosition));
}
Vector2 DistortUV(Vector2 uv)
{
float amount = 1.6f * Mathf.Max(Mathf.Abs(settings.intensity.value), 1f);
float theta = Mathf.Deg2Rad * Mathf.Min(160f, amount);
float sigma = 2f * Mathf.Tan(theta * 0.5f);
Vector4 distortionAmount = new Vector4(settings.intensity.value >= 0f ? theta : 1f / theta, sigma, 1f / settings.scale.value, settings.intensity.value);
uv = ((uv - half) * distortionAmount.z) + half;
Vector2 distortionCentre = new Vector2(settings.centerX.value, settings.centerY.value);
Vector2 distortionScale = new Vector2(Mathf.Max(settings.intensityX.value, 1e-4f), Mathf.Max(settings.intensityY.value, 1e-4f));
Vector2 ruv = distortionScale * (uv - half - distortionCentre);
float ru = ruv.magnitude;
if (distortionAmount.w > 0)
{
ru = Mathf.Tan(ru * distortionAmount.x) / (ru * distortionAmount.y);
}
else
{
ru = distortionAmount.x * Mathf.Atan(ru * distortionAmount.y) / ru;
}
uv = uv + (ruv * (ru - 1f));
return uv;
}
There may be an error here. After it didn't work for me initially (it was doing a distortion, but it was not the same as on the LensDistortion), I looked at the source code and in your interpreted version at line 16 there is distortionScale which uses x and y rather than z and w. Basically I think maybe distortionCenter and distortionScale are flipped.
I rewrote using the links you provided and then it seemed to be fixed (but please read on...).
I discovered another error from my part though, where in the Post-process Volume component, some of the options (namely CenterX and CenterY) were turned off, so they were rendered as zeros, but they had non-zero values, which meant this script used those values. Same for X and Y multiplier, in my case Y was deactivated, which means it was using the value from X (1 in my case). When setting the Y value explicitly to 1 and turning it on, your code kicked in and it was the same as the rendered version.
So basically for anyone else reading, make sure you set all the values explicitly in your post process component, else this uses the disabled values, not the default.
The answer by @anaswattar got me close to what I needed, but I did some more digging and found that the formula is changed a little between PostProcessing V2 and the version I was using (Universal Render Pipeline 12.1.6)
The location of the code changed too:
The C# code can be found at
com.unity.render-pipelines.universal@12.1.6\Runtime\Passes\PostProcessPass.csThe shader code can be found at
com.unity.render-pipelines.universal@12.1.6\Shaders\PostProcessing\UberPost.shader
I also converted the code a little differently than the other answer:
float intensity = 0.5f, scale = 1f;
Vector2 lensCenter = Vector2.one * 0.5f;
Vector2 lensXYMult = Vector2.one;
Vector2 half = Vector2.one * 0.5f;
float amount = 1.6f * Mathf.Max(Mathf.Abs(intensity * 100f), 1f);
float theta = Mathf.Deg2Rad * Mathf.Min(160f, amount);
float sigma = 2f * Mathf.Tan(theta * 0.5f);
Vector2 center = lensCenter * 2f - Vector2.one;
Vector4 p1 = new Vector4(
center.x,
center.y,
Mathf.Max(lensXYMult.x, 1e-4f),
Mathf.Max(lensXYMult.y, 1e-4f)
);
Vector4 p2 = new Vector4(
intensity >= 0f ? theta : 1f / theta,
sigma,
1f / scale,
intensity * 100f
);
Vector2 uv = (_uv - half) * p2.z + half;
Vector2 distAxis = new Vector2(p1.z, p1.w);
Vector2 distCenter = new Vector2(p1.x, p1.y);
Vector2 ruv = distAxis * (uv - half - distCenter);
float ru = ruv.magnitude;
if (p2.w > 0.0)
{
float wu = ru * p2.x;
ru = Mathf.Tan(wu) * (1f/(ru * p2.y));
uv = uv + ruv * (ru - 1f);
}
else
{
ru = (1f/ru) * p2.x * Mathf.Atan(ru * p2.y);
uv = uv + ruv * (ru - 1f);
}
return uv;
Ultimately this helped me do the kind of raycasting I needed to do. In my case I don't need to worry about the distortion changing, so you could hook up those variables to read information from the post process settings. Hope that my digging saves someone else some time.
