Means of Forcing a Rigidbody rotation to freeze.

Alright, so I decided to work on a project which involves manipulating the X and Y but not the Z rotation of an object and adding torque to those as needed. It was all going well until I set up a course and despite Freeze Rotation being checked, sure enough when I hit a wall all of the rotations got fiddled with. The X and Y is fine, but the will cause massive issues with control.

So I decided to script it in with

         gameObject.transform.rotation = new Quaternion(gameObject.transform.rotation.x, gameObject.transform.rotation.y, 0f, gameObject.transform.rotation.w);
 

But despite this ocurring in LateUpdate() it never corrects to 0, even if I manually edit it in the scene to 0 it starts straying.

Unfortunately for the project I want to work on, having physics based movement (as opposed to using character controllers and emulating physics. Is there any means of getting this to work? I noticed some other questions with a similar complaint (though fixed by 0ing out the rotation in question.

Thanks for your time.