Hinge Joints Ignoring Angular Velocity?

I've got a hinge joint that I'm trying to set the initial angular velocity of through a script, however, it seems that the joints themselves do not appropriately take in account the angular velocity of the attached rigidbody. Now I'm not using large numbers and going out beyond the angular velocity limitation; I'm just seeing a value set as the angular velocity on the z axis going from 2 (what I set it to) immediately to 0.5.

Here's a simple script you can just apply to a standard HingeJoint (configured along the Forward or Z axis) to get the right idea of what I'm talking about.

 void FixedUpdate () {
     Debug.Log(GetComponent<Rigidbody>().angularVelocity.z);
     GetComponent<Rigidbody>().angularVelocity = new Vector3(0, 0, -2);
 }

The HingeJoint is attached to a kinematic rigidbody anchor while the rigidbody on the HingeJoint is set to be non-kinematic and to not use gravity (to take it out of the equation). What will happen is I get the following values printed out when we first start: 0, -0.5054573, -0.8830371, -1.165025, -1.375587, ... (until we hit about -2 half a second later).

As you can see there's a bit of a lead up until we actually hit that -2 velocity on the Z axis. This will absolutely not work with what I'm trying to use these joints for. What I'm expecting is to be able to throw an initial velocity on the joint then let the simulation handle the rest. It also doesn't matter if I set the angularVelocity directly or if I use ApplyTorque with the ForceMode VelocityChange.

Am I doing something wrong? Is there a "correct" way to apply an initial angular velocity to a joint? (FYI I've also tried just doing a configurable joint with the same results)