RotateTowards seems to have non smooth steps

I have the following code which takes 2 objects in the game and finds the vector from A to B and then rotates the forward angle of A towards B. This works well but it appears to have non smooth calculation. For example if I grab object B and slide it slowly, the computed angle to look at does not move until enough distance has gone by and then it jumps to the exact right angle. So its kind of like its unable to do fine grained float calculations. Anyone know what I am doing wrong? Note I am only posting the part that exhibits the error, Debuging out the output vector shows the symptom which is the output stays fixed at the same value until enough distance has changed and then it jumps to the next correct value.

             Vector3 leadPosition = target.position;
             float distance = Vector3.Distance(leadPosition, transform.position);
             Vector3 targetDir = leadPosition - transform.position;
             float step = turnSpeed * Time.fixedDeltaTime;
             Vector3 newDir = Vector3.RotateTowards(transform.forward, targetDir, step, 0.0F);

Example output, while things are at rest its printing this

target dir=1.045,128.3109,13.74569 newDir=0.009199157,0.9942687,0.1065141 Then I slide the target along the x axis and you can see the target dir updates but the newDir does not target dir=1.18,128.3109,13.74569 newDir=0.009199157,0.9942687,0.1065141 Then I keep sliding it a bit more until it "clicks" and updates the output angle target dir=1.768,128.3109,13.74569 newDir=0.01368735,0.9942176,0.1065086