smooth transform look at with space self rotation

 transform.LookAt(vecPoint);
 transform.Rotate(pref.x, pref.y, pref.z, Space.Self);

these two lines in late update allowed me to have bone look at a vector3 point with specified extra rotations for different axis'. (some of my characters bones are not oriented correctly)

I am trying to get them to rotate smoothly because as you know, transform.lookat snaps. So far as I could find, there is only one way to smooth look at.

 var targetRotation = Quaternion.LookRotation(vecPoint - transform.position);
    transform.rotation = Quaternion.Slerp( transform.rotation, targetRotation,  Time.deltaTime * speed);
     transform.Rotate(pref.x, pref.y, pref.z, Space.Self);

This attempt makes the bone flip around super fast, non stop.

How do I write this properly? Thanks