slerp transform rotation is moving the gameobject aswell

Here's my script I found on Unity Answers, and everything works as intended. However my gameobject that's rotating to look at certain targets, slowly moves torwards the target by the rotationspeed I set. I'm able to stop it doing so by freezing the rigidbody position, but I feel this will cause problems with m,y actual character controller when i port this over to my actual project.

IMGUR GIF

using UnityEngine;

  public class SlerpToLookAt: MonoBehaviour
  {
      //values that will be set in the Inspector
      public Transform Target;
      public float RotationSpeed;
  
      //values for internal use
      private Quaternion _lookRotation;
      private Vector3 _direction;
      
      // Update is called once per frame
      void Update()
      {
          //find the vector pointing from our position to the target
          _direction = (Target.position - transform.position).normalized;
  
          //create the rotation we need to be in to look at the target
          _lookRotation = Quaternion.LookRotation(_direction);
  
          //rotate us over time according to speed until we are in the required rotation
          transform.rotation = Quaternion.Slerp(transform.rotation, _lookRotation, Time.deltaTime * RotationSpeed);
      }
  }