Jerky motion using moveTowards at high speed

I have two objects having a slightly different maxSpeed parameter and a camera attached to one of them. I am using the following script to move them:

 using UnityEngine;
 using System.Collections;
 
 public class Character : MonoBehaviour {
 
     public float speed = 0f;
     public float maxSpeed = 0f;
     
     protected int pathIndex = 0;
     protected int maxTrackIndex = 0;
     
     protected Vector3 targetDirection;
     protected Vector3 targetPosition;
     protected GameController gameController;
 
         void Start () {
         gameController = GameObject.FindGameObjectWithTag ("Scripts").GetComponent<GameController> ();
         transform.position = gameController.GetCheckpointAtIndex (0);
         targetDirection = gameController.GetCheckpointAtIndex (0);
         targetPosition = transform.position + targetDirection;
         maxTrackIndex = gameController.GetMaxIndexOfTrack ();
         StartCoroutine ("StartMovingMechanic");
     }
 
     protected IEnumerator StartMovingMechanic () {
         while (true) {
             if (Vector3.Distance (transform.position, targetPosition) < 0.1f) {
                 if (pathIndex < maxTrackIndex) {
                     pathIndex ++;
                 } else {
                     pathIndex = 0;
                 }
                 
                 targetDirection = gameController.GetCheckpointAtIndex (pathIndex);
                 targetPosition += targetDirection;
             }
             
             transform.position = Vector3.MoveTowards(transform.position, targetPosition, speed * Time.fixedDeltaTime);
             yield return null;
         }
     }
 
     void Update () {
                if (Input.GetKeyDown (KeyCode.UpArrow)) {
             StartCoroutine ("Throttle");
             StopCoroutine ("Decelerate");
         } 
         if (Input.GetKeyUp (KeyCode.UpArrow)) {
             StopCoroutine ("Throttle");
             StartCoroutine ("Decelerate");
         } 
     }
 
         IEnumerator Throttle () {
         while (speed < maxSpeed) {
             speed = Mathf.Lerp (speed, maxSpeed, Time.deltaTime * 1);
             yield return null;
         }
     }
 
     IEnumerator Decelerate () {
         while (speed > 0.05f) {
             speed = Mathf.Lerp (speed, 0, Time.deltaTime * 1);
             yield return null;
         }
     }
 }
 

I am using a list defined in the game controller that provides the checkpoints which is referenced in Start. Everytime the objects reach close to their destination, they get the next checkpoint.

What I observe is that in the game view, the second object with a slightly lower max speed is jerky when the speed gets higher. It's not consistent and to me it looks random. I am really unsure on how to tackle this. In the Scene View, everything looks ok (or I can't tell, because it's fast).

I am really not sure if this is the right way to go. I don't want to use rigid bodies at the moment, so I am using transforms. I tried searching for the issue, but I really can't figure out what to even search for. The jerkyness looks a lot like a frame or two is skipped. This is a very simple Scene with not a lot of objects/scripts.

Any ideas?

Thank you!