Physics in non-inertial reference frame resulting in jitter movement

I have attached camera to a rigidbody which is being moved by a player by adding forces (it's a rocket ship). Some of my physics in the game is simulated relative to this rocket. An example: I want other rigidbody (alien ship) to have a constant velocity (more explicitly only velocity.y) relative to this rocket, even if the rocket itself is accelerating / slowing down in the meantime.

I have tried to set the alien ship's velocity constant and set its parent transform to follow the rocket transform in "y" direction. However I am unable to get rid of the very bad jitter. I have tried every combination of Update / LateUpdate / FixedUpdate and interpolation for camera follow, rocket, alien ship and parent transform of alien ship, but the result is always bad (in the editor and standalone build too).

Is there some way how to make this work? Or is there different way to simulate physics relative to accelerating frame of reference?

Relevant code I used:

Camera:

 public class Camera : MonoBehaviour {
     Vector3 offset;
     public GameObject rocket; 
 
     void Start () {
         offset = transform.position - rocket.transform.position;
     }
     void LateUpdate () {
         transform.position = rocket.transform.position + offset;
     }
 }

Rocket:

 public class Rocket : MonoBehaviour {
         bool _AKey = false;
         bool _DKey = false;
         bool _WKey = false;
         Rigidbody r;
 
     void Start () {
         r = GetComponent<Rigidbody>();
     }
 
     void Update () {
         if (Input.GetKey(KeyCode.A)) { _AKey = true;  }
         else { _AKey = false; }
 
        if (Input.GetKey(KeyCode.D)) { _DKey = true; }
        else { _DKey = false; }
 
         if (Input.GetKey(KeyCode.W)) { _WKey = true;}
         else { _WKey = false; }
     }
     private void FixedUpdate()
     {
         if (_AKey) { r.AddForce(-20, 0, 0); }
         if (_DKey) { r.AddForce(20, 0, 0); }
         if (_WKey) { r.AddForce(0, 20, 0); }       
     }
 }

And transform parent of alien ship (non-kinematic rigidbody with constant velocity) is just doing this:

  private void Update()
 {
     transform.position = new Vector3(0, rocket.transform.position.y, 0);
 }