Jittery Rigidbody

Hello!

I'm working on a 3D endless runner and I'm having a jitter related issue with my character's rigidbody. He is moving with the same velocity on the Z axis, but it has a lot of jitter:

Video showing the jitter - you can see the jitter better when jumping, but happens when just running as well most of the time.

Here is the setup I have in my project (settings that directly affect physics):

And here is how I control my character (script attached to the Player object, that contains the rigidbody):

 void FixedUpdate()
 {
  // I want the player to maintain this velocity, since he can run over ramps that would slow him down.
   m_rigidbody.velocity = new Vector3(_rigidbody.velocity.x, _rigidbody.velocity.y, speed);
 }

Other than this I have a simple Raycast in the LateUpdate() function to check for grounded status.

  if (Physics.Raycast(_rayToGround, out _groundRaycastHit, _distanceToCheckGround, distanceCheckMask))
 {
   // a few lines of simple code here to manage the state of the m_grounded boolean
 }

For jumping I use:

 public void Jump(float strength)
 {
      m_rigidbody.AddForce(_transform.up * strength, ForceMode.Impulse);
 }

The camera follows the Player smoothly, using a Lerp function and a reasonable speed, so it is not rigidly fixed on the target.

 void LateUpdate()
 {
        m_transform.position = Vector3.Lerp(m_transform.position, target.position + offsets, Time.deltaTime * speed);
 }

No other expensive code whatsoever in my project so far.

It's worth mentioning that in Editor there is almost no jitter (I'm thinking because of better performance, I run Unity on an octacore AMD processor 4 GHz, 32gb ram and Nvidia GTX 970 video card). The device I test on (also recorded the short video) is Huawei P9 Plus (64 GB). There shouldn't be any performance issues since I setup everything in an optimal way (no lights in the scene and only custom Unlit shaders optimized for mobile platforms, all textures compressed properly, the code I write always follows strict guidelines and industry standards, no memory leaks or other issues like that).

Does anyone know what could cause all this?

Thank you!