Train simulator: When train goes at high speeds it starts jumping

I'm trying to do a train simulator with physics so for this I have this setup:

• 4 wheels attached to a bogie with a hinge joint, all 4 have sphere colliders and rigidbodys, so to prevent falling from tracks I have a Raycaster that hits the tracks and applies a force to the bogie so that it doesn't fall off the tracks (keeps it centered).

• 2 Bogies attached to the main body with a hinge joint, all with their own rigidbody and colliders.

• Main body with its own collider and rigidbody

• Tracks with mesh collider

Now the system itself works, it can do closed curves without derailing at a very low speed, but as soon as I go a little bit faster, it starts jumping for no reason (on straight tracks and also curve ones) and derails, making it impossible to go fast or even at a normal speed.

Any help is appreciated.