Rolling Ball Physics Conundrum

I'm having an issue with my physics in a game where a ball has force added to it and results in a roll. To be more specific, I have 2 issues. 1.When the ball is moving, it often skips up in the air or downright bounces backwards as if it hit a wall when going between two planes.(which are perfectly level and each have a box collider) 2.when the ball is shot at a higher speed, it can sometimes sneak through walls.

Now here's where the two become a big problem: when I edit physics properties in the game engine, I am able to fully remove the skipping or hitting effect when the ball is moving from one collider to another. However, this causes more clipping through walls.

HOWEVER: to counteract this, i decrease the physics timestep to 0.01, which still gives occasional clipping, but more or less fixes the clipping problem.

I am aware that i could decrease the timestep further to fully fix clipping, but I'm trying to make a mobile application so I have the feeling that 100 physics updates a second won't go over too smoothly, especially if there are other things going on in the scene as well.

Is there a way that i can get the best of both worlds, that is have a time step above 0.15 and still see no clipping or skipping over planes? I have also tried slightly increasing the sphere collider of the ball to decrease the chances of clipping and the max rotational velocity is currently about 1000. Additionally, the ball doesn't have to be moving that quickly either to escape when it clips