Sphere Friction Not Working Correctly When Bouncing Hard

I am making a marble game and there is an issue that I just can't figure out. When the ball bounces off the ground while spinning, it should bounce a bit in the direction it was spinning. This does happen when bouncing softly, but if the ball were to bounce off the ground hard, this doesn't happen. Spinning the ball has no effect in the bounce direction when bouncing hard. It acts if the ball that is bouncing is frictionless.

On the ball's Rigidbody, the collision detection is set to Continuous Dynamic. And the interpolation is set to interpolate. Changing the interpolation doesn't change anything with this.