How to fix physics penetration so that it doesn't make object "stick" during movement?

Sorry for the vague question it's hard to put in words but let me explain my problem. Basically I have a ball, and a level that I can rotate myself. The level is a collection of boxes with colliders and the gameobject (the level) containing all the boxes has a rigidbody component with IsKinematic set to true since I rotate the level myself. The player (ball) has a sphere collider and a rigidbody component as well. So basically when I rotate my level and the ball gets in a corner, the ball gets stuck in the corner and jitters until I stop rotating, the level isn't rotating that fast (80 degree per second). So I can do a full level rotation and the ball doesn't fall in the middle.

Here's a screenshot of the problem: http://www.evil-dog.com/tempshare/PhysicsProblem.jpg

Now I've played with these settings

Script: Time.fixedDeltaTime

Physics Settings: Min Penetration For Penalty, Solver Iteration Count

RigidBody Properties: Mass, Collision Detection (I set it to Continuous Dynamic and that reduces the problem a lot but the problem is still there)

Physic Materials: Dynamic Friction, Static Friction (I don't think that would do much)

I can't find a combination that fixes my issue, anyone has some more insight into this kind of stuff?

Much appreciated! Thanks!