Weird shaky player behavior (moving through colliders)

First off, both the character and ground have colliders, this is not the issue, please don't tell me this is why.

The level is made up of floating islands, and you control a cube that rolls around the level, killing enemies and looking great doing it. The movement system does not use Physics, instead manually rotation the object, and changing the position of the cube. Using physics did not yield the correct behavior I wanted for the movement (Perfect rolls, landing exactly 1 cube length away). I think this may be the issue, the cube is getting clipped into the island colliders and slipping through. The camera also shakes vigorously while going up or down a slope, but I will fix that after the primary concern is resolved, as I do not want the player falling through the ground.

Versus

I believe my best bet is to try and detect a slope, and move the character up more to prevent this clipping while on a slope. Would this be the best way to prevent this issue occurring? Or is there a blatantly obvious solution that I am apparently blind to?

Help would be appreciated greatly! Thanks!