Unity2D collision physics not exact

I recently decided to start playing with the new Unity2D stuff, and, while it really nice, it seems like something is wrong here.

I tried this in a more complex setup, but for this I created a new scene, created 2 objects with 2D box colliders, and created one object with a 2D box collider and a rigidbody 2d. I let the box fall and this was the result.

The grey box is the rigidbody and the other 2 are just box colliders. As you can see the rigidbody kind of floats off of the other boxes. If I apply a small force to the box in a direction it will just bounce back to this position after the force goes away.

Is this intended behavior or am I doing something wrong? I have thought about just writing the collisions myself using some ray casts but I want to make sure this isn't fixable first.

Thanks!