Camera falling through floor

Hey everyone,

I imported an asset from rhino as obj. This file is a building with surrounding. I used a video tutorial to script and implement auto walk for google cardboard but sadly my camera keeps falling through the rhino model. I tried solving it with colliders but I get either
a) fall through floor as soon as I click

b) I click and I take off to the sky

c) Until I click the camera bounces up and down very quickly and after I click it shoots up or down.

What am I doing wrong? Is there anything I should change in the collider setup?

Any help or pointers would be very appreciated.