Some collision occurs between objects despite that I am using Layer Collision Matrix to disable collision between them.

In the scene I have added an FPSController prefab from the Standard Assets Package (Standard Assets/Character/FirstPersonCharacter/Prefabs. Also I have added some imported 3d models (furniture) in the scene. The furniture models all have Mesh Collider components on them.

I have also created two Layers: "Player" and "Library". The FPSController and all of its children are in the "Player" layer. And the imported 3D models are all in the "Library" layer.

I have used the Layer Collision Matrix in Project Settings -> Physics to disable collision between the layers "Player" and "Library".

Despite the Layer Collision Matrix setting there is still some collision going on while walking through the 3d-models with the FPSController. It is like the FPSController is bumping into the furniture, but is able to push through, and sometimes it is pushed aside from bumping into the furniture. The furniture is not blocking the FPSController, just affecting it's movement.

I have checked that any children of the furniture models are all in the "Library" layer.

Any help appreciated.