Collider Performance Question

Hey all.

I just have a simple question that I can't seem to find the answer to (maybe not looking very well).

If I have colliders that are not set to collide with anything at all, not each other, not the player, not anything, then:

1. Are those going to be doing per-frame calculations to detect collisions (I would assume no because there's nothing they can collide with).

2. Will Physics.Raycast still work against them if I test for that specific layer?

The context of this question is: I have a voxel game. And Mesh Colliders are terribly slow to update, so I took them out and added a system where box colliders surround the player where solid blocks exist. This is much more efficient.

However, I was using raycasting to place and delete blocks. I would cast a ray at the ground, it would hit the mesh collider, I'd get the normal and use that to figure out where to put the block.

I can't do that anymore, because those blocks no longer have colliders. There are only colliders around the player. So I figured I'd write my own very simple ray casting system that creates a ray and uses Ray.GetPoint() to advance along it 1 unit at a time, checking each spot for a solid block. If it finds one, it returns that position and I can either delete it or go back 1 unit to place a block.

The problem here is this method is not accurate. Often, the block modified is the neighbor block from the one I clicked. And I'm not entirely sure how to make it accurate.

So I could put box colliders around every block. But that's a LOT of box colliders, and given that they're doing per-frame calculations, I can't afford to do that.

But, if I can put a LOT of box colliders that aren't doing per-frame calculations, then it might be more do-able. So I need to know the answer to this question to figure out if I can do that.

Though it might be pretty bad on memory usage...