Set raycast on player camera or object?

Hello everyone. I am currently working an interactive interior visualization product in Unity3D and I am know at the point where I need to implement raycasting in order to get the interactive elements working. I basically want to let the player change materials of the walls and furniture by clicking on the object (with raycasting) and I figured that out for the most part.

The way how it is setup now, is that a C# raycasting script is connected to the camera of a standard character controller. So the raycast is looking for the 'interactivity' tag on objects in the scene.

I would please like to know what the best way is to implement raycasting. Is it better to put raycasting on the player controller itself? Or is it better to put raycasting on each object that is 'interactive', so to say.

Thanks for your patience.