How can I get info on object in scene view without selecting it?

I am making an editor not unlike a terrain paint brush, but having trouble. I am trying to assign materials to game objects by first selecting a material and then clicking on the object in scene view. Problem is, when I click on the object in Scene View, it selects that object, my custom painting editor loses focus, and it all breaks down at that point.

How can I get the reference to the object clicked in scene view without it actually selecting that object?