Syncing data from client to server in 5.1 networking

I'm trying to implement multiplayer in an existing game using the new networking system. It's a first person beat 'em up (sort of) where you try to survive for as long as possible through waves of enemies. We're aiming for around 200 enemies of various types per round. Most of these are very basic and act as a kind of background noise for the more complicated encounters. They mostly wander around the stage using the navmesh, but use physics when attacking and being attacked. I use a heavily compressed sync system when they wander to save bandwidth, but the physics attacks need to be precise and clear. My idea is to give the client that is attacking/being attacked temporary authority over the object and then use a network transform or something similar to send the position data to the server/other clients. This way the affected player sees exactly what is happening and the other players (who are probably off doing something else) get a decent approximation. Kind of like how each player has local authority over themselves, but extended to certain rigidbodies. However, with the server authoritative system in Unity 5.1 I haven't been able to find a way to do this. SyncVars only send from server to client and commands don't work from objects that aren't players. I could set the enemies to temporarily become players, but that seems extremely clunky and weird. Is there an easier way to do it?