Did not find target for sync when client rejoins a game

So, everything is fine when a client initially connects, but if they close the game and then rejoin, they start getting messages saying "Did not find target for sync " and none of the objects seem synced. Any obvious ideas on what's going on? All of my objects are network transforms, and none of my connection logic is too customized. Should I be destroying old player objects on disconnect? I have a feeling it's due to something not being cleaned up from the prior connection. I'm using UNET