Configurable Joint attaches at different points

I am creating and destroying a configurable joint to simulate a player picking up a ball. The first time the ball is picked up the anchor is centered on the player regardless of where the player is in relation to the ball -- but after destroying the joint (throwing) and recreating it (picking up) the ball is no longer centered and is anchored at the closest point on the player to the ball at the time the joint is created.

The ball is always centered the first time it is picked up

Off center after dropping and subsequent pickups

What do you think could be causing this? I use the same code to create the joint every time.