Player transparency with FP/TP system

Basically, the character system I am designing is interchangeable between first and third person views (in a similar vein to how the Fallout/Skyrim/typical Bethesda system works) based on a key toggle. I have the actual mechanics in place and working fine; the Vector positions of each viewpoint are set in the inspector and the camera smooth lerps between them over a given time scale. This looks fine, until the camera gets inside the back of the characters head where you can see the back of the face mesh for a split second. Floating eyeballs and teeth look pretty weird. I've played around with mesh transparency and shader-changing based on camera position but neither came to much. What would be a better solution to this? Completely changing the player model to just arm meshes (i.e. "first person mode") upon toggling and forget about the lerp? Or am I on the right track without undoing my hard work?