Ball Character controlled by a single Joystick?

So I am having some issues with this setup. A ball as the character, that can roll around and has a rigid body. I can get that far, but when I start adding controllers and cameras, things start getting "wonky". So far, my best attempt got the ball to roll and a camera "hard attached" to it, but u cannot turn corners and stay behind the ball. If you go forward and are behind it, your fine, if you turn down a hall for example, now your beside it, because the camera is always facing a set direction. If I start to add in a way for the camera to really follow and not face the same direction all the time, then it either spins with the ball and / or will turn up to 90 deg and snap back when the joystick moves back to its 0 center.

I would challenge anyone to create a ball character that can roll around and properly be followed by a camera. This has proven to be much more of a problem than I imagined. Keyboard and mouse controls are easy, but a single joystick for a ball character is a demon all its own. Any advice?