2.5D lighting looks wrong when rotating camera

I use quad-textured 2D characters in 3D world and each character has a code below in Update() method so that they always face straight to camera.

transform.rotation = Camera.main.transform.rotation;

They also use modified transparent-cutout shader so they can receive lighting and cast shadows. It's working alright for most of the time, but in some light intensive area like when a character is sitting besides bright campfire in dark night, lighting effect looks weird on him.

As you can see, only camera angle has changed in the image above but lighting effects on the characters have a distinct difference(and ugly!) And this is what it looks like in the editor when rotating camera.

I can see why this is happening but can anyone give me some advices/ideas on how to fix this? Ideally, I want all the 2d characters to be affected by light only by distance from light sources and not camera angle. But it beats me since there are many light sources in actual game(using deferred rendering). Please help.