Camera leaning problems (Lerping)

I made a script that makes the camera lean (Directly inside the Camera Script). The rotation of leaning is fine, but when i come to lerp the camera, it seems as though it has a desired time to reach a certain position in a certain amount of seconds, which leads to the player being able to move faster than the camera. Now i do not like nesting the camera in the player, as it is a bad practice and can lead to camera streches when it rotates. I have put 2 gameobjects inside the player, and named them LeanLeft / LeanRight and positioned them to where targetPosition (camera holder basically, but it doesnt actually hold the camera, the camera moves to it). Then i pressed play, and noticed my player sounded ahead of my camera. I went into scene view, and sure enough, the player was ahead of the camera. Now i realised that when i did the lerping the camera never reached is main destination fast enough, then the other destination. I do not know how to fix it because it is super confusing, so maybe someone else can help me. here's the code below.

 if (Input.GetAxis("Vertical") < 0f || Input.GetAxis("Horizontal") < 0f || Input.GetAxis("Horizontal") > 0f || Input.GetAxis("Vertical") > 0f)
                         {
                             Vector3 tempTargetPos = new Vector3();
 
                             // Camera lerping does not work.
                             // Since we are lerping the camera, while we need to get to x as fast as b, while still lerping towards c, things get complicated.
                             // For example, targetPosition is able to move faster than the camera, which is never good.
                             // And since we NEED to lerp towards our lean transform, i do not know what to do.
 
                             if (Input.GetAxis("LeanLeft") > 0)
                             {
                                 tempTargetPos = Vector3.Lerp(transform.localPosition, LeanLeftTransform.position, 0.1f);
                             }
                             else if (Input.GetAxis("LeanRight") > 0)
                             {
                                 tempTargetPos = Vector3.Lerp(transform.localPosition, LeanRightTransform.position, 0.1f);
                             }
                             else
                             {
                                 tempTargetPos = Vector3.Lerp(transform.localPosition, targetPosition.position, 0.1f);
                             }
 
                             tempPos = tempTargetPos; // + trans; Ignore trans (it's for headbob)
 
                             // AddOffsetRotation(trans * 10); Rotation, also ignore this
 
                             transform.position = tempPos; // Moving towards, Position
                         }
                         else if (Input.GetAxis("Vertical") == 0f || Input.GetAxis("Horizontal") == 0f)
                         {
                             // lean left is Q, lean right is E.
 
                             if (Input.GetAxis("LeanLeft") > 0)
                             {
                                 transform.position = Vector3.Lerp(transform.localPosition, LeanLeftTransform.position, 0.1f);
                             }
                             else if (Input.GetAxis("LeanRight") > 0)
                             {
                                 transform.position = Vector3.Lerp(transform.localPosition, LeanRightTransform.position, 0.1f);
                             }
                             else
                             {
                                 transform.position = Vector3.Lerp(transform.localPosition, targetPosition.position, 0.1f);
                             }
                         }