camera follow rotation issue 3rd person controller

I'm creating a 3rd person controller and I'm having an issue with the camera rotation. The player is controlled with the left gamepad stick and the camera can rotate around the player with the right stick. Everything works fine when the player is on the ground, but my player is also able to walk on walls. I do that by setting the up transform of the player to the surface normal it is on.

When the player is on the wall I would like the camera to rotate with it, but I can't make it a child of the player because then it won't follow the player correctly. Right now the camera stays horizontal always. I made the controller with help from a tutorial so that is why it stays always horizontal, but I don't know how to change it to rotate with the player so it will always rotate around the players local up vector.

Here is the part of my code that is used for the camera follow and rotation:

 Vector3 _velocityCamSmooth = Vector3.zero;
 float _camSmoothDampTime = 0.1f;
 float _distanceUp = 2;
 float _distanceAway = 8;
 Transform _player;
 
 void LateUpdate ()
 {
     Vector3 cameraOffset = _player.position + new Vector3(0, _distanceUp, 0);
     Vector3 lookAt = cameraOffset;
     _targetPos = Vector3.zero;
 
     _lookDir = cameraOffset - transform.position;
     _lookDir.y = 0;
     _lookDir.Normalize();
 
     transform.RotateAround(cameraOffset, _player.up, _freeRotateDegPerSec * (Mathf.Abs(_input.R_Stick_X) > _rStickThres ? _input.R_Stick_X : 0f));
 
     _targetPos = cameraOffset + _player.up * _distanceUp - _lookDir * _distanceAway;
 
     transform.LookAt(lookAt);
 
     this.transform.position = Vector3.SmoothDamp(transform.position, _targetPos, ref _velocityCamSmooth, _camSmoothDampTime);
 }
 

I hope this is a clear enough explanation of the issue. The controller is supposed to be similar to the Ratchet and Clank controller where he can go up magnetic walls and the camera always rotates with him and around his local y.