Need help aiming at new world space co-ordinates for gun recoil/spray pattern

I have been stuck on this issue for a good few weeks and i still haven't figured it out. I feel it's an easy solution but i do need some support please.

I'm making a gun spray pattern system which follows the spray pattern of a given image, which you can see HERE. The idea is that the green pixel will be the initial shot and then will follow the blue pixels up until the red pixel.

I generate a list of Vector 2 based on the image to get the "co-ordinates" of each shot. The output of this particular image is:

• Shot 1 - X16 Y0

• Shot 2 - X16 Y1

• Shot 3 - X15 Y2

• Shot 4 - X14 Y3

• etc...

  ---

The idea was to simply add the X and Y vector from each shot to what the player is currently looking at. Though X16 would technically be X0 as thats the start position. For example, if my player is currently looking at X10,Y7,Z0 the first shot will add some recoil resulting in the camera now looking at vector X0,Y8,Z0, then for every shot after continue to add the difference between X and Y. This is the code i am currently using

 void FireWeapon()
     {
         RaycastHit hitInfo;
         if (Physics.Raycast(Camera.main.transform.position, Camera.main.transform.forward, out hitInfo))
         {
             Debug.DrawRay(Camera.main.transform.position, Camera.main.transform.forward * 100, Color.red, 100f);
             GameObject obj = Instantiate(bulletHolePrefab, hitInfo.point + new Vector3(0f, 0f, -0.02f), Quaternion.LookRotation(-hitInfo.normal));
         }
 
         //Original kinda working
         Camera.main.transform.localRotation = Quaternion.AngleAxis(-generateScript.GetVector(imgPoint).y, Vector3.right);
         playerTrans.localRotation = Quaternion.AngleAxis(generateScript.GetVector(imgPoint).x, Vector3.up);
 
         //print(generateScript.GetVector(imgPoint));
         imgPoint++; //The current pixel to be firing at
     }

THIS is what currently happens. The first shot which you can see in the bottom right is where the player was initially looking, but the second shop instantly snaps to an entire new position and then follows the pattern.

My THEORY is that i am not adding to my players current view rather than i am setting it. Though i am stuck on what i need to change/implement. Any suggestions would be great. Thanks