Why is my texture offset accelerating?

I'm trying to linearly increase and decrease the offset of a background texture based on the movement of my player character. It starts well but for some reason, the more the player moves, the faster the offset increases and decreases. At the end, the background is scrolling way too fast.

 public GameObject player;
 private Vector3 previousPosition;
 private Vector2 savedOffset;
 public float scrollSpeed = 0.001f;

 void Start()
 {
     previousPosition = player.transform.position;
     savedOffset = GetComponent<Renderer>().material.mainTextureOffset;
 }

 // Update is called once per frame
 void FixedUpdate()
 {
     float vertical = player.transform.position.y - previousPosition.y;
     float horizontal = player.transform.position.x - previousPosition.x;
     float y = Mathf.Repeat(Time.time * scrollSpeed * vertical, 1);
     float x = Mathf.Repeat(Time.time * scrollSpeed * horizontal, 1);

     Vector2 offset = new Vector2(Mathf.Repeat(savedOffset.x + x, 1), Mathf.Repeat(savedOffset.y + y, 1));
     GetComponent<Renderer>().material.mainTextureOffset = offset;
     savedOffset = offset;
     previousPosition = player.transform.position;

     Debug.Log("offset y: " + GetComponent<Renderer>().material.mainTextureOffset.y);
 }

The calculations of x and y are not the problem. I've tried setting offset:

 Vector2 offset = new Vector2(x, y);

Then, it increases and decreases linearly. The problem occurs when I add it to the previous offset, but those are just additions so I don't understand how can it accelerate the offset.

 Vector2 offset = new Vector2(Mathf.Repeat(savedOffset.x + x, 1), Mathf.Repeat(savedOffset.y + y, 1));

I'm adding a screenshot to put things in context. The idea is to scroll the background in the opposite direction to the player. Thank you all in advance.