Unity - Problem when dragging game object

I am using the c# code below to drag a game object within a specific rectangular area. This works fine but the problem I have is that my object moves too fast and to slow it down I have tried using a speed variable. It limits the speed in the x direction correctly but for some reason it completely stops the object from moving in the Y direction. This happens even when I am only attempting to slow down the X direction as a test. My aim is to slow down the speed in both direction within the area.

Please advise what I am doing wrong here and any improvements I can make.

 using UnityEngine;
 using System.Collections;

 public class drag : MonoBehaviour {

 public float maxXValue = 9f;

 Vector3 dist;
 float posX;
 float posY;
 float speedX = 0.3f;
 float speedY = 0.3f;

 void OnMouseDown(){
 dist = Camera.main.WorldToScreenPoint(transform.position);
 posX = Input.mousePosition.x - dist.x;
 posY = Input.mousePosition.y - dist.y;
 }

 void OnMouseDrag(){

 Vector3 curPos = new Vector3(Input.mousePosition.x - posX, Input.mousePosition.y - posY, dist.z);  
 Vector3 worldPos = Camera.main.ScreenToWorldPoint(curPos);

 if (GameManager.instance.gameStart == false) {
     worldPos.x = Mathf.Clamp (worldPos.x, -5f, 5f);
     worldPos.y = Mathf.Clamp (worldPos.y, -13f, -13f);
 } else {
     worldPos.x = Mathf.Clamp (worldPos.x * speedX, -maxXValue, maxXValue);
     worldPos.y = Mathf.Clamp (worldPos.y, -17.2f, -13f);  
     //even when just limiting the speed in X direction, it stops movement in Y direction.
 }                                                               
 transform.position = worldPos;
 }
 }