Raycast not consistently detecting collision.

In my game, I have a ball that bounces around as it hits stuff it gets continuously faster. First I had walls around my ball to keep it in and I was using raycasting to detect whether or not it was within the walls. That worked decently, but the problem was that after the ball got fast it sometimes would shoot through the wall and was gone from the screen. I decided to change how I keep my ball on the screen by calculating the height and width of the screen and making sure the ball doesn't leave that location. However, when I did this, it broke the other raycasting that I was doing. There are some objects that the ball needs to collide with still. What is weird is that sometimes it detects the collision, but sometimes it doesn't. At first, I didn't change the raycast at all, but I tried fixing it using a single linecast. This does the same thing as the raycastall. it sometimes detects the collision, other times it doesn't. Here is my code for the ball.

     private void Awake()
     {
         this.speedAmountX = Random.Range(-14f, -8f);
         this.speedAmountY = Random.Range(-6f, 6);
     }
     void Start () {
         this.mPrevPosition = transform.position;
         this.numOfPaddleHits = 0;
         this.restart();
 
         
     }
     
     // Update is called once per frame
     void Update () {
 
         this.mPrevPosition = transform.position;
         this.advance();
         this.OnRayCastHit();
 
     }
     public void advance()
     {
         this.transform.Translate((this.speedAmountX * Time.deltaTime), (this.speedAmountY * Time.deltaTime), 0f, Space.World);
     }
 
 
     private void OnRayCastHit()
     {
         RaycastHit hitInfo;
         if (Physics.Linecast(this.mPrevPosition, this.transform.position, out hitInfo))
         {
             if (hitInfo.rigidbody.gameObject.name == "Paddle" && this.speedAmountX < 0)
             {
                 this.numOfPaddleHits += 1;
                 this.bounceHorizontal();
             }
             if (hitInfo.rigidbody.tag == "Sqaure")
             {
                 hitInfo.collider.gameObject.GetComponent<Square>().restart();
                 this.bounceHorizontal();
             }
         }
     }