Trying to click an object and have that object see what objects are near it and manipulate them based on certain rules

Trying to click an object then have that object look for objects near it and manipulate them. Pleae help, I've been working on this for a week!

 function Update (){
         
     var up = transform.TransformDirection(Vector3.up);
     var right = transform.TransformDirection(Vector3.right);
     var left = -transform.TransformDirection(Vector3.right);
     var down = -transform.TransformDirection(Vector3.up);
     var hit2 : RaycastHit;
     
         if ( Input.GetMouseButtonDown(0)){
         var hit : RaycastHit;
         var ray : Ray = Camera.main.ScreenPointToRay (Input.mousePosition);
             if (Physics.Raycast (ray, hit, 100.0)){
                 if (Physics.Raycast(transform.position,up,hit2,5) || Physics.Raycast(transform.position, down, hit2,5) || 
                      Physics.Raycast(transform.position,right,hit2,5) || Physics.Raycast(transform.position,left, hit2,5))
                      {
                      Debug.Log("go");
                      Debug.DrawRay(transform.position, up
 * 7,Color.white);
                     Debug.DrawRay(transform.position, down * 7,Color.white);
                     Debug.DrawRay(transform.position, right * 7,Color.white);
                     Debug.DrawRay(transform.position,
 -right * 7,Color.white);
                     if (hit2.collider.gameObject == this.collider.gameObject){
                     Destroy(hit2.collider.gameObject);
                     }
         }
         }
     }
     }
 

I have many plane objects. Think cards laying flat on a table with an orthographic camera facing down at them. (it's a 2d puzzle game) I want to click on one of the colored square plane objects and when I click on it, I want the object I click to send out a ray to each side to see what color the objects are that are directly next to it. Then I want to change the color of the objects next to the one I clicked based on a set of rules I will come up with. I've tried many many things over the past week. This is for android but right now I have a script that lets me click the object in the editor so I can use debug.ray to see if the rays are shooting and where to.
I also have seen rays shoot in the wrong direction. I havent been able to change it. It will shoot left and right of the object but not up and down. I shoots forward and back. Please help!