Using a list of arrays to make Unity UI 2D button toggle colors

TLDR: I am trying to recreate the old vintage Light'sOut game - So here's what I have thanks to an awesome answer I received on another sub....

     public List<int> Row1 = new List<int> { 0, 1, 2, 3, 4 };
     public List<int> Row2 = new List<int> { 5, 6, 7, 8, 9 };
     public List<int> Row3 = new List<int> { 10, 11, 12, 13, 14 };
     public List<int> Row4 = new List<int> { 15, 16, 17, 18, 19 };
     public List<int> Row5 = new List<int> { 20, 21, 22, 23, 24 };
     public List<List<int>> Grid = new List<List<int>>();
 
     public List<int> Neighbors = new List<int>();
 
     private void Awake()
     {
         Grid.Add(Row1);
         Grid.Add(Row2);
         Grid.Add(Row3);
         Grid.Add(Row4);
         Grid.Add(Row5);
     }
 
     private void Start()
     {
         Vector2 Grid_Location = GetNearby(32);
         print(Grid_Location);
         if (Grid_Location.x >= 0)
         {
 
           
             Neighbors.Add(Grid[(int)Grid_Location.x][(int)Grid_Location.y]);
          
             if (Grid_Location.x > 0)
                 Neighbors.Add(Grid[(int)Grid_Location.x - 1][(int)Grid_Location.y]);
 
             if (Grid_Location.x > 0)
                 Neighbors.Add(Grid[(int)Grid_Location.x][(int)Grid_Location.y - 1]);
 
             if (Grid_Location.x > 0)
                 Neighbors.Add(Grid[(int)Grid_Location.x - 1][(int)Grid_Location.y - 1]);
 
             if (Grid_Location.x > 0)
                 Neighbors.Add(Grid[(int)Grid_Location.x][(int)Grid_Location.y]);
 
 
         }
 
     }
 
     public Vector2 GetNearby(int currentLocation)
     {
         foreach (List<int> row in Grid)
         {
 
             foreach (int item in row)
             {
 
                 if (item == currentLocation)
                 {
                     int myRow = Grid.IndexOf(row);
                     int myColumn = row.IndexOf(item);
 
                     return new Vector2(myRow, myColumn);
 
                 }
 
 
 
             }
 
 
         }
         return new Vector2(-1, -1);
         
 
     }
 
 }

But the problem still remains...

I have a 5x5 grid established through the canvas using the Grid layout component. I have 25 buttons that are all colored yellow and should turn grey when pressed. I have no clue how to make a button when clicked change itself and the four other buttons around it to grey by using this script or any other script for that matter. I am still learning so be gentle....thanks!