Why do my matching functions match more gems than they should?

Hey there.

I am new to unity programming, so I thought I could start with something rather simple and probably even overdone, and create a match 3 game. I found some tutorials here and there, and managed to start with something that a person can actually play, but I ran into a few problems that I don't seem to be able to fix. I noticed that unity Answers are for one specific question, so I'll try to be as specific as possible and focus on the most baffling issues.

In my game I have a script that describes the behavior of the "board". This is the script responsible for generating the "gems", for checking the swapping and pretty much everything that is not specific to one particular gem. In there, I have a function that is responsible for creating lists with the neighboring gems that have the same color with a specific gem. This function is called first of all when the player makes a swap, so I can see if there is a match, and also it's used for every gem on the board right after every move so any other matches that might have occurred will disappear as well. The code of this function is here:

 public void ConstructMatchList (string color, Gem gem, int XCoord, int YCoord, ref List<Gem> MatchList)
     {
         if (gem == null)
         {
             return;
         }
         else if (gem.gemColor != color)
         {
             return;
         }
         else if (MatchList.Contains (gem))
         {
             return;
         }
         else
         {
             MatchList.Add(gem);
             if (XCoord == gem.XCoord || YCoord == gem.YCoord)
             {
                 foreach (Gem g in gem.Neighbors)
                 {
                     ConstructMatchList(color, g, XCoord, YCoord, ref MatchList);
                 }
             }
         }
 
     }

The problem with this function is that it detects ALL neighboring gems of the same color, including the ones that do not form a straight line with the gem passed as a parameter in it. To solve this problem I use another function that divides the list in two different ones, one for the x axis, one for the y axis. And here it is (Note: the constant ammountToMatch is set to 3):

 public void FixMatchList (Gem gem, List<Gem> listToFix)
     {
         //This function is needed to make sure our match lists contain the proper amount of gems because
         //without it, the function ConstructMatchList counts all gems of the same color, regardles of
         //being vertically or horizontally attached to the gem that was swapped.
         List<Gem> rows = new List<Gem>();
         List<Gem> collumns = new List<Gem>();
 
         for (int i=0; i<listToFix.Count; i++)
         {
             if (gem.XCoord == listToFix[i].XCoord)
             {
                 rows.Add(listToFix[i]);
             }
             if (gem.YCoord == listToFix[i].YCoord)
             {
                 collumns.Add(listToFix[i]);
             }
         }
 
         if (rows.Count >= amountToMatch)
         {
             isMatched = true;
             for (int i=0; i<rows.Count; i++)
             {
                 rows[i].isMatched = true;
             }
         }
         if (collumns.Count >= amountToMatch)
         {
             isMatched = true;
             for (int i=0; i<collumns.Count; i++)
             {
                 collumns[i].isMatched = true;
             }
         }
 
     }

Now to get to the point, the problem is that I still encounter issues of incorrect matching. While these two functions called one after the other should ensure the creation of a correct list of matches, they actually still fail to do so and I can't find the problem so far. After a match is made by the user, I often encounter cases of L shaped matches occurring as the new gems appear, which makes no sense... I also tried to print out the size of the lists (rows and collumns) and what I found was bizarre to say the least. It seems that rows holds the VERTICAL matches while the collumns holds the Horizontal. How is that even possible? I mean... I am using the x and y axis to determine which goes where, and if I haven't forgot everything yet, they should be the other way around. Any ideas/theories why this does not work properly?

Here's how the properties XCoord and YCoord are declared:

 public int XCoord
     {
         get
         {
             return Mathf.RoundToInt(transform.localPosition.x);
         }
     }
 
 public int YCoord
     {
         get
         {
             return Mathf.RoundToInt(transform.localPosition.y);
         }
     }

Thank you in advance for your time, and I hope my question is not too lengthy or confusing. I would really appreciate any help because I have to admit I'm rather stuck as it is.