"InvalidCastException: Specified cast is not valid." Error

Hello there! I have a script that will create the menus in my game procedurally so I don't have to make them all myself. It has all been working perfectly fine up until now, and I haven't changed anything that should affect it. I am getting this error :

 InvalidCastException: Specified cast is not valid.
 BuildingMenus.ClearButtons (System.Boolean buildMenu) (at Assets/Scripts/Menus/BuildingMenus.cs:65)
 BuildingMenus.DrawBuildMenu () (at Assets/Scripts/Menus/BuildingMenus.cs:48)
 UnityEngine.Events.InvokableCall.Invoke () (at <4a29708cee654ac0aca16c9d5624806a>:0)
 UnityEngine.Events.UnityEvent.Invoke () (at <4a29708cee654ac0aca16c9d5624806a>:0)
 UnityEngine.UI.Button.Press () (at Library/PackageCache/com.unity.ugui@1.0.0/Runtime/UI/Core/Button.cs:70)
 UnityEngine.UI.Button.OnPointerClick (UnityEngine.EventSystems.PointerEventData eventData) (at Library/PackageCache/com.unity.ugui@1.0.0/Runtime/UI/Core/Button.cs:114)
 UnityEngine.EventSystems.ExecuteEvents.Execute (UnityEngine.EventSystems.IPointerClickHandler handler, UnityEngine.EventSystems.BaseEventData eventData) (at Library/PackageCache/com.unity.ugui@1.0.0/Runtime/EventSystem/ExecuteEvents.cs:57)
 UnityEngine.EventSystems.ExecuteEvents.Execute[T] (UnityEngine.GameObject target, UnityEngine.EventSystems.BaseEventData eventData, UnityEngine.EventSystems.ExecuteEvents+EventFunction`1[T1] functor) (at Library/PackageCache/com.unity.ugui@1.0.0/Runtime/EventSystem/ExecuteEvents.cs:272)
 UnityEngine.EventSystems.EventSystem:Update() (at Library/PackageCache/com.unity.ugui@1.0.0/Runtime/EventSystem/EventSystem.cs:501)

Quite a long one! The thing is, it only happens when I select the tab the SECOND time. So after the game objects are already there. The error is from the ClearButtons() method. My script is this:

 using System.Collections;
 using System.Collections.Generic;
 using UnityEngine;
 using UnityEngine.UI;
 using TMPro;
 
 public class BuildingMenus : MonoBehaviour
 {
     private BuildingManager buildingManager;
     private Decorating decorating;
 
     [SerializeField] private GameObject buttonPrefab;
     [SerializeField] private GameObject decorateButtonParent;
     [SerializeField] private GameObject buildButtonParent;
     [SerializeField] private GameObject buttonStart;
     GameObject previousButton;
 
     [SerializeField] private float offset;
 
     private Button[] buttons;
 
     void Start()
     {
         buildingManager = GameObject.Find("Building Manager").GetComponent<BuildingManager>();
         decorating = GameObject.Find("Building Manager").GetComponent<Decorating>();
     }
 
     public void DrawDecorateMenu()
     {
         previousButton = null;
         ClearButtons(false);
         foreach(DecorationAsset decoration in decorating.decorations)
         {
             if (decoration.unlocked)
             {
                 if (previousButton == null) { previousButton = Instantiate(buttonPrefab, buttonStart.transform.position, transform.rotation, decorateButtonParent.transform); }
                 else { previousButton = Instantiate(buttonPrefab, new Vector3(previousButton.transform.position.x + Screen.width/offset, previousButton.transform.position.y, previousButton.transform.position.z), transform.rotation, decorateButtonParent.transform); }
                 previousButton.GetComponent<Image>().sprite = decoration.icon;
                 previousButton.GetComponent<Button>().onClick.AddListener(delegate {decorating.SelectDecoration(decoration); });
                 previousButton.name = decoration.name;
             }
         }
     }
 
     public void DrawBuildMenu()
     {
         previousButton = null;
         ClearButtons(true);
         foreach (CoursePieceAsset piece in decorating.coursePieces)
         {
             if (piece.unlocked)
             {
                 if (previousButton == null) { previousButton = Instantiate(buttonPrefab, buttonStart.transform.position, transform.rotation, buildButtonParent.transform); }
                 else { previousButton = Instantiate(buttonPrefab, new Vector3(previousButton.transform.position.x + Screen.width / offset, previousButton.transform.position.y, previousButton.transform.position.z), transform.rotation, buildButtonParent.transform); }
                 previousButton.GetComponent<Image>().sprite = piece.icon[0];
                 previousButton.GetComponent<Button>().onClick.AddListener(delegate { decorating.SelectCoursePiece(piece); });
                 previousButton.name = piece.name;
             }
         }
     }
 
     public void ClearButtons(bool buildMenu)
     {
         if (buildMenu) {
             foreach (GameObject obj in buildButtonParent.transform) { Destroy(obj); }
         }
         else {
             foreach (GameObject obj in decorateButtonParent.transform) { Destroy(obj); }
         }
     }
 }

I have tabs that you click on to switch between different sections of the building menu. Here is a screenshot of what one of the tabs click events look like if this is relevant at all:

I am assuming that this is something that I have just completely missed and I am just being dumb, but I hope this is enough info to help! If you need to see anything else please let me know.