Reset the last local players toggle button while keeping online players selection?

The server side of things seems to be working but local update of the toggle buttons is overwriting the entire toggle group instead of just the last button they specifically selected.

 private Toggle lastSelectedBtn;
 
  public void toggleSelection(Toggle clickedButton) //local
             {
                 if(lastSelectedBtn != null){
                     lastSelectedBtn.interactable = true; //this is setting the entire toggle group to interactable rather then just the last local players selection.
                 }
     
                 ButtonSelectionCommand command = new ButtonSelectionCommand(null, clickedButton.gameObject.name, null);
     
                 OnSelection?.Invoke(clickedButton.gameObject.name);
     
                 lastSelectedBtn = clickedButton;
             }

Any one know how to make sure I am only reenabling the local players previously selected toggle if they change their mind and not overwrite any other players selections? Preferably using built in unity ui Toogle or Button rather than making a custom multi-select button group from scratch.