In a tile-based game, best way to select multiple tiles via dragging?

So I'm making a game with a grid of tiles that represent trees, and I'm trying to write a function where you drag the mouse over the trees you want to chop down, and one by one they're destroyed.

The way I have it right now is the mouse is constantly shooting out Raycasts to the board while the mouse is held down, then the tiles that are hit get added to a list of tiles to be destroyed. It works for the most part, except:

1) The same trees can be added multiple times. I've gotten around this by using RemoveAll for each tree that's destroyed, but I'm worried that this is adding to my second problem, which is:

2) Once the first tree is removed, I get NullReferenceException error about the tree not existing. What's weird is that sometimes it will work for the first two trees, but never more than that. If someone could take a look at my code and tell me what's wrong, I'd greatly appreciate it.

     IEnumerator Axe(){
         while (Input.GetMouseButton(0)){
             Ray ray = Camera.main.ScreenPointToRay (Input.mousePosition);
             RaycastHit hit;
             if (Physics.Raycast (ray, out hit, 1000)) 
             {
                         
                 splashZone = Physics.SphereCastAll(transform.position, splashWidth, ray.direction, 200);
                 for(int i = 0; i<splashZone.Length; i++)
                 {
                     if(splashZone[i].collider.gameObject.name == "Fire Tree(Clone)"){ 
                         chopList.Add(splashZone[i]);
                         splashZone[i].transform.renderer.material.color = new Color(.8f,.2f,0f);
                     }
                 }
             }
 
                 yield return null;
         }
 
         if (chopList.Count>0)
             StartCoroutine(Chop(chopList));
     }
 
     IEnumerator Chop(List<RaycastHit> todoList) {
         for(int i = 0; i<todoList.Count; i++)
             
         {
             yield return new WaitForSeconds(axeDelay);
             Destroy(todoList[i].transform.gameObject);
             todoList.RemoveAll(chopped => chopped.transform.position == todoList[i].transform.position);
 
 
         }