Remove a single tile from a tilemap

For starters, a weird issue I am having is that when my 3 tilemaps are set to 0 for "order in layer", my floor (dirt) map pretty much is the only art that shows, but the colliders of the wall map etc still work, so as of now the wall tilemap is set to 1 in the order so it shows up. I am trying to remove a single tile from the wall after a raycast, and want to replace it with the dirt floor after this action. The relevant code is here:

 public void Harvest()
 {
     Debug.Log("Harvest time");
     if (this.transform.localScale.x == 1)
     {
         RaycastHit2D hitInfo = Physics2D.Raycast(transform.position, transform.right, reach);
         if (hitInfo.collider != null)
         {
             currentTileMap = hitInfo.collider.gameObject.GetComponent<Tilemap>();
             Vector3 worldPoint = hitInfo.transform.position;
             Vector3Int intPosition = currentTileMap.WorldToCell(worldPoint);
             tile = currentTileMap.GetTile(intPosition);
             Debug.Log(currentTileMap);
             Debug.Log("Current tile is: " + tile);
             currentTileMap.SetTile(intPosition, null);
         }
     }

The raycast calls back that it has collided with the wall and names it in the Debug.Log("Current tile is: " + tile); but after this, it calls null and the actual tile still shows up after it is set to null.