Can't get Physics2D.IgnoreCollision to work on multiple objects

I'm making a 2D side scroller and the player character has a dashing action. I'd also like to have this dashing action ignore the colliders on specific objects, so the player can pass through them.

For a while I was simply disabling the player collider while dashing for prototyping purposes but I obviously can't keep it this way because the player can pass right through the scenery and fall off the game. So this is what I tried to do to create my own ignore collision mask kind of thing:

 private Collider2D col;
 public Collider2D[] dashMasks;
 
 void Awake()
     {
         col = GetComponent<BoxCollider2D>();
         dashMasks = GetComponents<Collider2D>();
     }
 
 void Update()
     {
         if(dashing)
         {
             foreach(Collider2D mask in dashMasks)
             {
                 Physics2D.IgnoreCollision(col, mask);
             }
         }

No errors but it just doesn't work. Player will still collide with whatever objects I put in the inspector field. What did I do wrong?