two boxCasts casted from same point give different results

Hello, I'm creating a game scene by this steps:

1. I create a room by pilling up random rectangles of tiles on each other, creating a 2d array containing floor tiles, delete overlaps, then step by step surround the floor by walls. A tile is a prefab, it has a Collider2D component with box filling the full square, I change it's layer and sprite when generating room. I instantiate tiles as Instantiate(prefab, parent, true) to maintain it's world position.

2. I create few of these rooms, then move them apart of each other by so they don't overlap. Rooms maintain the world position the same way.

3. I create a room surrounding all of the generated room by creating a big 2D array of all walls and floors from all rooms, then fill the empty spaces with floor and surround with wall the same as I did in first step.

4. I create net of nodes in diamond shape so that they are on every square on points (-0.5, 0) (0, 0.5) (0.5, 0) (0, -0.5) starting from the square center.

5. For every created node I cast a box from it's position in all 8 directions (excluding left, for leftmost nodes, etc.). This box is a standard size collider that I planned for my moving character (a little bigger than a player collider that I refer to later) so it should check if movement between nodes is possible.

6. I disable bad nodes based on number of proper connections. Nodes placed directly inside a wall should have 0 proper connections as queries start in colliders.

7. I disable bad nodes and later on use this net of nodes to navigate with A* algorithm. Player can move freely in all directions using arrows (WSAD) but he can also move through the net of nodes by mouse clicking.

So, the problem is this:
At first, I used a tilemap for my test map and build it manually placing tiles on different tilemap for floor and wall layer. Nodes worked perfectly.
Then I switched to procedural generation of map, placing each tile with steps described above.
And on procedurally generated map strange things happen.
As the player moves, the wall colliders work perfectly, but when I'm generating nodemap almost none of the nodes collide with walls, 4 of them collide with one particular wall in every direction which is impossible, as all of them are placed together in the different tile (1 to left).
What's more curious, when I place player exactly so it's movementCollider position is at a node position it collides properly.
The BoxCast2D gives different results for these objects, given the same parameters.
I would gladly share some lines of code, but the problem is so complicated in it's nature I couldn't decide on what might be crucial for solution.
I think that I might be misunderstanding something about position. Or maybe Instantation is some late process so that colliders are not set properly in the moment of checking the node connections? I change the layer after instantiating, maybe that's the problem?
Some general fragments of code are like this:
LevelManager does this on start of the stage:

 GameManager.instance.mapGenerator.InitiateMap(level, stage); // build map from tiles
 GameManager.instance.gridManager.levelRoom = GameManager.instance.mapGenerator.levelRoom; // set main room for grid manager
 Vector2 colliderSize = DetermineColliderSize(); // determine how big should be collider for creating nodes
 GameManager.instance.gridManager.InitializeMap(colliderSize); // build node map

For the grid generation I do it like this:

     Width = levelRoom.boundX;
     Height = levelRoom.boundY;
     Offset = new Vector2(levelRoom.transform.position.x, levelRoom.transform.position.y);
     Debug.Log("T = " + Top.ToString() + " B = " + Bottom.ToString() + " L = " + Left.ToString() + " R = " + Right.ToString());
     Nodes = new Node[GetXSize() + 2, GetYSize() + 2]; // bigger for out of bounds null checks
     Debug.Log("Nodes X = " + (GetXSize() + 2).ToString() + " Y = " + (GetYSize() + 2).ToString());
     for (int x = 0; x < GetXSize(); x++)
     {
         for (int y = 0; y < GetYSize(); y++)
         {
             float ptx = Left + x * UnitSize + MapTile.offsetX;
             float pty = Bottom + y * (UnitSize/2f) + MapTile.offsetY;
             int offsetx = 0;
 
             if (y % 2 == 0)
             {
                 ptx = Left + x * UnitSize + (UnitSize / 2f) + MapTile.offsetX;
                 offsetx =  1;
             }
                             
             Vector2 pos = new(ptx, pty);
             Node node = new(x + offsetx, y, pos, this, nodePrefab);
             Nodes[x + offsetx, y] = node;
         }
     }

And then for checking connections it goes like:

     if (X > 0)
     {    
         if (X > 1 || Y % 2 == 1)
         {
             //Left
             valid = !this.BadNode;
             hit = Physics2D.BoxCast(Position, colliderSize, 0, new Vector2(-1, 0), GridManager.UnitSize, layerMask);
             Debug.Log("hit (-1, 0) X - 1, Y");
             if (IsCollisionHit(hit))
             {
                 valid = false;
             }
             Left = new NodeConnection(this, grid.Nodes[X - 1, Y], valid);
         }
 
         //TopLeft
         if (Y < grid.GetYSize() - 1)
         {
             valid = !this.BadNode;
             hit = Physics2D.BoxCast(Position, colliderSize, 0, new Vector2(-1, 1), diagonalDistance, layerMask);
             Debug.Log("hit (-1, 1) Y % 2 == 0 ? X - 1 : X, Y + 1");
             if (IsCollisionHit(hit))
             {
                 valid = false;
             }
             TopLeft = new NodeConnection(this, grid.Nodes[Y % 2 == 0 ? X - 1 : X, Y + 1], valid);
         }

IsHitCollision just checks if hit.collider is null and dumps collider info if it's not.
layerMask is LayerMask.GetMask("Wall", "BlockingObject"); - the same as for the player
Speaking of, the player motor upadate script:

     float distance = Mathf.Sqrt(moveDelta.x* moveDelta.x + moveDelta.y * moveDelta.y) * Time.deltaTime;
 
     hit = Physics2D.BoxCast(
         GetMovementColliderBoxPosition(),
         movementCollider.size,
         0,
         new Vector2(moveDelta.x, moveDelta.y),
         distance,
         colissionLayerMask
         );
 
     if (hit.collider == null)
     {
         OnMove();
         transform.Translate(new Vector3(moveDelta.x * Time.deltaTime, moveDelta.y * Time.deltaTime));
     }
     else
     {
         GameManager.instance.debug.IntervalLog("Collision of " + name + " position " + GetMovementColliderBoxPosition().ToString() + " local " + movementCollider.transform.localPosition.ToString() + " with "+hit.collider.name);
         UpdateMotorForSeparateAxes();
     }

I checked by Debug.Log and doublechecked by stopping at breakpoints, that both of the BoxCasts, called with the same position, layerMask, almost the same colliderSize (slightly bigger), and the same direction (I checked the collision with Vector3.zero too, to not move the box from it's starting position) give different result.
The only difference I see between them is that one is called really early, before even populating the game with characters and the other is during the Update while playing the game.
I guess I'm missundersting some mechanic behind positioning or instantiating, please give me some advice as where to look for the problem.
EDIT:
I changed Instantiating to be PrefabUtility.InstantiatePrefab, then set correct parent transform. Unfertunately effect stays exactly the same.