GameObject Tilemap conflicts with the Pixel perfect camera

So I'm trying to make a pixel perfect game using Unity's Pixel Perfect Camera Along with that I also wish to add normal maps to the tiles, I was able to achieve this using Unity's 2D Tilemap Extras that allowed me to draw GameObjects as to the tiles. The issue is here:

As you can see there are two types of tiles: The one that reflects light is a GameObject tile while the the line of bricks in the middle is a regular sprite tile. If you look closely you can see close to the middle black line, it is the color of my camera background. How do I get rid of it? if I go up or down, this black line appears between different tiles but only in horizontal.

Meanwhile the regular sprite is not affected with this black line. If I cancel the pixel perfect camera script there is no black line but it's not pixel perfect as well.

This is my camera settings: