Unity floor is light without any light shining on it

I have spent multiple hours trying to figure out why the floor isn't pitch black when the scene doesn't have any lights and the ambient lights is set to black. No idea why is this happening but it is the case with all planes and object's face that is facing upwards. What is causing this issue?