HDRP double sided light problem

Hi.

I am new to HDRP, and I am trying to make an indoor environment. I have this problem, when I bake the light on the wall model that contains the door, that the lighting only works properly on one side. The normal wall works fine, but I assume it is because the "doorwall" material is double sided and that there is some problem connected to that.

As you can see the light is fine on the other side, and when all the light is realtime, the problem does not occur, only when baked.

I am still a newbie in unity, so I hope I am missing some obvious mistake, all though I have tried many things to solve this by now.

All help is appreciated, thanks!