A problem with cutting holes in objects (terrains) using depth mask

So I've been at this for a couple of days now and I've been through several solutions, first thing I tried was using a depth mask shader to cut a hole in an object, it works but the problem is that the player is supposed to walk through the hole and upon doing so the player can sometimes see the wall behind the depth masked object or parts of it.

(this happens when the mask object is a tiny bit "off" the wall, I tried multiple near/far settings for the camera as well, all that does is move the problem back/forward)

Now the problem with that approach is also that I can't put the depth masked object exactly where the wall is because then I get the z-fighting effect (I'm also surprised that this is usually an accepted method for cutting holes in terrains, even saw a tutorial on it using this technique)

So after this attempt also failed I went for a direct approach of making my terrain support transparency, this works but only for very special scenarios and it takes a lot of extra work getting it just right depending on what angle the "entrance" hole needs to be due to the fact terrains work with height maps making the texture get stretched.

So I was wondering if anyone knew if I am doing something wrong when it comes to the depth mask approach?

Is it supposed to actually be perfectly aligned with the object and I'm just messing up somewhere with my shader (is z-fighting supposed to occur when you use depth mask?)

Are there any other ways to go about this?

(this only happens when the mask object is "fitted" to the cube or perfectly aligned with the cube)