How can I get the closest point on a concave mesh to a world position?

In our project we have several walls which are using a concave mesh collider. We're using the mesh collider because the walls are all connected and we don't want to manually create dozens of box colliders which could take hours and prevents easy iteration.

Here's my problem: We have an object which, when colliding with a wall, needs to be offset from the wall to keep it from clipping into the wall. This is not a physics object, and we are not using physics in our project for the most part and would like to keep it that way (we are building to low-end VR devices).

I have been trying to solve this problem by getting the closest point on the wall collider to the object that needs to not clip into the wall, and then calculating the direction and offsetting the object out of the wall. The problem is that all of Unitys solutions for getting the closest point on a mesh do not support concave meshes.

For context: The object we are trying to eject from the wall is a teleport marker, where the player can choose their next spot to teleport to.

Any ideas? Thanks in advance.