Removing Tris/Verts/Polygons from an merged mesh? (C#)

Hello~ I need help with optimizing an cube-based terrain :)

The terrain is split into chunks which can hide themselfs, this what you see is roughly about a tenth of the map size.

But this should be enough for our purpose, you see that we have about +500k tris and +1.1kk verts and quite a batch of other data that probably is beating the GPU or CPU bloody... Also an sh...whatchamacallit-load of colliders.

The probably only way to reduce the strain should be by merging the meshes, the same for the colliders.

There is the Mesh.CombineMesh method, however this is just literally merging the... Who ever wants to count this... cubes. The problem is that after the merge the cubes retain their individual shape, I could live with repetions on the Z axis (Image top right for the axis ^^) but when there are thousands of repetions on the Y and X axis its to much of wasted calculations that you dont need.

Keeping it short and simple: I am looking for an way to remove the unnecessary internal parts.

A bit of additional information.

The Terrain is based on the Perlin-Noise method, Mathf.PerlinNoise as such I can easily pre-calculate what the next column will cover and what what it had covered. As such I know which sides will be covered.

The only way thats I could think of was creating several cubes which can fit in every possible situation (Open Left, Open Right, Open Bottom, Open Top and the combinations of these), however considering that it isnt just a block or two that is planned I would prefer having something to run over the individual blocks.

Thanks to everyone who bothered to reading till here.