Accessing mesh vertices is extremely inefficient; any workarounds?

Apparently when you use a Mesh in ReadWrite mode, it stores it's vertices, uv coords, triangles, etc on both the RAM and VRAM instead of just the VRAM alone.

That makes sense and all except for one problem. I can't access this data on the RAM without making an entire copy of the array each time.

Now, I could store my own Vector3[] and reassign it each time I want to apply changes to the Mesh. (Still inefficient because it has to do a full copy with each assign, but much better than doing two full copies on both the read and assignment just to change a few vertices)

The problem with that is now there are two copies of the vertex array on the RAM along with the copy in the VRAM. All that wasted RAM makes this pretty unscalable.

Why can't I have raw access to the actual vertice/triangle/uv/etc arrays on the Mesh itself and then call some function when I'm done making changes?