Huge world without huge lag?

Hello all. I'm making a pirate game where you sail the seas as a pirate. The world would be composed of meshes with height maps to simulate islands. In total, the world would be 50 - 60 of these height maps wide, and 50 - 60 of these height maps long (world map would be flat, so I'm not trying to make a sphere with them). Each of these terrain pieces would have 128x128 vertices on them. And each vertex would be roughly 3.9 of unity's units, or in the game terms, about 230 feet. In total the map would be about 340 miles^2. Inside these terrain maps are harbors and stuff populating them. I have already made the terrain generator, problem is, how do I go about making all of these terrain maps without killing any computer that starts the game?

Here is what one of the terrain chunks looks like:

So basically I would need 3600 of those chunks. Quite a ridiculous amount if I do say so myself. I know it is possibly because I have played Just Cause 2, which had a map more complex, and larger, then the one I am setting out to create. So at long last my question is, how would you begin to optimize something like this? To my knowledge all I can do is make so you don't draw the other chunks, but the memory strain from their existence alone would be incredibly lag inducing.