Get the closest vertices in a specific area

I've been having a problem with this for a while now. The end goal is when I "build" an object on my terrain the terrain changes it vertices in the area below the object build to raise to fit the object better and make it look more natural.

In the picture above the grid is the vertices of the terrain. The red is the building I "built", and the blue is what vertices I need to get to be able to change them.