Selectively deform a mesh by dragging?

I am trying to create a teaching and design tool. The idea is that the player adds the initial template (for example as in picture 1) to the screen. I would like the player to be able to deform the object by dragging various parts of the object. For example the size of the object can be changed by dragging one of the sides (picture 2). The shape of the object can be changed by dragging one of the corners (picture 3).

I am just very new working directly with meshes.

I cannot figure out what is the best way of approaching the problem. If I make each part (the four sides and the four corners) a separate game object and attach a DragObject script to it I can drag each of the parts, but because they are separate it will take quite involved logic to get the meshes of the adjacent parts to deform as the selected part is moved.

If I make all the parts into one game object with a single mesh I cannot figure out how to just selectively drag a part of the mesh (for example clicking on the left side, just the left side must be active for dragging).

Has anyone done something like this already?

Picture: