2D displacement maps?

I'm trying to find a way to implement a 2D displacement map in Unity, but my terminology is failing me, since in 3D "displacement map" doesn't quite mean the same thing as it does in 2D.

Essentially, what I'm looking for is something like a heat distortion effect, or simple fake refraction--it's a 2D distort on a texture based on a grayscale image.

If you apply a displacement of, say, -100 horizontal, any white pixels in the grayscale image would move the corresponding pixels in the texture -100, while black pixels would result in the pixels moving +100. Pure gray pixels wouldn't move at all. It's not quite that simple, because you need the result to not be a crazy broken aliased mess--it's a distort, not a direct pixel remapping--but you get the idea.

Of course, any search for "displacement map" is sending me in the wrong direction. For all I know there's a really simple solution and I just don't know the right term. I've looked at stuff for heat distorts, but they're usually full-screen image effects, and I want to apply this to a specific texture on a plane.