I'd like to simulate a fish.

What my original idea was, to setup two flat hinged rigid bodies underneath a cloth object, connecting them with a hinge. But the main object doesn't move when the flat parts move. What is the best collider/ rigidbody/ hinge/ hierarchy combo. I tried different combinations, but I can't one that works.