Affecting angular momentum on bounce

I'm looking to create a trampoline like bounce when the player (top rectangle) hits the "trampolines" (bottom rectangles).

I have created the proper velocity bounces using Vector3.Reflect but this doesn't accomplish what I'm looking for. My original solution was simply

 angularVelocity = -angularVelocity;

...but as you can see in the example below, when the rotation is "away" from the bounce per se it should increase the rotation rather than reverse it.

I would like to avoid using unity physics for this to retain complete control and to provide myself with an educational opportunity but I'm a bit stuck here. Is there a way to find the relative velocity of a contact point or perhaps a more elegant way to approach this? Thank you.