Animating physics to be run in the background

I want to have a cube building collapse in the background of my scene, but I would like to make them into an animation rather than giving them actual physics components since the player has no interactions with them, and this way I can always know how it will look.

I made a scene with them, recorded all the transforms (key framing essentially) at 24 times a second(Thats what blender uses for its animation I believe), and just lapsed through them using a script, but it just feels like a badly implemented hack, so I was wondering if its possible to make a proper animation that can run through a normal animator component.

I also tried animating it in Blender, but for one I suck at physics animations in blender (or at blender in general) So if anyone knows anything about this your help would be appreciated.

P.S. I couldn't find anything by googling it, all I find is blender animations