What is the most performant way to handle huge List containing data to frame-sync Objects to 360 video?

In my current project a 360 video is playing. In AfterEffects we traced the appearance of certain elements in the video and I converted the tracked position in the video to world space. When a tracked element appears we show a caption pointing to the element in the video.

We have tracked 6 elements that are visible at certain points in the video. There are around 16500 frames in the video. Currently I store the world position in a json file. When starting the applicaton I parse the file and store the value in a six lists which overall contain 99000 entries. Each element has a default position to mark it as invisible (Vector3.zero), so I store a value for each frame.

While the video is playing I get its current frame and check if one of the elements should be currently shown at the stored position.

My question is: is there a more performant or elegant way to handle such a huge amount of data?