Filling gaps in baseball tracking when the object detector misses frames

my best friend and I have been working on tracking a hit baseball from a single side-view 240 FPS camera. the left side is the original footage. the blue boxes are frames where our object-tracking model successfully detected the baseball. as you can see, there are a lot of gaps. the right side is a temporal frame-differencing pipeline we tuned specifically for tracking a hit baseball from this camera angle. what surprised me is how clearly the baseball separates from the scene temporally. even when the object model misses it, the temporal tracker can often keep following it very consistently.