Oh, huh my mistake. I was thinking about how PNG compression is entirely based around using the previous pixels to predict the next one. But as far as I know, apng doesn't update the filters to use information from the previous frame. I guess you can use transparent pixels to compress pixels that don't change though.
Oh, yeah, it definitely resets the compression context between frames. But so does a video format.
No format that I'm aware of supports lossless compression that spans frames. Which leads to a funny situation where some images compress best by custom outputting a .gif or .png with no compression, and then zipping it.
As I understand it all video compression uses delta compression between frames. And you mention another issue I forgot, that it can't reuse Huffman codes or any common segments between frames. Since they are independently compressed.