I wrote some AI Upscaling Neural Networks from Anime4K in WebGPU, turned it both into an open source SDK (websr.dev) and a free tool to upscale videos. Nowhere near as good as well known AI Upscaling models like ESRGan, but ESRGAN would never in a million years work on a chromebook at 30fps
I trained a Machine Learning model which groups countries by how similar their voting patterns are in the UN General Assembly (graphs, and link to source code in post).
There are already vector-graphics runtimes like WebGL and SVG that are more than capable of rendering "video", even within the html video tag, and through modern video streaming architectures like Dash and HLS (will share the link to the talk, and demo links for these things next week).
Those aren't "codecs" in the traditional sense, and I think there is an open question as to whether a "codec" is even necessary. Scrimba (https://scrimba.com/) uses "HTML" as a video codec just fine in production, and it works perfectly, and there's no "codec" per se behind it.
That said - in 2020, web architectures already exist in a way that you could easily make a "video codec" for vector graphics, and some standardization would help - thought not specifically necessary for adoption in the way it would be for a "regular" video codec which doesn't just enjoy native vector graphics runtimes like SVG or WebGL.
If you needed a file format, I honestly think that Lottie is the best option for a "file format" for vector-video, since it's open source, and mostly just based on the old Flash standard (https://www.adobe.com/devnet/swf.html)
What's missing from Lottie for it to be a "video" format would be easier integration into video streaming architectures like HLS or DASH, and that's honestly something I'd like to do as an open source project - essentially a way for video players to "play" lottie video files as one of the video options (like, on top of 1080p, 720p and 480p versions, you have the vector version as well)
From the perspective of Vectorly, we clearly understand that there's a lot of skepticism around the idea of a new "codec", and we wanted to avoid the idea that we are actually building a new codec.
Our preferred framing is this: There are already "codecs" for vector graphics that are open, and as well established as H264 is. We're just working on a converter / transcoder from raster to vector, which admittedly will always have artifacts of some kind - though you could just as easily take source vector files and stream them and transcode them to a vector codec (SVG or WebGL) without doing that kind of raster to vector conversion that we're proposing.
Good luck with the company, I think what you're doing is very cool and I can see the use case for low bandwidth conversion for educational/whiteboard video in particular.
I think you're right about current web tech being able to support this. Lottie extended to an open source vector "video" format would be awesome, with HLS/Dash streaming and especially audio (streaming in sync) too. Hope you can find a sponsor for such an open source project
It would be useful to detect browsers that don't work at all (e.g. Safari) and at least warn that it's coming. Right now it doesn't inspire much confidence...
