I'm not super clear on why speed was a primary goal if the intended application is upscaling anime. If this were intended for, say, sharpening the graphical output from a game console, sure, but why does premade video content like anime need upscaling that only takes 3ms instead of 6ms or even 60ms?
So you can simply have it as an option in a media player (as they indeed have theirs) instead of requiring a cumbersome preprocessing pass which will in addition produce a much larger file size.
Agreed. Being able to do it real time is definitely nice but I don't think it's very important. I'd rather optimize for quality.
FWIW I tried doing the same thing using waifu2x, but it was about one or two orders of magnitude too slow. I don't remember the details but I think it worked out to about 2 weeks of 24/7 operation on a 1070 to upscale a full show (don't remember if it was 1-cour or 2-cour) to 1080p. Results were okay, gave kind of an oily texture to it but the denoising worked quite well. If it took only a day or two to convert a full show I'd consider doing it on some old 480p shows with bad quality, though I probably would just watch the original video myself.
