I'm not sure I understand how the author compares the quality in the preprint.
In the chart, it says to compare "perceptual quality", but the axis is only marked with "blurry" and "less blurry". Sharpness is not the only thing about the (perceptual or not) quality. I can tell that Anime4K's result is indeed very sharp, but the quality of the edges/lines are very unnatural even for the examples author provided. I personally would prefer a slightly blurry lines with less "oily effect".
Also, I didn't see any comparison with ground truth, i.e. having a high-resolution image first, resize it down, use the proposed algorithms (among existing ones) to upscale it back, and then compare the upscaled results with the original image. I understand it may be hard to find enough examples of 4k animes, but we can do so with 1080p -> 480p -> 1080p etc.
(I am not familiar with this domain, do similar researches normally do this or not in their analysis?)
There is no ground truth because AFAIK there are no native 4k anime produced yet. There are _very_ few produced at 1080p. Most 1080p that are released by studios are just an upscaled 720p master. Fansubbers will sometimes release their own upscaled 720p when the studio upscale was done very poorly.
To my knowledge, not much has changed since 2017 where only a single anime (Clockwork Planet) was produced in 1080p. The only two studios I can name offhand that I know have done 1080p masters are KyoAni and JC Staff.
Excuse my ignorance, I would have thought Anime would be much easier to be produced in 4K or even 8K when compared to movies that requires 4K / 8K Camera.
It is actually harder to produce in 4K/8K due to more details that need to be drawn to not make it looked too empty and need to make sure the lines are not too thick (e.g. by using a larger paper). TV series are usually drawn on an A4 paper with 1-2 inch margin while a proper theatrical releases are drawn on a B4 paper.
Another factor, I believe, is the know-how. In my opinion, despite anime being broadcasted in 16:9 for so long, it is only in recent years where the extra width are put in a good use during layouting.
A reasonable substitute for ground truth might be to get a single drawing done in an anime style with the appropriate pixel dimensions. There are many artists out there who can produce such a thing for a modest fee.
> Interesting enough, waifu2x performed very poorly on anime. A plausible explaination is that the network was simply not trained to upscale these types of images. Usually anime style art have sharper lines and contain much more small details/textures compared to anime. The distribution of images used to train waifu2x must have been mostly art images from sites like DevianArt/Danbooru/Pixiv, and not anime.
Your Name and the 2 Gundam Thunderbolt movies have 4K BDs, but I don't know if they are upscaled or native 4K. There's also a Gundam F91 and Space Adventure Cobra movie 4K remaster, but those were animated with cells, so I don't know how useful those would be for testing an upscaler.
There is a 4k remaster of Akira being produced from the original 35mm film that should be out next year. 35mm to 4k should be a downscaling, so that might do when it becomes available.
Not sure how the myth that anime isn't made at 1080p perpetuates itself, but it's not true. It only holds true for most anime made between ~2000 and ~2007, because those were made already digitally, but with DVD in mind. Anything prior is hand drawn on cells, anything after is done in 1080p or above, with some exceptions. There are a couple of 4K anime already out there.
The list I provided was from 2017 and only a single anime of an entire season was mastered at 1080p. Even in 2018 and 2019 that's still typical. Maybe 1-2 series produced at 1080p with every other series being an upscale. Some seasons even have 0 series produced at 1080p. Instead of calling it a myth, try to find a season where a large number of anime were produced at 1080p as counter-evidence.
Being upscaled and released on DVD or BluRay at 1080p (which most anime have been for most of the past decade) is not the same as being produced at 1080p.
I wasn't aware of the two Gundam movies mentioned by fireattack, but I can't confirm they were mastered at 4k and aren't just upscales. So if you could name some of those 4k releases that would be helpful, especially if you can provide information as to them being mastered at 4k and not just upscaled to 4k.
This is not correct. Many productions are produced at in-between resolutions between 720p and 1080p and then upscaled to 1080p. It's even in the link you provided.
The point is they are upscaled to 1080p and few are produced at 1080p. Yes, I didn't label every in-between resolution that they get produced at. This seems rather needlessly pedantic, since as you cited, the resolutions are in the link I provided.
Would you feel better if I said "they upscale 720p and 837p and 900p and 810p and 806p and 873p and 864p and 957p and 878p and 719p to 1080p"? I excluded non-standard resolutions for simplicity since it doesn't really change my greater point: most 1080p releases are just upscales. 19 of 41 listed are 720p and 720p is the most common resolution listed.
I'm in the same boat. There are different metrics to judge the quality of an upscale (peak signal to noise ratio comes to mind), but it's obvious they are limited in that they can't capture perceptual quality very closely. While it's obvious the filter is much sharper than even NGU sharp, it also seems to come with a weird gradient effect and some artifacting. Another thing I find is that sharper filters like NGU sharp don't upscale as well as other upscaling options on frames with edges that aren't supposed to be sharp, probably because in some sense they try to hard ink in parts that aren't supposed to be so. This can happen either because the source is composited to be blurry for artistic effect, or because the source is low quality. I admittedly have not tried Anime4K, but I imagine Anime4K will have a similar effect.
Of course, in the end, it's an entirely subjective thing. Personally I hold off using on using NGU sharp and use NGU Anti-Alias instead for the above reasons.
EDIT: this is addressed in the readme:
I think the results are worse!
-Surely some people like sharper edges, some like softer ones. Do try it yourself on a few anime before reaching a definite conclusion. People tend to prefer sharper edges. Also, seeing the comparisons on a 1080p screen is not representative of the final results on a 4K screen, the pixel density and sharpness of the final image is simply not comparable.
EDIT: I just tried this filter on a 4k monitor. To honest I don't think is very good. To me it reminds me of the bad parts of sharpeners turned up to the max. All the edges turn into a weird, sometimes jagged, smear, and originally blurry but detailed backgrounds just become a weird mess. I really don't think even people who like sharpness will prefer this filter for general viewing, and I find the chart given in the preprint (https://raw.githubusercontent.com/bloc97/Anime4K/master/resu...) extremely dubious.
Edges seem pretty reasonable to me in the 1:1 image. I'd dramatically prefer to watch shows with the sharpness of the post-processed image. This is a taste thing, and different folks will have different strokes.
In the chart, it says to compare "perceptual quality", but the axis is only marked with "blurry" and "less blurry". Sharpness is not the only thing about the (perceptual or not) quality. I can tell that Anime4K's result is indeed very sharp, but the quality of the edges/lines are very unnatural even for the examples author provided. I personally would prefer a slightly blurry lines with less "oily effect".
Also, I didn't see any comparison with ground truth, i.e. having a high-resolution image first, resize it down, use the proposed algorithms (among existing ones) to upscale it back, and then compare the upscaled results with the original image. I understand it may be hard to find enough examples of 4k animes, but we can do so with 1080p -> 480p -> 1080p etc.
(I am not familiar with this domain, do similar researches normally do this or not in their analysis?)