Unsurprisingly, Apple goes for the patent-encumbered video format [1]. I'm disappointed that Firefox never really had the opportunity to "win" the browser wars and push more user- and developer-friendly formats on the world.
Instead, Chrome "won" the browser wars and pushed a different, incompatible format -- VP8 -- which [1] they bought from a private company on the world; a format they quickly deployed across Google properties, but took years to document.
Further, they claimed VP8 infringed no patents but refused to indemnify users against patent suits; the MPEG LA began forming a patent pool on behalf of companies that claimed to hold patents on the techniques in use in VP8 and the situation didn't clear up until 2013, when Google essentially paid the license fees for everyone and agreed to sublicense these out [2].
They continued evolving the bitstream with incompatible revisions, which haven't even been finalized [3]. Nonetheless, this VP9 is widely deployed in Google properties today.
In the future, a consensus codec AV1 developed by a consortium of companies will be ready for public consumption; this codec descends from VP9 while incorporating significant enhancements from Cisco and Xiph.Org. The participant companies are numerous and include big chipmakers and content delivery networks, banding together to perpetuate a royalty-free video codec that mounts a competition against MPEG's efforts, whose rules don't exclude techniques that are patented. Until AV1 is ready and good, MPEG formats form a superior choice on technical merit, as well as rigor in documentation should you choose to make an independent implementation.
To be fair, VP9 and WebM both may as well be considered de-facto proprietary formats. Chrome is almost certainly the majority browser and youtube is likely to be the most used video platform. Sending video from youtube to Chrome is probably the main use-case for that protocol. I'm not even aware of any commercial video platforms that offer VP9 as anything more than a novelty option (although maybe Anvato will now that they are slowly being integrated further into the Google universe). Even DASH is years away from real wide-spread adoption.
IMO, video formats will live and die based on hardware support. If your mobile device doesn't come with a AV1 hardware decoder that works with the majority of players out-of-the-box than chances are streaming services won't support it. Apple's advantage with HEVC adoption will be their hardware.
>To be fair, VP9 and WebM both may as well be considered de-facto proprietary formats.
What about how they're used makes them "proprietary"? If there's open source implementations of them that aren't patent-encumbered, then that's terrific.
I think the point is that they can be useful formats even if they aren't adopted outside of Google simply because YouTube -> Chrome is a large portion of internet video. Most other video formats are seen as failures if not widely adopted.
This is to support sites like imgur, reddit that are essentially distributing GIFs as MP4s now. They didn't really determine the format that the community adopted
To be quite fair, the only reason the community adopted these is because it's the only format that works across all browsers. If Safari and IE had supported free video codecs they would have very likely supported that.
By the way, MP4 is a completely open standard (open does not equate 'free') and is free to use for videos that are also freely distributed. Of course there are also free and open source implementations of the codec. Being patent encumbered sucks but in principle there should be no practical limitation for gifs at least.
mp4 is a container. It typically contains an audio stream and a video stream. The streams are encoded and the majority of the formats are patented with license fees.
I've never really understood this. I previously had figured that the popularity of GIFs on reddit had been about bandwidth and load times. In that past, video would be too slow for the rapid novelty stimulus reddit offered, while GIFs were lightening. But now every "GIF" people post is a just full movie, and it loads more slowly than YouTube or other websites that are smartly structured to handle video.
It definitely makes sense technically. It should in most cases reduce file size, and also allows you to stream content in buffered so you don't need to wait for the whole thing to load for it to play. (Though I believe streaming in gifs is teeeechnically possible? I feel like I saw a demo one time that made a clock by streaming in gif frames)
Hey thanks, this is super useful! I still don't quite get it though. I understand that converting huge HD GIFs to a standard video format allows steaming and better compression, but why encourage people to upload it like this rather than just upload videos?
The only explanation in the post is "the culture of the GIF now trumps the file format. With Project GIFV, Imgur is reimagining the looping GIF video with all the richness it deserves as a key piece of Internet culture." Is the idea just that people want soundless looping HD video, and they're accustomed to uploading GIFs, so Imgur is going to accommodate them by doing the conversions server-side?
I guess this is an explanation, but it's a surprising one. For instance, it doesn't really fit well with the observation that people uploaded GIFs of content that was obviously native video and not connected to internet GIF culture, e.g., sports videos.
Yes it's all about no sound. Everyone likes opening up a bunch of tabs that include auto-playing funny looping videos, but nobody likes auto-playing sound, especially if it overlaps with some other sound.
I'm not sure if it is Chrome's doing or YouTube's, but if I open a couple of youtube links in tabs (a selection from the "related" list to the right of one I'm currently watching for instance) the clips in the new tabs don't start playing until that tab gains focus. This avoids the auto-playing sound overlapping problem and means you don't miss the start of the clip and drop in part way through. Surely this is the solution to that selection of requirements?
I've noticed the Chrome feature as well. I watch a lot of Twitch streams, and it causes a frustrating experience, albeit with possible workarounds by Twitch. It's that Twitch is about live streams and live chat, but now the video autoplay is delayed until I activate the tab, but chat is still live from the moment I clicked on the link. It basically buffers the stream now, but makes me go out of sync with chat. One alleviator could be Twitch adding a "go live" button that shows up when I'm not live.
There are still two classes of problems unaffected by this change.
1) As is popular with gifs, there could very well be multiple of these autoplaying videos on the same page, probably even multiple fitting in the same viewport area. If they had sound, these would interfere with eachother.
2) My understanding is that many listen to music from some source unrelated to the webpage containing these autoplaying videos. So even if there's only one video, its autoplaying sound could easily mix with my tunes.
Doing autoplaying sound is hard, but with gaze tracking and knowledge of system sound usage status, I think a pretty good solution is possible.
Yea, it is such weird behavior. I can count the number of times I've ever wanted to adjust the volume of the ringer, rather than just turn it on/off, on one hand.
Luckily on newer version of Android there is always an option to quickly access the media volume after you press the volume button once, but it's still unintuitive.
Windows phone nailed this. As you change volume, a panel comes in with two sliders: one media, one ringer, so if you were in the wrong context and changed the wrong volume, you right away have access to the other one. I find (on iOS now) I fiddle with the ringer volume a lot throughout the day. Loudest possible is waaaay too loud for me, unless I’m in a noisy place.
It doesn't have much to do with the gif/jpg formats though, it works with anything. It's just a multipart HTTP response that keeps sending entire new files to replace the previous one. It's not even a jpeg/gif encoding of the differences between frames which would be much more efficient.
Part of the popularity of GIFs is that editing them is easier. You can edit them in photoshop or gimp or other image editors, which allow you to use skills learned for editing images (like putting those large black and white texts on top of an image) with it.
Video require more technical skill and different software people would have to learn (and time in case you need to edit it frame by frame).
What I find nonsensical is that imgur only lets you upload gif which it then converts to video instead of allowing you to upload video directly.
FYI you can edit videos in Photoshop and paint on every individual frame. I used they feature back in 2005? (might be bad memory). In any case just opened an mp4 in Photoshop to make sure I wasn't crazy.
Most gifs these days is via custom tools that will basically rip the frames and assemble the gif. Just give it a start and stop time, a video file, and grab a latte...
In case I don't have a source video file, one of my personal favourites is peek [1]. A small gtk app that allows you to quickly record a portion of your screen and save as a GIF.
The compression MP4 (or any other modern video format) offers is much better than what you can obtain with GIF. MP4s are, perhaps surprisingly, actually the better choice in terms of bandwidth and load time.
Yes, the same HD content is better as MP4 than GIF, but in the past a "typical" GIF loaded much faster than a typical video. It may even be true (any experts?) that there is an effective lower limit on video quality/framerate for MP4 that prevents it from being the super fast novelty juice that a low-quality GIF could be.
GIF is expensive to transport but cheap to decode. Compression that is in a sense more appropriate for video is the other way around. You may blow through a data cap (people still have these) quickly with animated GIFs, but at least you can put a hundred of them on a web page and have them all play. Eventually. If the downloads ever finish.
What he is talking about is that back when gifs and videos were both postage stamps pushed over modems, gifs won out.
But what changed was that people simply started ripping frames from HD videos, dumping them into gifs and plastering them all over social media.
End result was that the gifs ballooned in size because they now held many more images, and each images was much higher resolution.
What is more wacky is why gifs returned to fashion at all. They were dead for nearly a decade after people stopped doing their own web sites, and used gifs for things like animated "under construction" signs.
> What is more wacky is why gifs returned to fashion at all. They were dead for nearly a decade after people stopped doing their own web sites, and used gifs for things like animated "under construction" signs.
Same reason that H.264 won the web video standards war: mobile.
While mobile browsers can now embed videos reliably that wasn't always the case, and if there's one thing mobile users hate it's links opening other apps. GIFs allowed "video" content to be displayed inline easily in a mostly universal fashion.
- Many websites support embedding images in user content but not videos (Gmail, Hangouts, and Stack Overflow, to name a few).
- A "minimum viable animated thing" tag is more complicated for videos:
<video autoplay loop muted src=video.mp4>
- Supporting videos in img tags and sending the right Accept header means that image hosting sites and CDNs can immediately start serving video files to browsers that support them, without the page needing to change. This applies to every hotlinked gif out there.
The UX of short, looping, soundless, autoplaying videos without player controls is popular. This is partially for simplicity and consistency. Gifs have been less efficient than modern compressed video for a long time.
Desktop too. The absolute main feature of such a site is the absolute link to the picture. That is the most important thing you need after you uploaded an image. Yet imgur tries to hide it as much as possible, forcing you to use the "show url to image" stuff in browsers.
It's been some time since that was the main feature of imgur.
For example, imgur redirects hotlinked images to its embedded view in some cases. One of those cases is when they decide that your domain is using them as a CDN, which they forbid.
H.264 has hardware support and is already supported by the video tag. It was a battle during the video tag debate on WHAT-WG, but the decision was made a long time ago.
This I can deal with, everyone else has been supporting this for ages. The bigger question for me is whether Apple will support AV1 like the rest of the industry or skip over it like they did with VP9.
Patents help pay for research and keeps a lot of smart and hard working tech people employed. Stop bad mouthing it as though you are losing one of your God given rights. There are surely issues with greedy companies doing greedy things with it. But then there are issues with everything... like greedy Ad companies selling your profiles and collecting more data in the name of AI.
A good patent brings us a more competitive solution to a problem than we otherwise would have had. A bad patent is a land grab, a punitive unearned tax on the entire problem, and those have been getting worse.
Not sure why you are downvoted, ( may be because of the tone and languages used ). But people from the academics, and companies do spend a lot of money on R&D in video encoding. These patents are not those silly one click web purchase patents, there are lots of thoughts, experiment into it. I would have thought anyone who has been to Uni / College would know this, after all it is part of how University gets funding these days.
The problem is price. What should these patents cost.
I don't see why the "issues" with the <video> tag couldn't be fixed in the video tag?
"Autoplay video is used for ads and thus controlled by browsers" (and <img> video won't have the same development?!) and "<video> can't be saved" (yes, if you use a JavaScript player that interferes with it, if you just link in a file through a <video> tag at least FF and Chrome do have a save option) seem especially nonsensical.
I guess there is some semantic argument for "videos that should act like images", but I somehow doubt that'll happen nicely...
the biggest challenge I've run into is big/legacy CMSes that simply can't get the templates updated to use <video> instead of <img>. It's why we can't have nice things.
> I don't see why the "issues" with the <video> tag couldn't be fixed in the video tag?
You could make that argument for a lot of html tags. <address>, <section>, <figure>. Perhaps this is the safari teams opening argument for a new tag to be added to the spec.
Yup, and Safari may be signaling that <video> doesn't provide the right semantics for the expected behavior. <img> probably isn't the right place either and a new tag might be necessary, but I'll bet the desired code path for the browsers is closer to the img implementation then the video one.
But a GIF was never an image, it's like now they are saying okay we called this video an "animated image" which is somehow different from a video so therefore all videos are images and should go in the image tag.
But in reality, it feels quite true to HTML. The HTML author is declaring to the browser the purpose of this asset. This one is to be viewed as an image. This other asset is to be viewed as a video.
The encoding format shouldn't dictate the tag, the content and intent should.
We've been okay with animated GIFs being images for long enough now that we've kind of opted into animated images being okay in general.
APNG is also a very good way to achieve that! No limits in the number of colors, better compression. No patents?
All major browsers now support APNG, Chrome being the last in the bunch :) For example, we use it in our README screencast for https://github.com/wallix/awless
Oh, huh my mistake. I was thinking about how PNG compression is entirely based around using the previous pixels to predict the next one. But as far as I know, apng doesn't update the filters to use information from the previous frame. I guess you can use transparent pixels to compress pixels that don't change though.
Oh, yeah, it definitely resets the compression context between frames. But so does a video format.
No format that I'm aware of supports lossless compression that spans frames. Which leads to a funny situation where some images compress best by custom outputting a .gif or .png with no compression, and then zipping it.
As I understand it all video compression uses delta compression between frames. And you mention another issue I forgot, that it can't reuse Huffman codes or any common segments between frames. Since they are independently compressed.
This usually happens when you have colour pallet increases. If you had a simple animated Gif with only 256 colours and convert it to any video you instantly have more data to compress. The better use case is where you start with video and convert down to gif (like when you use gif.ski)
I never had an issue with this... But if Safari does, perhaps they should have fixed that there?
> 3. Autoplay abuse
Mute the video or remove the audiotrack. If even this in the future gets prevented to autoplay, then I don't see how <img> tags wouldn't as well. After all, they're just becoming the same thing.
1. The issue here is what browsers are expecting with a <video> tag, which is long-form content and that's what they are optimizing for.
2. Depends on the browser and the player that is used.
3. Well, the <img> tag is by default much more limited, hence it is less of interested to the ad companies and the browsers allow it more freedom. But yes I agree that this is a point the browser developers should improve upon.
This page crashes on chromium before you even have the opportunity to read half of the article. I wonder if there is a memory leak or it's just the GIF implementation in chromium.
The crux of the problem seems to be where we encode the information "this video should loop".
We did it before in surrounding HTML structure. The proposal is to now do it different, slightly shorter, surrounding HTML structure. As far as I can see, there's no fundamental difference, just syntax sugar and a battleground reset.
How do we get the video to loop when viewed directly? When set as the background? When saved as a file? These are use cases that .gif satisfies and videos don't, or only awkwardly and inconsistently.
Also note that content is inherently suited or unsuited to looping. It's rarely useful to play a normal video on a loop, or a video meant to loop just once.
All this points to one solution: a new file extension that means "a video meant to be played in a loop". The format of the file need not be any different from a video. Maybe ".mp4l", ".webml", etc.
That would have some downsides. You to browser behavior to url in a way that never been done before, it doesn't really account for mime types or content encoding, it isn't backwards compatible at all, it doesn't allow you to use one resource in both ways, and it relies on duplicating all extension names 2x for all video formats and times no one unrelated will make a video formats ending in 'l'. If we must change this some combination of css and/or js would probably work better. Like 'img { repeat: none; }' for example
One of the reasons GIF's are popular is that it's a format with nearly universal support, with no codecs to fuss about. Anyone can see a GIF, and anyone can edit a GIF.
The proposed "improvement" offers the key limitations of GIF without the key advantages, and lacks many features.
Like, what about variable frame rate? Transparency? Not fuzzing out the pixels in pixel-art?
The last one specifically: a great use-case for GIF is capturing screen animations (see https://www.cockos.com/licecap/). Video codecs aren't optimized for this - you won't see the comparison for such GIF's on the linked webpage.
It's not supposed to replace GIFs for the things they're actually good at, like your examples. It's supposed to replace the abuse of GIFs for things they're not good at, like long, high frame rate, and/or high color depth videos.
The majority of the GIFs being produced today are not things the GIF format was designed to handle.
I've floated the idea of allowing various video formats in <source> for the <picture> element. Would allow for the right codec to be selected, and would easily allow for a <img src="foo.gif"> as the final fallback. Might be worth pushing on more.
Don't know if you need a spec change since this is technically now supported in Safari TP. (See the example) Really it's up to the browser to decide if it wants to use the media type or not.
The article mentions removing the audio track. Is that required for this to work or would it play if it's included or is that just for efficiency? i.e. will Safari simply ignore it if it's included?
An incomplete quote from the policy statement is below. I strongly encourage y'all to read the entire article, since there's more conditions than I'm pasting here, but these are the highlights that answer your question.
Incorrect. If an audio track is present, the mp4 does not qualify for autoplay unless other conditions are met.
EDIT: Context is unclear about whether the parent's guidance is about <video> or <img>. No idea what <img> does. I sure hope it doesn't play due to the wasteful audio track, so that people are forced to do it right instead of lazy, though!
Site is down for me so I have no idea if they addressed this in the article or not, but what's wrong with the `<video>` tag?
As a user, I greatly prefer that option since it lets me control playback (via the "show controls" menu option) whereas GIFs (and presumably also mp4 "images") don't.
the biggest draw back is the browser preloader and general performance. For cinemagraphs & clips this will be great. Anytime you want to add audio or pause/control playback, this will be the wrong solution.
> the biggest draw back is the browser preloader and general performance
So why can't we just fix that issue with the `<video>` tag? (Maybe with a `preload` attribute or something.) It just seems semantically incorrect to be putting videos in `<img>` tags (and yes, I realize this same argument applies to GIFs as well).
> Anytime you want to add audio or pause/control playback, this will be the wrong solution
I'm referring to me as a user showing controls even when the site has them hidden. Obviously as a developer I can always just choose to not use this feature and use a video tag instead.
WebP and APNG are both good formats instead of GIF. But the problem is that Chrome developers stubbornly do not want to add APNG support, while Firefox developers, also stubbornly, do not want to add WebP support. "Thanks" to both those teams we are forced to do such hacks...
Yes, the solution to that is to make opcodes for the special instructions typically available in hardware, e.g. discrete cosine transforms; or a more coarse grained approach if necessary. Eventually, it will be up to platform owners to provide translations from the bytecode to the underlying hardware.
[1] http://scratchpad.wikia.com/wiki/MPEG_patent_lists#H.264_pat... (the vast majority of the patents will expire by 2023, but 2027 is the last probable date that it will be encumbered)