For YouTube, there are several click-to-play extensions that work with the HTML5 player. There's a Firefox preference media.autoplay.enabled that works for most HTML5 video other than YouTube; ideally that could be allowed on a per-site basis. For ads, use an ad blocker; they know how to block HTML5 ads just fine.
Particularly for mobile, the latest video-ad rendering is happening in JS [0]. This is because autoplay video is disabled by default on mobile, but video rendering in ASM JS / WebGL bypasses such restrictions. Yahoo [1] is doing this, for instance.
How can I block HTML5 videos and audios or at least stop them from playing automatically?