Edit: BTW on your picture it's not (50,145) but (50,415) for the bottom left corner.
Edit: if quvi supports still works, you can ffplay "https://www.youtube.com/watch?v=rMXD5DxbXog" -vf perspective=60:90:589:147:50:415:582:418
(otherwise something similar should be doable with youtube-dl)
Edit: you can also do better and enable the perspective only in the part where it matters. Typically with something like -vf "perspective=60:90:589:147:50:415:582:418:enable='not(between(t,0,5))'"
Completely unrelated, but what do you think about vips (http://www.vips.ecs.soton.ac.uk/)? I recently discovered the difference in performance between im and gm and that let me to investigate vips, which is presumably (a lot) faster than both.
In principle, it looks promising (in particular, using ORC to compile image manipulation kernels to SIMD), but it is a different thing to im or gm. It's a library and a GUI. The library is slightly too low-level for most things I'd use gm for, but a GUI is too laborious.
Perhaps it could be integrated into GM or something similar, and used as a back end for certain operations.
If you're dumb like me and don't realize what's going on at first: the source clip was of some guys talking about a game being shown on a video monitor at an angle. But he just wanted to see the game, not the guys. So he extracted the video and made it fullscreen. He did it by writing a script to turn every frame into a PNG, run them through an image-processing tool, then recompress a new video. Thankfully the monitor did not move much so some fixed distortion parameters worked.
Thankfully the monitor did not move much so some fixed distortion parameters worked.
Even if the monitor moved alot (http://hyperboleandahalf.blogspot.com/2010/04/alot-is-better...) it'd be fairly easy to write an algorithm to detect the four corners of the video. You basically want to throw some edge detection at it, and then look for anything that seem like corners.
> You basically want to throw some edge detection at it, and then look for anything that seem like corners.
Which, if you're a web dev like I am, seems scary at first, especially when you're (like me) lacking a CS education. However, as it turns out, understanding edge detection and Haar cascades (for feature detection, that was the problem I was solving) enough to be dangerous with OpenCV is surprisingly easy! I recently built some facial feature detection stuff in it that is in production right now, and it only took me a couple weeks :) So, have a play!
With some fiddling in matrices this could also be done live in the browser as a CSS 3D transform using matrix3d or in a WebGL context on the iframe or video element.
Have no time to do the rectifying math on the components today but basically it would be some kind of inverse of this projective transformation:
I have played a little bit with OpenCV in the past (http://silveiraneto.net/tag/opencv/) but I wouldn't know how to do it with OpenCV, while with ImageMagick it seems simple. I'd love to see an example with OpenCV as I could use it one day in real time with the camera input.
I always assumed OpenCV was about computer vision, and I'm (pleasantly!) surprised it's this extensive. Does this mean it's good as a general-purpose image processing library? Not a fan of Imagemagick and have been looking for something better and faster.
if the camara and tv are stationary, sure. The most interest is being able to calculate the transformation and then apply in faster than realtime - especially for AR/VR applications as you can move around and it would still work.
Aw, when I clicked this I expected it to be something much cooler: taking a video of one of the old, 2D-tile-based Zelda games, and then doing recognition on the video against the game's tile "alphabet" in order to correct for both perspective and noise. (Basically, a crude implementation of a live-video to machinima converter.)
If you're interested in ImageMagick take a look at Fred's ImageMagick Scripts (link below). There is some really interesting stuff there. I spent a good 2 hours on Friday evening just trying to digest some of it.
With someone else mentioning Avisynth here in the comments (which is actually still used widely by video enthusiasts today), I became curious if there was a plugin for this sort of thing, and sure enough, I found Reformer[1]. With that, I figured I'd try reproducing OP's results. (By the way, while Avisynth is Windows software, as far as I know it works quite well under Wine).
Step 1: Download the video with youtube-dl. This gives us zelda.mp4 (original video) and zelda.m4a (extracted audio, requires ffmpeg)
Step 2: Write the Avisynth script (in AvsPmod). Plugins used: LSMASHSource, Reformer, RemapFrames
lwlibavvideosource("zelda.mp4")
deskewed = q2r(last,"lanczos",60,90,51,412,588,148,581,417)
normal = "[0 149] [1488 2063]" # frame ranges where to use original video
ReplaceFramesSimple(deskewed,last,mappings=normal)
Plugins used: LSMASHSource, Reformer, RemapFrames
Step 3: Pipe it to x264 with avs2yuv (necessary under Wine and with 32-bit Avisynth & 64-bit x264) to encode the video.
Step 4: Merge encoded video and original audio back together with mkvmerge from mkvtoolnix. You could use ffmpeg here as well, but I find mkvmerge much nicer for simple muxing like this.
mkvmerge -o zelda_muxed.mkv zelda.mkv zelda.m4a
And with that, we're done. The whole process took about 20 minutes (of which ~13min was spent encoding the video) and a few hundred megabytes of space (since there's no need to have all the frames as individual image files several times over). Other benefits include having the video run at the original framerate of 29.970 (OP's video runs at 25.000 since he forgot to set the framerate when encoding the processed images), including the audio as well not having distorted picture when the TV isn't visible (which was simple enough to do with the ReplaceFramesSimple function from RemapFrames). You can see the end result here:
As a fan of AviSynth, the scripts that I build often make that line look so, so simple. :) And you can use ffmpeg (and a couple of other tools) in VirtualDub 1.10+ to directly render to MP4 with no interstitial file (it'll handle all the pipelining for you). They're all useful tools.
>As a fan of AviSynth, the scripts that I build often make that line look so, so simple. :)
And now imagine what those scripts would look like as an ffmpeg -vf command! That was basically the point - the -vf line is already pretty messy with just one range-applied command, and would become even more so if you started doing something more complicated. Avisynth on the other hand has actual scripts for the video processing, which scales to much more complex processing while still remaining accessible.
ffmpeg -vf might be good for doing one or two simple things to the whole video, but for anything more complex than that you really should use an actual video processing solution instead.
Back in the day there was AviSynth to do this kind of thing. Unfortunately the version 3 rewrite which was supposed to use GStreamer and Ruby never went anywhere. Is there finally something similar for Linux?
VapourSynth[1] is what the AviSynth rewrite was supposed to be from what I have gathered. Hence the name ;). OTOH, it is still for the Windows platform, so not really what you were after but I thought I'd mention it anyway.
Hi, I'm the author of the post. I've been using JDownloader on Linux for years and never noticed a problem with Adware. Can you please elaborate? I'd really like to know more.
It seems to occur only using the web-installer, which is the one you're offically prompted at by the creators of JDownloader.
Well, I removed JDownloader and manually fixed the start page and search engines of firefox to remove the sweet-page stuff, but somehow my firefox is still screwed.
Questionable decision by the maker of JDownloader. I can't imagine that the money he gets for this is worth the damaged reputation.
Edit: I just ran an adware scanner. Besides the Sweet Page stuff, WindowsManagerProtect and Hold Page was installed. Several Registry Entries and Firefox setting were infected. Unbelievable ...
Since you didn't give alternatives, allow me: the program you should use is called quvi. It comes with a large number of scripts for extracting videos from sites and you can easily write more or patch existing ones, since they're all written in lua.
Also, a while ago I was working on making JDownloader less terrible (it should work like wget, UNIX philosophy and all). It's very labor-intensive though, but I may continue if enough people are interested (or help out): https://github.com/espes/jdget
My favorite us of this technique was at Hack Princeton a while back. Some people made an app that lets you take pictures of blackboards, automatically cropping and fixing the perspective of the board for later use.
I don't know if it was based on that, but a co-worker had a point-and-shoot digital camera a few years back (2009~2010) that had a similar feature for taking photos of black-/white-boards
For windows users (and with simple video like this one)you can do that in two minutes.
Drop the video in Virtuadub 1.10.4 and use it's buit-in "perspective" filter.
Using Blend mode and curves editor, you can only apply the filter to the parts of the video you want, so the part when gamepad is shown is not affected. (tutorial is here: https://www.youtube.com/watch?v=2MWoVY9mYbk)
Or OpenCV even, which is open source. So no, I don't think the plus of the article is that it's avoiding the use of closed source. If you read some of the comments by the poster, you'll see that it's simply him solving his problem using the tools he knew and spliced together.
This seems a bit tortuous. You can do this sort of thing in Blender or After Effects (even very old versions that you can get cheap or free) very easily, and skip the PNG conversion stage altogether.
ffmpeg -i zelda_720p.mp4 -vf perspective=60:90:589:147:50:415:582:418 output.mp4
Edit: BTW on your picture it's not (50,145) but (50,415) for the bottom left corner.
Edit: if quvi supports still works, you can ffplay "https://www.youtube.com/watch?v=rMXD5DxbXog" -vf perspective=60:90:589:147:50:415:582:418 (otherwise something similar should be doable with youtube-dl)
Edit: you can also do better and enable the perspective only in the part where it matters. Typically with something like -vf "perspective=60:90:589:147:50:415:582:418:enable='not(between(t,0,5))'"