Hacker News new | past | comments | ask | show | jobs | submit login

One possible extension of this is to erase the original subtitles using something like gimp-resynthesizer's Heal Selection, and then replace them with translated ones, all automatically. (I've redrawn the video frame from the post[1] so you can see what I mean.)

[1]: https://static.linestarve.com/ext/ycombinator-news/itm144408...




Using a 2d image resynthesis algorithm is sub-optimal for video. There is basically no chance of it picking the same result every single frame, you will see a flickering shadow of weirdness where the old characters were.

You have lots of infomation of what image should be there; You have motion vectors signaling when an object or background has moved into (or out of) a blocked area. Or a maybe the motion vectors indicate object/background is stationary and you can use infomation from before/after the subtitles to fill it in.

I'm not aware of such a 3d video resynthesis algorithm existing, but it should be possible.


It's called video inpainting. I'm not familiar with it, but here's one paper with video examples I found: http://perso.telecom-paristech.fr/~gousseau/video_inpainting...


Google Translate is able to do this if you point it at live video.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: