It's still got all the noise and shadows frame-to-frame. It wouldn't be historically accurate, but it strikes me that an ML algo could probably smooth the noise and shadows, colorize, and even add frames into a more immersive video to bring Twain even closer to the viewer.
There's always the ethical and artistic arguments around colorizing and so forth, but if one of the values of old video reels is to connect our conscience to the past, it's my opinion that smoothing, colorizing, de-noising and so forth can be benefits in that goal, even if the frames are no longer "authentic". (It's not like the real Twain was choppy and black-and-white)
I'd like to see ML colorize, sharpen, etc., older B+W movies. Silent movies would be greatly improved with the addition of a soundtrack (music[2], foley effects[1], and replacing the dialog cards with voice actors).
1. Yes, they already do add some foley effects. But very little.
2. Yes, they do add a music soundtrack. But it's almost always awful period music.
> the ethical and artistic arguments
Those are all baloney. Doing what I suggest does not destroy the originals in any way. It's simply making the movies more enjoyable. Apparently, a lot of purists want people to suffer watching them: suffer with grainy, blotchy film, difficult to make out flickering black & white, and bad piano music.
There's a reddit newsgroup where people colorize old B+W photos. They really do an amazing job. By colorizing key frames in a movie, ML can take care of the rest.
BTW, I've seen many bad attempts at colorizing old footage, like those "WW2 In Color" documentaries. You can always tell because the teeth & gums of the people are grey :-/ and the faces are a uniform brown. ML ought to be able to do a lot better.
>Those are all baloney. Doing what I suggest does not destroy the originals in any way. It's simply making the movies more enjoyable. Apparently, a lot of purists want people to suffer watching them: suffer with grainy, blotchy film, difficult to make out flickering black & white, and bad piano music.
I think there's 2 separate issues actually: there's the "artist's intent", and then there's refusal to utilize any modern technology. The latter I agree is baloney: if you can use modern technology to clean up and improve old stuff (such as taking an original Star Wars film print and removing the scratches in the film, improving flaws in the FX, etc.), I think it's ridiculous to argue against that kind of thing. Then there's the artist's intent: if the original artist intended that the movie have "bad piano music" as you call it, should you be changing that? It's like changing the dialog in Lord of the Rings so the characters all talk like modern American hipsters: how is this helpful? These movies are a product of their time, and the music is too.
If you could bring the original artists back from the dead, and ask them if they'd like to use modern technology to clean up their old silent movies to sharpen them, remove the noise, etc., I'm sure they'd agree: why wouldn't they want the print to look as good as possible? If you ask them about colorizing, there might be some debate; some might like it, others might not. It's not like they had a choice back then, but even today some people like B&W photos for artistic reasons. If you ask them about adding modern music, I don't think they'd like that much at all.
How many movie arteests today would choose black and white? A number indistinguishable from zero. How many would choose dialog cards over voice? Zilch. How many would make a film and says to the movie theater "play any music you want for the show"? Zip.
If I made a movie, I'd be pleased if people far in the future watched it. If that meant using their technology to improve it, that's fine with me. After all, I've suggested to many people that they improve my game Empire with modern graphics and AI and sound.
Worth noting, among the “number indistinguishable from zero” are contemporary classics such as: Schindler’s List, Dead Man, Good Night & Good Luck, Ed Wood, Clerks, Raging Bull, Nebraska, et al…
Anyone who studied Media Theory will be well aware of the ubiquitous Marshall McLuhan quote, “The medium is the message,” and understands that the means with which a piece of media is produced and distributed tells a story in and of itself. Sometimes that’s in service to the broader narrative, and sometimes it’s simply a product of circumstance, but either way it carries its own value for the audience, and I would argue that it would be arrogant for one to unilaterally decide whether such things are important or not to preserve.
I don't think you can change a work that drastically without the change itself being a piece of art. And the original artists might not want to be left out of that process. See David Simon's experience with the HD remaster of the Wire for similar themes: https://davidsimon.com/the-wire-hd-with-videos/
> ... Apparently, a lot of purists want people to suffer watching them
The tech and information of the time is worth preserving, as is the experience. I, for one, enjoy seeing the original, more analog versions of things. I cringe inside when I watch much of ML filtered or otherwise artificially colorized footage. Many films were shot with the intent of leveraging the strengths of the medium. Limitation often promotes its own form of innovation which can only be recognized in the context it was created in.
Saying that purists want people to suffer is quite misleading. A similar statement in the other direction might be, ML enthusiasts want people to forget what art actually is.
Ever listen to a Gregorian chant? They are cool, partially because they had no idea what they were (musically) doing... much like the product of an ML model today.
> Saying that purists want people to suffer is quite misleading
Not at all. I've watched enough movies where the director clearly wanted the viewer to suffer, based on the idea that only through suffering can the viewer be enlightened. The suffering is in the form of making a bad movie.
Solaris comes to mind. I've tried to watch it multiple times, but the director (and the director's fans) clearly revel in the notion that one must suffer through the movie, and that's the point of it. The movie is boring on every level - bad dialog, bad plot, bad cinematography, bad pacing, bad acting, everything is bad about it. And it's (small) number of fervent fans love it for its wretchedness. They can keep it :-)
I saw a colorized version of Metropolis a few decades ago. The technology to do it right wasn't there, but it was enjoyable in a way the original was not. I've since watched the full version when the missing footage was found, and there's an ideal film that would benefit from the application of modern technology. I bet Fritz Lang would love it.
TCM also ran a version of the silent "Wings". They added some color in a few spots, and some airplane engine sounds. Even just that little bit made the picture much more watchable.
I'd like to see the FM Busby musicals fixed up.
Some movies I wouldn't touch would be Top Hat. It's perfect as it is :-)
> ...I'd ike to see ML colorize, sharpen, etc., older B+W movies.
In case of old media, I think their historical value is not in their own existence but in their effect on people of the time.
Older movies were perceived back in time through the old technology, that is b/w, small screen, choppy, silent or with musical accompaniment.
It's a document, similar to language dialects used back in time - localizing this for present time would shave away the cultural skin it had for its time.
I can recommend a film festival and silent movies with live bands.
If it works, it works great. For me the silent movie is always the highlight at the midnight sun film festival.
It's a bit closer to theater or reading a book. It's less directly realistic. Somehow your brain fills the gaps more and you get even more invested.
But if that ML tool is trained on modern video, depending on its capabilities it may subtly modernize aspects of the video in ways that lessen the "past is a foreign country" effect, leading us to believe that the past was more like the present than it in truth was. It's a trade-off between removing technological barriers between us and the person, so we can connect to their recorded legacy, and removing sociological barriers, which we probably want to preserve.
The past was more like the present than we like to believe. The past was no more sepia than it was black and white. Just like the 80s wasn’t as bold and neon as pop culture portrays.
In media we use colour to exaggerate the differences of the past, to make it feel different and foreign to our senses, rather than represent them accurately. So if you want a more accurate representation then you absolutely should be training your models on modern video.
If the colors are just someone guessing I would honestly not want to see it. I find those kind of wishful fantasies harmful. Not much different than when favorite book is converted to a movie and actors stomp out your imagined characters.
If you hadn’t had a picture of your grandma at all, would you want AI to generate one based on a description of her and tell you it’s a picture of your grandma?
There is a short movie on Google which is a film of a train rolling into a station, and then the same one sharpened by AI. The second one is marvelous to watch.
There's another of a colorized ancient movie of a trolley ride in Europe. It's very fun to watch.
Ya know, when people restore old cars, if they want to drive them, they'll usually do a few upgrades to make the car safer and more driveable. And there's nothing wrong with that.
I'm curious. Do you ever watch and enjoy movies from the 20's and 30's? For the silent flicks, do you think that the music played during the screening by some random starving artist plinking on a crummy piano or squeaking away on a violin is part of the "vision" of the director?
Lots of people today simply turn away from any movie that is black and white.
I’d like to see rolling data on how people are turning away from lower quality media. Or what premium is being placed on 720p+ video versus standard.
There’s a reason video sites offer a filter for HD and YouTube videos get keyword stuffed for 8k even when they aren’t.
I am very curious how Apple’s spatial video lands.
From reviewer descriptions, the experience is such a major change from standard HD video I am wondering if depthscaling “old” video will become a thing too.
Personally, I find the constant breaking of the flow in a silent movie so the dialog card can be shown very irritating after a while. It's also annoying that the card is shown for too long (for slow readers) and there's a fair amount of dialog you can lip read that is not on the cards.
I seriously doubt dialog cards are part of any director's "vision".
The bad music slapped on the DVDs is also a serious turnoff. Music easily makes or breaks a movie. For example, the music on Lord of the Rings adds greatly to my enjoyment of it. The Hobbit music is utterly forgettable, and makes the poor movie even worse.
For another example, the movie "Hawaii" has a most excellent music soundtrack. Its sequel, "The Hawaiians", has a completely forgettable soundtrack, which contributes to the movie falling flat on its face.
I suspect much of "Star Wars" success came from the music.
If the colors are just someone guessing I would honestly not want to see it. I find those kind of wishful fantasies harmful
I think disclosure is high priority here. If you know it's been cleaned up, and colourized, then you can watch it as you would the original.
Because, what I think people are saying here is that older cameras already have made the equivalent of "someone guessing". The framerate is variable and jumps, the colour is off and variable, rhe film is degraded, and even the people in the films act quite unnatural, for they are very aware of being filmed.
I get what you mean about authenticity, but I think full disclosure takes care of that.
I'll do this and let you know how it goes, I don't think I can easily add colour but I can work some other magic. ETA is likely 2-3 hours + whatever it takes to upload (assuming I get back to it tonight - otherwise I’ll upload tomorrow morning).
First attempt has processed at 1080p 60fps, uploaded to YouTube - it will take some time for YouTube to finish processing it in 1080p - https://youtu.be/hlhn9HSH4OI
I think I can do better though, so will try another method in the morning.
It’s quite hard with such low quality footage. I think could be done is use some sort of algorithm to stop that sort of wavy chemical look that might be beyond me anyway that video did clean up a bit!
AI could do a 1080p with sharp details. Someone just needs to process the frames, and use something like DALL-E or stable diffusion with the right settings.
Thank you, good heavens...the other video is awful by comparison, besides just needing cleanup and being terribly compressed it was horizontally backwards and played at the wrong framerate (so common with silent film).
Interestingly... on looking closer at the "restored" version, he is now sipping his tea with his right hand, as are his daughters, but he is smoking with his left hand.
He was right-handed, arthritic in the right, so he learned to write with his left; it doesn't seem reliable to know what hand he'd use for what activity
There is a point after which 'restoration' should be called 'deep fake'. Like intentionally modifying (mirroring in this case) the original. It may look better, but...
At what point does it stop being film of Mark Twain and start being what someone/a computer imagines film of Mark Twain to be like? Most of the data in this, from the color to the frames themselves, did not actually exist and, while looking reasonable, very likely does not agree with what the actual data would have been.
One other downside of this kind of approach is it makes high quality versions of the source significantly harder to find (I doubt the 240p encoding with blocks bigger than Mark Twain's head is the best copy of the source available).
For me there's a big difference between applying a simple algorithm to the data, a formula which someone could scribble down on the back of an envelope, on the one hand, and, on the other hand, some neural network thing with millions of weights that might be hallucinating.
To be fair, all photography is an approximation of reality. Whether that approximation is improved by color data added at time of capture or a century later, or if the image is cleaned up by AI instead of a human, doesn't change that fact.
Most images taken with a smartphone today don't capture reality reliably either, and you're allowed to alter them to look whichever way is most pleasing.
Even our eyes perceive reality slightly differently, and photography is not much different. If anything, looking at 2D reproductions of reality adds another layer of subjective perception.
Arguably the tech at the time did not allow Mark Twain to be properly captured at all, so even the original contains imperfections that distort the image. A high res AI version that restores it based on statistical training is still better and would show an "educated guess" of what he looked like.
Mark Twain had just finished building that house the year prior, and he would die the next year; the house itself would burn down not long after that :-(
It’s always neat when you think of some historical figure as ancient then see a video of them right in front of you, like history is not as far away as you thought. Similar to how there’s a recording of Johannnes Brahms (https://youtube.com/watch?v=0d848uKtdsA), and a video of the funeral of the last veteran of the war of 1812: https://youtube.com/watch?v=eMlKtfBHiAQ
Unlike what the title of this HN thread suggests, according to the two descriptions on the YouTube page and in the video clip, there is no indication that the film was shot by Edison himself, although it was shot using Thomas Edison’s Kinetograph camera and it was Edison’s film project.
I don’t know, but according to the following Library of Congress articles, the film camera was not invented by Edison, but by his assistant William Kennedy Laurie Dickson. The camera apparently was very bulky. I doubt that a bossy person like Edison would want to operate it by himself.
According to Mark Twain researcher Robert Slotta, who has studied the origins of this film for several years, it all began as a simple intro to a feature film Thomas Edison wanted to make based on Twain's 1881 novel, "The Prince and the Pauper" (published in America 1882). Twain reluctantly agreed as long as the material was never used for any other purpose. The intro consisted an edited version of the first three scenes of this film. Scenes included: Twain standing in the doorway, the walk around the house, and the second walk around was another "take" to cover a late camera start on the part of the cinematographer.
Since there was still unexposed film left over, it was at the film crew’s urging that the scenes with Twain and his daughters, Clara and Jean, be photographed as a gift with the promise that footage was strictly for their own private enjoyment.
I would think it would depend on the year. When he first invented it, probably him. Later on when he was selling the things, not.
Edison was obsessed with making money from the patents he had, applying them to anybody making movies; the reason Hollywood (the industry) is in LA is that filmmakers fled the East Coast to the wild west to get away from Edison's lawsuits.
so perhaps (speculating) if Edison was spending a lot of time with lawyers, I tend to doubt he was doing the filming.
Mark Twain was a celebrity though, so perhaps it was a friendly visit.
If this interests you then you might like to check out “They Shall Not Grow Old”[0] - a Peter Jackson movie which is based off AI/ML restored[1] WW1 footage with color, sound, and dialog added to silent B&W original stock.
Twain/Clemens was up there with Arthur Conan Doyle as the highlights of my childhood reading. I'll happily watch him take a stroll and puff a pipe, it's cool to see a historical figure I care about moving around in a real recording.
Whoever cast him in that Star Trek: The Next Generation episode did a fantastic job. I don't have anything intelligent to add, just wanted to say this is pretty cool to see.
I am surprised. Samuel Clemens was a good friend of Nikola Tesla, so I am surprised he was around Edison. I forget the timing of Clemens' friendship with Tesla, though I thought it was much earlier than this.
Tesla and Edison were rivals, but the idea that there was some deep personal animus between them isn't well supported. Tesla was awarded (and accepted) the Edison Medal in 1917, and he wrote a friendly (if also somewhat catty and critical) eulogy for Edison in 1931. By 1909 the Current Wars had been over for almost two decades and both men had moved on.
Interesting detail: in the film, his daughter used two long hair needles to fix the hat on her head. Is it just me who don’t know this use of hair needles?
Hairstyles and accompanying hat wear are definitely not a hobby of mine, and yes, I was aware. But I also have watched a lot of period films where they’ve shown such a thing.
I am looking forward to the next (5? 10?) years where AI will be cheap and fast enough, so that we can all (from the comfort of our own homes) process old photos and videos, enhance and enrich.
It's a pity that there is no audio in this video, but if the image is clean enough and we have an AI that can read lips.. one can only hope.
i think even just masking the background with a static image for a few of the scenes would be a gigantic improvement to hide the inconsistent quality of each frame
The digitally restored version is linked in the description of the video and has more information about the provenance of the video. Pretty cool to watch this.
No need for AI, simply clean it up and cut and paste him into a movie like they did when they spliced in some B&W footage of Humphrey Bogart in the movie Last Action Hero. The line was something like the Desk Sargent telling someone to take the “black and white”, and there was Bogey.
One about his life would be interesting enough, if done well. He spend lots of it as a tramp on the streets. There he learned to tell good stories out of neccesity, because he needed to find ways to convince people to give him various things like food and shelter ..
I feel like you wanted to mock Hollywood but you missed the point of the current debates.
Hollywood already can (and has) brought Mark Twain into movies by use of an actor. There’s nothing really different about this act in that lens if it’s done with AI.
Twain's Advice to The Youth is delightfully funny [1].
"Always obey your parents, when they are present. This is the best policy in the long run, because if you don’t, they will make you. Most parents think they know better than you do, and you can generally make more by humoring that superstition than you can by acting on your own better judgment."
Well that put a dent in my respect for Twain. Obeying your parents, the more experienced humans, is generally the better advice. There are exceptions of course.
I wonder if Twain had particularly difficult parents, hence his view.
Twain was smart enough to know most teenagers will never listen to someone that says "Your parents are more experienced humans and you should obey them".
Those rare kids that would listen to such advice have plenty of people telling it to them anyway.
"Age is no better qualified for an instructor as youth, for it has not profited so much as it has lost. One may almost doubt if the wisest man has learned anything of absolute value by living."
He was just trying to present the advice in a form that would be believed by his audience. As for his audience, he likely assumed that they were like him when he was young.
"When I was a boy of 14, my father was so ignorant I could hardly stand to have the old man around. But when I got to be 21, I was astonished at how much the old man had learned in seven years."
Twain is an interesting character. But he shouldn't always be read in a straightforward manner.
"The idyllic tableau of peace and genteel elegance was the image he [Twain] wanted seen and remembered. The carefully cultivated projection of a happy family, an elderly father with his patrician daughters (both looking well and pretty), was an opportunity for the media-savvy Twain to recapture some of the credibility he must have felt he had lost in his battle with Isabel and Ashcroft."
From Mark Twain's Other Woman by Laura Skandera Trombley. It explores the forgotten media scandals and controversies that plagued the last years of Twain.
Interesting! Note, though, that according to the restored version [0], the scenes with his daughters was for Twain's personal use only and was not seen by the public.
The others parts with him in the doorway and walking around the house probably were very carefully choreographed as you suggest.
