March 01, 2020

Putting Robots to Work on the Past


Putting Robots to Work on the Past

In 1896, Frenchman Charles Moisson, one of the few cameramen working for Lumiere company, was in Moscow to attend the coronation of Russia's last tsar, Nicholas II. He made a widely-known short clip of bustling Tverskaya street, showing the corner of the still-standing National Hotel and passing walkers and carriages.

Now Russian techie Denis Shiryaev [about] has put Moisson's camera work through several neural networks, to enhance and colorize the original monochrome clip. Internet users have praised his work as an incredible "time machine" that gives us a glimpse of the past that looks more realistic than ever.

We asked Shiryaev about his interest in using the work of robots to produce this nostalgia-laced result. For more technical information about his work, click on the clip to read the YouTube description in Russian and English.

 

RL: In a nutshell, how does this work? Do you upload the video into a program? What else does it need to know about the video to get the colors right?

DS: I put the video through an ensemble of neural networks, each of which plays a role. One cleans the video of noise, another creates additional frames to make the video play more smoothly, a third one increases the resolution, and finally one "colors" the video. Each algorithm has been honed on its own set of data, for example, the coloring neural network knows that people's faces are usually beige, grass is green and the sky is blue, because that's how the neural network was trained. 

RL: Do you think the result would be better if the video was colored by hand?

DS: Robots and algorithms are cheaper and faster than human work. Restoration can definitely be done by people, but with each year, work by algorithms will improve. People historically lose the competition to robots, and in five years' time neural networks will not only improve data that exists in an image, but also add additional faces, objects, and anything else. In fact, restoration is basically not done purely by hand anymore, because restorers use a lot of computer programs to enhance the image.

RL: Why do you choose clips like these to work with? Are you nostalgic for the era before the internet?

DS: I like history and that the image comes alive when you put it through the different algorithms. I choose the clips based on my own taste and the advice by YouTube commentators, who suggest other ideas for restoration.

 

 

RL: Reading the comments, especially to the clip "Arrival of the Train", it seems that a lot of people are struck by the thought that "everyone in the video is now dead" even the little children. Isn't it strange that a restored video from 1896 draws this response? 

DS: Old clips that we usually see look different from videos that we're normally used to seeing on the internet. People on them are usually seen running to the sound of music or the projector noise, so the videos look "ancient" and you don't associate yourself with the people in them. When I restore the clips, people come alive and the clip stops seeming artificial. You can no longer get rid of the feeling that all of those people are real.

 

Like this post? Get a weekly email digest + member-only deals

Some of Our Books

93 Untranslatable Russian Words

93 Untranslatable Russian Words

Every language has concepts, ideas, words and idioms that are nearly impossible to translate into another language. This book looks at nearly 100 such Russian words and offers paths to their understanding and translation by way of examples from literature and everyday life. Difficult to translate words and concepts are introduced with dictionary definitions, then elucidated with citations from literature, speech and prose, helping the student of Russian comprehend the word/concept in context.
Murder and the Muse

Murder and the Muse

KGB Chief Andropov has tapped Matyushkin to solve a brazen jewel heist from Picasso’s wife at the posh Metropole Hotel. But when the case bleeds over into murder, machinations, and international intrigue, not everyone is eager to see where the clues might lead.
Jews in Service to the Tsar

Jews in Service to the Tsar

Benjamin Disraeli advised, “Read no history: nothing but biography, for that is life without theory.” With Jews in Service to the Tsar, Lev Berdnikov offers us 28 biographies spanning five centuries of Russian Jewish history, and each portrait opens a new window onto the history of Eastern Europe’s Jews, illuminating dark corners and challenging widely-held conceptions about the role of Jews in Russian history.
The Little Humpbacked Horse

The Little Humpbacked Horse

A beloved Russian classic about a resourceful Russian peasant, Vanya, and his miracle-working horse, who together undergo various trials, exploits and adventures at the whim of a laughable tsar, told in rich, narrative poetry.
Davai! The Russians and Their Vodka

Davai! The Russians and Their Vodka

In this comprehensive, quixotic and addictive book, Edwin Trommelen explores all facets of the Russian obsession with vodka. Peering chiefly through the lenses of history and literature, Trommelen offers up an appropriately complex, rich and bittersweet portrait, based on great respect for Russian culture.
Fearful Majesty

Fearful Majesty

This acclaimed biography of one of Russia’s most important and tyrannical rulers is not only a rich, readable biography, it is also surprisingly timely, revealing how many of the issues Russia faces today have their roots in Ivan’s reign.
The Samovar Murders

The Samovar Murders

The murder of a poet is always more than a murder. When a famous writer is brutally stabbed on the campus of Moscow’s Lumumba University, the son of a recently deposed African president confesses, and the case assumes political implications that no one wants any part of.
A Taste of Russia

A Taste of Russia

The definitive modern cookbook on Russian cuisine has been totally updated and redesigned in a 30th Anniversary Edition. Layering superbly researched recipes with informative essays on the dishes' rich historical and cultural context, A Taste of Russia includes over 200 recipes on everything from borshch to blini, from Salmon Coulibiac to Beef Stew with Rum, from Marinated Mushrooms to Walnut-honey Filled Pies. A Taste of Russia shows off the best that Russian cooking has to offer. Full of great quotes from Russian literature about Russian food and designed in a convenient wide format that stays open during use.
22 Russian Crosswords

22 Russian Crosswords

Test your knowledge of the Russian language, Russian history and society with these 22 challenging puzzles taken from the pages of Russian Life magazine. Most all the clues are in English, but you must fill in the answers in Russian. If you get stumped, of course all the puzzles have answers printed at the back of the book.
Murder at the Dacha

Murder at the Dacha

Senior Lieutenant Pavel Matyushkin has a problem. Several, actually. Not the least of them is the fact that a powerful Soviet boss has been murdered, and Matyushkin's surly commander has given him an unreasonably short time frame to close the case.

About Us

Russian Life is a publication of a 30-year-young, award-winning publishing house that creates a bimonthly magazine, books, maps, and other products for Russophiles the world over.

Latest Posts

Our Contacts

Russian Life
73 Main Street, Suite 402
Montpelier VT 05602

802-223-4955