Nostalgia takes shape: Melting ‘frozen memories,’ AI helps Japanese recall war days

“In colourised photos, people come to life,” said Hamai, now 86. “I often played near (the picnic site), and sometimes I would do some naughty things and get scolded by my father.” The power of a colourised photo to reignite lost memories was eye-opening for Anju Niwata, a student who gave Hamai the colourised photo as a present three years ago.

The 75th anniversary of the end of World War II is Saturday, and Niwata, now 18, said she hopes it will bring attention to her project with a Tokyo University professor to painstakingly colourise photos using artificial intelligence and their own research to spark lost memories for the rapidly aging generation who experienced the war.

“Seeing Niwata share the colourised pictures with Hamai, and then watching him recall his old memories one after another, made it feel like the ice around his frozen memories was melting away,” said Hidenori Watanave, the professor who taught Niwata how to colourise monochrome pictures using AI. Niwata and Watanave call their photo colourisation project “Rebooting Memories,” and they published a book last month of the colourised versions of about 350 monochrome pictures taken before, during and after the war.

Watanave and Niwata use three different types of AI photo colouring software. The AI is useful in identifying the accurate colours of natural things, such as the sea, the sky and human skin, but it cannot accurately colourise human-made objects like roofs and clothes, Watanave said.

So Niwata and Watanave painstakingly finish the AI-colourised photos by hand to get more accurate colours based on the photo owners’ memories and advice from experts. They also look through historical documents and archives that show what the colours should look like.

Some photos take a few months to finish.For Watanave, Twitter has become a powerful platform to pursue the colourisation project. When he posted a picture of the Hiroshima atomic bomb mushroom cloud that the Al software had colourised as white, a film director suggested that it should be more orange. Watanave checked the testimonies of those who saw the mushroom cloud and also researched the components of the atomic bomb to see if it could actually make an orangish colour. After he confirmed that it could, Watanave added orange to the picture. While the accuracy of the colour is important, Niwata and Watanave said the most vital thing is that the colourised photos match the memories of the photo owners.

Time, however, is running out; the average age of the atomic bomb survivors is about 83.

There are often moments of wonder when the elderly see the revitalised photos. When Niwata showed the colourised version of a family photograph to a war survivor who had dementia, for example, he remembered the type of flowers in the photograph; just a few weeks later, he was unable to speak. Niwata said that publishing the colourisation book during the coronavirus outbreak has made her think about the pandemic’s link to the war. “Our everyday lives have been stolen away by the coronavirus in a flash, which I think resembles what happened in the war. That’s why I feel like now is an opportunity for people to imagine (wartime life) as their own experience,” she said.

Watanave hopes that using new technology will help younger Japanese feel more of an attachment to those who lived through the war.