Nvidia's amazing deep learning tool can reconstruct incomplete photos

[farzanaSadia]

It might look like witchcraft, but researchers at Nvidia have developed an advanced deep learning image-retouching tool that can intelligently reconstruct incomplete photos.

While removing unwanted artefacts in image editing is nothing new – Adobe Photoshop's Content-Aware tools are pretty much the industry standard – the prototype tool that Nvidia is showcasing looks incredibly impressive.

Don't take our word for it – check out the two-minute video below to get a taste of what this new technology is capable of.


What differentiates Nvidia's new tool from something like Content-Aware Fill in Photoshop is that it analyzes the image and understands what the subject should actually look like; Content-Aware Fill relies on surrounding parts of the image to fill in what it thinks should be there.

Nvidia's tool is a much more sophisticated solution. For instance, when trying to fill a hole where an eye would be in a portrait, as well as using information from the surrounding area Nvidia's deep learning tool knows an eye should be there, and can fill the hole with a realistic computer-generated alternative.   

"Our model can robustly handle holes of any shape, size location, or distance from the image borders," the researchers write. "Previous deep learning approaches have focused on rectangular regions located around the center of the image, and often rely on expensive post-processing. Further, our model gracefully handles holes of increasing size."

For the moment at least there's no word when we're likely to see this tool become more widely available. For now though, it gives us a glimpse into the near future of image editing.