Accusations of fraud hang over some of the world’s most highly valued artworks. Machine learning engineers are evaluating the authenticity of these famous pieces.
What’s new: Independent researchers determined that Salvator Mundi, the most expensive painting ever sold, was not painted entirely by Renaissance master Leonardo da Vinci, as had been claimed. In addition, the Swiss authentication company Art Recognition found that Samson and Delilah, a work credited to Peter Paul Rubens that hangs in London’s National Gallery, probably was painted by someone else.
How it works: Da Vinci produced few paintings, and he was also known to enlist assistants to help with his projects. Because of this, independent researchers Andrea and Steven Frank had just 12 verified da Vincis to train and test their system.
- The team augmented the dataset by dividing each painting into hundreds of overlapping slices of 350x350 pixels. They filtered the slices so that only those whose entropy, roughly indicating the amount of non-redundant information, was greater than average. They gathered similar slices of 33 paintings by other artists for a total of 17,520 training images and trained a convolutional neural network to classify whether or not a slice was painted by da Vinci.
- After training the CNN, they trained a linear model to decide, based on the CNN’s classifications of slices, whether a given painting was created by da Vinci. The system accurately determined that Seated Bacchus, a painting once thought to have been painted by da Vinci but now attributed to a protégé, was not the master’s work.
- By taking into account the classification of each slice of a given painting and their overlap, the researchers created heatmaps that showed areas most (shaded red) and least (shaded blue) likely to have been painted by da Vinci. This analysis showed that Leonardo da Vinci likely did not paint Salvator Mundi’s background or the figure’s raised right hand, but did paint its face and some of its body.
- Art Recognition, whose method appears to work in a similar way based on an unpublished description they provided to The Batch, trained its system on a mixture of 2,392 image segments drawn from works by Rubens and other artists. It determined with 91.78 percent accuracy that Samson and Delilah was not painted by Rubens.
Behind the news: Salvator Mundi was painted in the early 1500s and thought destroyed around a century later. The heavily damaged painting resurfaced in London in 1948. Experts there determined it was painted by one of da Vinci’s pupils, and it sold at auction for less than $50. After another sale, for $10,000 in 2005, evidence obtained during restoration convinced experts that it was an authentic da Vinci. It sold at auction for $450 million in 2017.
Why it matters: Fine art is a big business, and so is art fraud. Human experts often disagree in their assessments — and it may be impossible to establish the provenance of some works with complete certainty — but neural networks can supplement their judgments.
We’re thinking: If a human and a neural network disagreed about who created a picture, we’d just call it a draw.