An Algorithmic Approach to Artwork Comparison for Condition Assessment

We demonstrate a method for obtaining crucial clues for artwork authentication and identifying alterations from cleaning or retouching by comparing a target digitalisation against a trusted one.
Our system emulates the analytical process of an art expert, performing distinct color, tonal, and surface feature analyses that are integrated into a coherent conclusion. It provides objective, quantifiable metrics to supplement traditional methods of evaluation.
This article focuses on a specific use case: a high-resolution image of Caravaggio’s Saint Catherine of Alexandria (1571–1610), which, contrary to expectations, revealed intriguing structural differences when compared to our archived version.

Overview of the Workflow

Our study addresses two central questions: First, is a new photograph a true digitization of the authentic artwork, or merely a reproduction? Second, how has the painting’s physical condition changed over time, for instance, due to restoration or aging? Understanding these aspects is critical because the art market increasingly relies on digital surrogates for initial assessment, thereby making the ability to trust an image’s fidelity to its source essential.

To answer these, we designed a fully automated pipeline. The process begins with preprocessing to minimize the influence of lighting or camera variations, followed by three analytical steps, each yielding a similarity score from 0 to 1. First, color analysis involves an examination of the overall color makeup, palette, and channel distributions. Next, tonal analysis assesses the light/dark balance, contrast, and shadow structure. Finally, surface feature analysis conducts a micro-textural inspection of features like craquelure and brushstrokes. Each of these pillars of analysis is designed to be as independent as possible, ensuring that a flaw in one dimension (e.g., poor color capture) does not invalidate findings in another (e.g., surface texture).

Individual scores and data visualizations are then compiled into a comprehensive report, which includes a global similarity score and is generated within minutes, providing detailed explanations accessible to non-technical users. Our approach is analogous to a forensic investigation, correlating multiple lines of evidence to form a conclusion. By quantifying visual data, it provides an objective and reproducible assessment, complementing the subjective nature of traditional connoisseurship.

As an illustration, we consider the painting *Saint Catherine of Alexandria* (133.0×173.0cm) by Caravaggio. This work, notable for its dramatic chiaroscuro, depicts a richly dressed martyr among the instruments of her torture. Its complex interplay of deep shadows and brilliant highlights makes it an excellent candidate for testing the robustness of our tonal and surface analyses. We compare a high-resolution image found online with a trusted version from our archive.

Image Preprocessing

For a meaningful comparison, both photographs are processed to ensure sufficient digitization quality. This involves adjusting overall lightness and color contrast, cropping extraneous elements like frames, and performing geometric registration, where one image is warped to align perfectly with the other, thereby minimizing the influence of capture conditions. This procedure allows us to discern meaningful differences from merely incidental ones.

Color Analysis

To compare the color profiles of the two digitisations, we first compute a distance between the histograms of their RGB channel distributions, derived from the frequency of pixel values in each image. This provides a high-level statistical fingerprint of the work’s chromatic character. For a quick visual comparison, we also extract a representative color palette using a quantization algorithm.

From these analyses, a color similarity score is derived. In our report, we supplement this score with data visualisations, such as 3D plots and difference maps. In the Caravaggio example, the two photographs show a similar color histogram and palette, with the most notable divergence observed in the blue tones. This resulted in a relatively high chromatic similarity score of 0.7.

While color is an intuitive starting point, it can be affected by external factors that preprocessing cannot fully mitigate. These include the natural yellowing of varnish over time or subtle differences in photographic white balance, which can alter the perceived color without indicating a change in the underlying artwork. Consequently, we weight color similarity lower than other factors in the final global score and proceed with more structurally-focused analyses.

Michelangelo Merisi da Caravaggio – Saint Catherine of Alexandria – Surface Plot Comparison

Tonal Analysis

The next phase involves an analysis independent of color, focusing exclusively on the tonal *values* (shades from black to white). By converting the images to grayscale, we isolate luminance information from hue and saturation. A histogram of brightness levels is then used to compare the images’ overall tonal balance. This analysis is crucial, as reproductions often fail to capture the full dynamic range of an original painting’s light and shadow, while digital enhancement may artificially extend it. Caravaggio’s mastery of chiaroscuro, for example, depends on subtle gradations within the darkest parts of the composition, which are often lost in inferior copies.

For our use case, we observed a high tonal correspondence of 0.9 between the two digitalisations, even in the dark areas of the Saint’s clothes, which are difficult to compare with the naked eye.

Michelangelo Merisi da Caravaggio – Saint Catherine of Alexandria – Zenital Comparison

Surface Feature Analysis

Finally, we perform a micro-textural analysis, examining fine details of the painting’s surface. This includes patterns such as craquelure, canvas texture, and brushstrokes, as well as features resulting from the aging of paint and varnish layers. These features constitute a unique physical fingerprint of an artwork, developed over centuries and extremely difficult to forge accurately.

Technically, our process applies proprietary computer vision algorithms for edge detection and high-pass filtering, thereby effectively removing the broader shapes and colors of the painting to reveal high-frequency information. We then compute the similarities of the craquelures using a Scale-Invariant Feature Transform (SIFT). A difference heatmap is then used to visually highlight any areas with significant discrepancies.

Surprisingly, in our case study, the online image did not closely match our trusted reference. The craquelure pattern differed in multiple details, yielding a low micro-textural similarity score of 0.5. The heatmap identified suspect regions, primarily within the saint’s clothing. While some minor deviation is expected due to microscopic shifts, the detected inconsistencies were systematic and spatially clustered, indicating a non-random cause.

Michelangelo Merisi da Caravaggio – Saint Catherine of Alexandria – Craquelures Comparison

Conclusion

Our method provides more than a simple ‘yes/no’ answer. The global similarity score—a weighted average of the three analyses that de-emphasizes color—was 0.65 in our case study. More importantly, we deliver a reasoned report, backed by multiple data points, regarding the artwork’s condition, which empowers the user to understand the reasoning behind the similarity score and points directly to the visual evidence.

This body of evidence cast significant doubt on whether the painting was in the same condition when the second photograph was taken. Indeed, subsequent research confirmed this, revealing that a restoration had been performed on the artwork. This restoration involved a cleaning process that revealed the rich hues, subtle flesh tones, and delicate textures of the fabrics. Notably, these changes occurred precisely in the areas of the clothing our system had flagged. In short, without prior knowledge, our analysis detected the alteration and identified the affected regions. This constituted a successful blind test and served as a powerful validation of our model’s sensitivity to subtle but significant changes in an artwork’s physical structure.

More generally, this multi-step analysis addresses two use cases. First, regarding authentication, it can instill confidence in researchers, historians, and buyers regarding a painting’s condition and authenticity, particularly in an increasingly digital art world where preliminary assessments often must be conducted remotely. Second, for art conservation, it can be used to track subtle material degradation over time and to objectively document the impact of restoration work, thereby enabling informed preservation decisions.