The Geography of Art: Imaging the Abstract with GIS

The use of technical imaging of art works has a long history, dating from the very early days of x-radiography up to today's computer imaging technologies. Besides the use of x-rays, the

"Conservators, conservation scientists and art historians have turned to another technique well known to the GIS community, multi-spectral imaging."

ultraviolet and infrared (IR) regions of the electromagnetic spectrum have found the most extensive use in this field. These examination tools yield significant information about the construction of a work of art, the materials the artist or subsequent restorers may have used, and generally the creative development of the work.

For example, infrared examinations, usually done with reflected light using vidicons or PtSi cameras, are very effective at revealing the underdrawing or other preparatory composition in a painting. This is achieved by taking advantage of the fact that many paints are relatively transparent in the short wave infra-red range while the preparatory drawings on the canvas are typically more absorbent, being done in carbonaceous materials like charcoal.

Computers have opened up yet more possibilities in this field, as they have in so many technical imaging fields of study. Remote sensing has also provided a number of useful tools, one of the earliest being mosaicing software. Just as with remote sensing images, the spatial resolution of technical images of art works can be preserved by mosaicing a smaller group of images into a larger, seamless mosaic. One of

"...encouraged by the overall accuracy Idrisi achieved in imaging the test panel, we looked at a painting by Jackson Pollock."

the first software programs to do this that was directly written for conservation research applications, Mosart, was the work of scientists experienced in remote sensing.

In recent years conservators, conservation scientists and art historians have turned to another technique well known to the GIS community, multi-spectral imaging. At the Museum of Modern Art we have been investigating this technique as a way to more clearly image the paints in abstract paintings. So far we have restricted this imaging to the infrared spectrum where we use an Inframetrics SWIR PtSi camera and a series of nine narrow-band filters from 1.1 micron to 2 microns, which can be inserted into the lens, to create the image set. The filters thus pass about 0.1 micron bands of light: plus or minus 0.05 microns around the central wavelength of each filter. These images are captured into a laptop computer using the VideoPort Professional PCMCIA video capture card. When the raw images are captured we also capture, under the same lighting conditions at each wavelength, a white Spectralon target. This image is then used to adjust the corresponding data image to remove any lighting artifacts and account for the uneven spectral output of the light source.

One reason the near IR spectrum was chosen for this initial study is that it holds the potential to differentiate paints on the basis of the medium as well as pigment composition. For example, if the technique were to succeed, it would ideally separate two paints that look similar in the visible spectrum due to similar or identical pigment composition but which are in fact made of different media.

To test this we painted out a panel of known grounds and paints and imaged it as outlined above. Figure 1 shows the visible light image. The panel consists of three different white gesso grounds that are, from left to right, done in an acrylic medium, an oil/egg mixed medium, and an oil medium. The blue paints are all cobalt blue pigment. The top row uses an oil medium and the second row an egg medium. In the third row the oil paint has been painted out on top of the egg medium-based paint and in the fourth row the egg medium paint has been painted out over a layer of the oil medium paint.

We then turned to Idrisi32 as our software package for classifying these paints. The best results were obtained with supervised classification and maximum likelihood estimation with 5% exclusion. Figure two shows some results. The three grounds are imaged quite effectively. The second and fourth rows, which have the egg-based paint on top also, are discretely separated, whereas the rows with oil on top are somewhat more ambiguous. This is probably the result of the fact that cobalt blue pigment in oil is more transparent than in the egg medium. The IR light is penetrating this layer, allowing underlying layers to contribute some reflectance to the image also. This is a problem that is very common in painted images and poses one of the chief challenges to the application of this technology to real world situations. It is also clear that this test panel offered an ideal situation for picking training sites, one that would not happen very often on a real painting.

Figure 3 - Shimmering Substance

Aware of these limitations, but also encouraged by the overall accuracy Idrisi achieved in imaging the test panel, we looked at a painting by Jackson Pollock. This painting, Shimmering Substance (Figure 3), was painted in 1946 just as the artist was entering his most productive period leading to his well known "pour" paintings. The goal of multi-spectral imaging in a case like this would be to clearly isolate the paints in an effort to visualize the patterns of paint application, to extract any forms or other features that were ultimately obscured by the unified final composition.

Imaging a detailed section from this painting, again following the same procedures as outlined above, yielded more ambiguous, but still promising, results. Figure 4 is of course the visible light image, Figure 5 is the full set of narrow-band images and Figure 6 is the maximum likelihood classification image: each color represents a different spectral class. Again the maximum likelihood (with 5% exclusion) supervised classification yielded the most information. Of particular help in this situation is the Idrisi feature that allows one to click on the color representing any particular cluster: this immediately presents an image of just that cluster. Applied over a larger area than this single detail, this feature offers the potential to isolate distinctive patterns that are difficult to perceive in either the visible light image or the classified false color image.

This application of GIS software, specifically Idrisi32, is still experimental. There are a number of clear limitations, such as the obvious fact that these IR images are not all surface reflectance images. Thus there are, as noted earlier, varying contributions of underlying layers to the image that can lead to misclassification of some pixels. Yet by using the software as a qualitative analysis tool, reliant upon operator judgment of the results, rather than a quantitative tool, there remains significant potential in the application.

The similarity of the problems that both remote sensing and technical imaging of art works face suggests that more focused and rigorous application of GIS imaging techniques to the art imaging field may well produce important results, especially in the area of deciphering abstract imagery.