Current

Sparse Coding

Edo Smitshuijzen | 16 January 2010

Expertise in visual communication and information architecture is a professional claim of many working in the field of graphic design. Maybe we should make these claims more specific since 'communication' and 'information' have become crucial topics in almost all branches of knowledge.

The spiritual and the physical world can be explained by the way the individual components communicate with each other or in other words how information is gathered, processed and exchanged. The human way of information processing seems to be mostly based on our own assumptions initiated by a few essential clues. The major part of our visual reality is a product of the phantasy of our brain, based on a few important visual data. We 'see' only 30% of a chair the rest is filled in by our imagination based on experience. Obviously, this is a very efficient way of data processing but not a very reliable one. We are 'primed' to visually scan our environment only for novelty, because that is all that counts. This is, possibly, the reason why we are crazy about ‘new things’. Maybe an unusual, or even a handicapped, processing of the essential associative visual imagination is what makes some people visual artists or designers.

A sample of a sparse coded image.

Computers have become indispensable to do any kind of work and soon we can't live without them either. Our electronic and physical self are merging into one identity. Notions and capabilities typical for the digital world have become part of our habits, and so does a new lingo that attempts to describe the digital reality. It's not easy to keep up with digi-speak when over a certain age. I realised that when I read an article on the BBC website about a method to spot art fakes using computer software. The article explained the computer program as follows: 'The approach, known as 'sparse coding', builds a virtual library of an artist's works and breaks them down into the simplest possible visual elements. Verifiable works by that artist can be rebuilt using varying proportions of those simple elements, while imitators' works cannot.
The method works by dividing digital versions of all of an artist's confirmed works into 144 squares - 12 columns of 12 rows each. Then a set of 'basis functions' is constructed - initially a set of random shapes and forms in black and white. A computer then modifies them until, for any given cut-down piece of the artist's work, some subset of the basis functions can be combined in some proportion to recreate the piece. The basis functions are refined further, to ensure that the smallest possible number of them is required to generate any given piece – they are the 'sparsest' set of functions that reproduces the artist's work.'

I read it and thought: 'Huh? So I read the text four more times but my 'huh' didn't get resolved, and it still blocked any understanding. I felt suddenly very old imagining that the average young reader had no problem whatsoever with digesting this sort of text. I needed Google to shed some light on what seemed like ‘abacadabra’ to me. Well, there I plunged into the world of high level mathematical statistics related to visual data: PCA's (principle component analysis), IPC's (principle components of images), SPCs (energy spectra of images) and wavelets. All these things are mathematical formulas that can be applied to digital images as filters to create new images. Every computer users is accustomed to using filters for images, we do that all the time, or we intervene directly in the visual representation of specific statical data of the image, like we do with the sliders of gray value 'levels' for instance. Image manipulation can be used to 'deconstruct' an image but also to 'reconstruct' it again. Working with advanced statistical data of images has an immense wide field of applications, like fast image recognition (fingerprints, faces), image compression, advanced image retouching and it seems to help to explain how our own visual system works: using as sparse coding as possible to built up a complete image. Apparently, when you manipulate the digital image of original art you can eventually bring it down to a few essential (sparse) visual characteristics. With these characteristics only originals can be rebuilt and not the fake copies. Potentially, the computer could be a better analyst and judge than the expert human eye.

Of course, I still don't really understand how this all works but my ignorance has been brought to a higher level so to speak. And that is the best one can hope for these days.