A Fire within the Eye: The biological system that underlies the visual arts

❉ Number 16 in a series of blogs on diagrams in relation to the arts. This was the subject of my PhD at Kyoto City University of the Arts, Japan's oldest art school. Feel free to contact me if you have questions about studying in Japan, the Japanese Monbusho Scholarship (MEXT), or what a PhD in Fine Art involves.

Figure 1: Pupil, Michael Whittle, 2013, Ink on paper, 82 x 82 cm.

Vision is our mind's highest bandwidth connection with external reality. As an artist with a background in biology, the mechanism of sight is a recurrent theme in my drawings and sculptures. I am fascinated by the strengths, weaknesses, and evolutionary origins of the "wet-ware" that underlies visuality.

We often assume we see the world "as it is," but the scientific understanding of vision suggests something far more complex. We do not just record reality; we construct it. This post explores the history of how we have diagrammed this process, from ancient optics to modern neuroscience, and how these biological realities influence the way we create and experience art.

The First True Diagram of Sight

Figure 2: Ibn al-Haytham: Diagram of the human visual system.

From a 1083 copy of his Kitab al-Manazir, Süleymaniye Library, Istanbul.

It is now over a thousand years since the Golden Age of Arabic Science, during which time Ibn al-Haytham (Alhazen) published his magisterial seven-volume treatise on optics, the Kitab al-Manazir (1015), from which the diagram in Figure 2 is taken (1).

Translated into Latin in the 13th century as De Aspectibus, and again in 1572 after the advent of the printing press, the ideas it contained influenced generations of European thinkers throughout the Middle Ages and into the Renaissance. Al-Haytham's methodical attention to detail and insistence on evidence-based reasoning meant that he is often referred to as not only the father of modern optics, but one of the world's first true proponents and practitioners of the scientific method.

Figure 2 is particularly interesting because of the artistic restrictions on Muslims against depicting living or sentient beings, known as Aniconism. However, Al-Haytham's use of the abstractive powers of the diagram allowed him to bypass this: he drew an elegantly stylised nose, the eyes and their lenses, and the optic nerves leading to the visual cortex at the back of the brain (shown here at the top of the figure). Remarkably, he even included the chiasma, or 'crossing point' of the optic nerves within the brain.

Al-Haytham's work on optics proposed a systematic solution to the problem of vision that combined experimental investigations into the behavior of light with inventive geometrical proofs and constant forays into the psychology of visual perception. His insight that light is composed of particles, for example, would later be described by Newton and finally proven by Einstein in his work on the 'photoelectric effect' in 1905, for which he later won the 1921 Nobel Prize for Physics.

The "Divine Fire" vs. The Mechanism

The treatise offered a coherent alternative to the Euclidean and Ptolemaic theories of visual emission, in which visual rays were believed to be emitted from the eye—curiously emphasized by the peculiar hand gestures in Figure 3 below. This esoteric idea was first proposed by the 5th-century Greek philosopher Empedocles, who claimed that vision was enabled by the goddess Aphrodite, who lit a "divine fire" within the human eye.

Figure 3: Johann Zahn - Emission Theory,

“Oculus Artificialis Teledioptricus Sive Telescopium”, 1685.

One important aspect of vision that Al-Haytham did get wrong was the fact that an image projected by a lens is upside down and flipped right-to-left. Even though this is clearly contained within his optical formalism, it was apparently more than he could accept in a theory of vision. Leonardo da Vinci (b.1452) also failed to accept this when he approached the problem almost 500 years later.

Johannes Kepler (b.1571) directly followed Al-Haytham's formalism to its inevitable and logical conclusion in his mechanistic theory of the retinal image, which replaced its impressionistic predecessor. However, even Kepler struggled to accept what Al-Haytham called the "monstrous" distortion of sensation that would be provoked by such an inversion. It remained for René Descartes (b.1596) to compile his own theory of vision, beautifully illustrated with diagrams and far closer to our current understanding (Figure 4).

Figure 4: The Descartes diagram from 1664 shows how physical impulses,

transmitted in the form of rays from the arrow into the eye, give rise to physical impulses

through the optic nerves and then into the brain to the pineal gland.

When contemporary scientists talk about vision, there is no mention of pictures traveling intact from the eye to the brain. Instead, they describe systems of synapses, electrochemical responses, and complex notions of encoded information transfer that somewhat support the theory of Kepler, but most definitely that of Descartes.

Figure 4 above depicts Descartes' theory of how physical impulses are transmitted to the pineal gland, which he incorrectly believed served as the nexus between the immaterial mind (which he referred to as res-cogitans) and the physical body (res-extensa).

Figure 5: Target, Michael Whittle, 2013, Wood, perspex,

Japanese archery target, strobe-lights, 42 x 89 x 19 cm.

Coding the World

Descartes seems to have been the first person to clearly express the correct solution to the 'monstrous' problems of the visual system. He starts by pointing out why the problem arises: it is tempting to think that the act of 'seeing' amounts to having, somewhere in the brain, a little picture that can be looked at by the mind. Descartes dismisses this notion by pointing out that the code which instantiates sensations in the brain need have no resemblance to the sensation itself (2).

Figure 6: Descartes' blind man probing the environment with sticks.

No confusion arises concerning the location of objects,

despite the fact that the sticks are crossed.

Evolutionary Compromises

Until only recently, the idea of fully understanding the inner workings of the eye was considered a problem of such intractable complexity that it was often held as evidence of a supernatural engineer. Even Darwin himself appeared troubled at first, remarking in an often misquoted aside in Origin of Species that the whole idea that something so flawless "could have been formed by natural selection, seems, I freely confess, absurd in the highest degree."

However, as with all complex biological features, the human visual system arose unimaginably slowly over hundreds of millions of years of clumsy trial and error. On closer inspection, we find numerous examples of evolutionary concession and compromise.

As a result, the human eye is just as good as it needs to be and no better, and systemic errors have to be accommodated using sophisticated cognitive processing techniques (the blind spot is a good example of this).

For an excellent article covering this topic click here: The Poor Design of the Human Eye

Figure 7: Tear glands, tear ducts, Michael Whittle, 2013, ink on paper, 45 x 84 cm.

Mapping the Ghost in the Machine

Design limitations aside, humans are primarily visual creatures. Almost half of the brain is either directly or indirectly involved in processing visual information. The last decade of brain research has revealed more about the human brain than all the years of prior research combined. The number of neurons devoted entirely to visual processing is now estimated to be in their hundreds of millions, taking up about 30% of the whole cortex, as compared with 8% for touch and only 3% for hearing.

Each of the two optic nerves that carry signals from the retina to the brain consists of a million fibers, whereas each auditory nerve carries a mere 30,000. Al-Haytham's 1000-year-old diagram actually captured the surprising fact that the retinas of the eyes are literally outgrowths of the brain itself.

Figure 8: Optic tract, Michael Whittle, (Neural network detected retinal vasculatures with catenoid and two point source interference patterns), 2017, 97 x 92 cm, Ink, pencil, and watercolour on paper.

Figure 9 shows a simplified anatomical diagram showing the various routes of the optic nerves,

and how data from the various quadrants of the visual field gets divided up en route

to the visual cortex at the back of the brain.

Figure 10 below shows the results of an important 1988 experiment designed to physically demonstrate the relationship between visual images and their perception in the visual cortex. A Macaque monkey was shown a striped visual target and given a dose of glucose with a radioactive marker attached. Glucose molecules are most rapidly absorbed by the most active cells; in this case, the scientists were interested in the cells of the visual cortex.

When the visual cortex of the Macaque was removed and simply pressed onto a radio-sensitive plate, the radioactive marker had become concentrated enough in the most active cells to leave a trace on the plate. The external, striped target on the left was shown to map directly onto the internal, organic material of the brain on the right. Point F represents data captured by a small area of incredibly highly sensitive retina known as the Fovea, where detail is at its highest.

Figure 10: A flickering stimulus (left) and its 'retinotopic representation' in layer 4C of V1 in the visual cortex of a Macaque monkey (right), revealed through CO staining. Reproduced from Tootell et al (1988a).

It is probably worth repeating—what you are looking at in Figure 10 is an actual metabolic imprint of the external environment being mapped onto the structure of the brain at the cellular level. More details of how this image was created are included in the footnotes below as note 4 (4).

Figure 11: Model for the reorganization of visual information (Retinotopic visual stimuli map of primate striate visual cortex with retinal vessel shadows), Michael Whittle, 2017, 140 x 108 cm, Ink and watercolour on paper.

Figures 12a and b present hypothetical examples of how complex, real-world images are believed to be processed and mapped out within the structures of the brain as 'Retinotopic Representations' in the two lobes of the visual cortex at the back of the brain.

Active Diagramming: The Saccade

Visual data from the tiny foveal region is given maximum priority over that collected from the rest of the retina. Approximately half of the nerve fibers in the optic nerves carry information from the fovea alone, while the remaining half carry information from the rest of the retina.

It is an interesting realization that this tiny, high-resolution part of your vision is about the same size as your thumbnail held at arm's length in front of your eye. Rather than the eye acting as a camera that takes in a complete image, the act of looking is in fact a process of "diagramming" the world using that tiny region of high-resolution information provided by the fovea.

One of the pioneering researchers of how the human visual system relies on such diagrammatic eye movements was the Russian psychologist Alfred Yarbus (1914-1986). Yarbus conducted several influential experiments using suction caps attached to the eyes to track eye movements while participants viewed various images and scenes. His most famous study, published in the book "Eye Movements and Vision" (1967), demonstrated that the way people move their eyes while looking at a picture depends on the task they are given or the information they are seeking.

Figure 13 shows the results of one such experiment, and the rapid series of twitching eye movements, known as saccades, that scan the world around us as we process our surroundings. Nicholas Wade from the University of Dundee (UK) & Galina Rozhkova from the Institute for Information Transmission Problems at the Russian Academy of Sciences (Moscow) have compiled an excellent website detailing the work of Yarbus, and how his discoveries laid the foundations for the study of human vision and cognition: https://yarbus.eu/faces/.

Figure 13: Girl from Volga, Alfred Yarbus. Record of the eye movements

during free examination of the photograph with both eyes for 3 minutes

Al-Haytham was rightly concerned about the "monstrous" distortions involved in human vision and visual processing. During their decoding, images are not only inverted but divided and distorted within the visual cortex, and yet our visual experience of the external world remains immersive and wholly convincing.

Finally, to end the post, here is a clip from an interview with the Nobel Prize-winning physicist Richard Feynman, who spent many years of his distinguished career investigating the fundamental nature of light at the level of photon-photon interactions. During the clip, he discusses the phenomenon of vision and the tiny segment of the electromagnetic spectrum that it evolved to detect.

https://video.wixstatic.com/video/ed8cef_91bd7fa14fad40539477187e7fb6d0bd/360p/mp4/file.mp4

Notes:

1) al-Haythem wrote the series on optics whilst working at Cairo's al-Azhar Mosque sometime between 1028 and 1038

​2) "Apart from that, it is necessary to beware of assuming that in order to sense, the mind needs to perceive certain images transmitted by the objects to the brain, as our philosophers commonly suppose; or at least, the nature of these images must be conceived quite otherwise than as they do: For, inasmuch as [the philosophers] do not consider anything about these images except that they must resemble the objects they represent, it is impossible for them to show us how they can be formed by these objects, received by the external sense organs, and transmitted by the nerves to the brain. And they have had no other reason for positing them except that, observing that a picture can easily stimulate our minds to conceive the object painted there, it seemed to them that in the same way, the mind should be stimulated by little pictures which form in our head to conceive of those objects that touch our senses; instead we should consider that there are many other things besides pictures which can stimulate our thought, such as, for example, signs and words, which do not in any way resemble the things which they signify." Descartes, Optics, Fourth discourse (p. 89 in trans. by P.J. Olscamp)

​3) "So that you must not be surprised that the objects can be seen in their true positions, even though the picture they imprint upon the eye is inverted: for this is just like our blind man's being able to sense the object B, which is to his right, by means of his left hand, and the object D, which is to his left, by means of his right hand at one and the same time. And just as this blind man does not judge that a body is double, although he touches it with his two hands, so likewise when both our eyes are disposed in the manner which is required in order to carry our attention toward one and the same location, they need only cause us to see a single object there, even though a picture of it is formed in each of our eyes." Descartes, Optics, Sixth Discourse, trans. P.J. Olscamp, p. 105.

4) Figure 10 shows the results of an experiment in which an anaesthetized Macaque monkey viewed a flickering bulls-eye pattern, and was then injected with radioactively labeled glucose (a special type of glucose with a radioactive Fluorine molecule attached). The glucose was taken up in higher amounts by the most active neurons. Area V1 of the visual cortex was then surgically removed and flattened before being used to expose radioactively sensitive film. The result is a picture of regions of activity evoked by the bulls-eye pattern, creating what is known as a 'Retinotopic Map'.