The “where” and the “what” of bistable perception

How your brain decides what you see – again and again

Barbara Nordhjem, 29^th November 2011.

Published in the dutch magazine Psyche en Brein [pdf] and my translated version [pdf].

Look at the figures and try if you can see them change. Ambiguous figures like the well-known Necker cube and Rubin vase can be experienced in two different ways. When you look at these types of figures they spontaneously ‘flips’ back and forth between two different percepts. Usually one interpretation stays stable for a while before it flips again. Because theses figures can be experienced in two different ways they are called bistable. When there are more than two interpretations of an ambiguous stimulus it is called multistable.

The Necker cube is one of the most classic examples of a bistable figure. Sometimes it looks like it is facing upwards while at other times it is facing downwards. You can practice and get better at making the figures change when you like, but they will flip, even when you just look at them without making any effort. So why doesn’t your brain just stick with one way of seeing these figures? The most direct answer is that there is not one way of seeing such a figure that is more correct or valid than the other. Without any definite solution, your brain just keeps alternating between the two versions of the figure. There can still be some preferences, typically people see the Schröder staircase as the kind of stair you can walk on before it flips and looks like it is facing downwards.

One of the reasons I am so fascinated with bistable perception is that these illusions show that seeing is far more than just the direct experience of visual input being projected in the brain. Seeing is also the process of making sense of the world. You can imagine visual ambiguity within a continuum of different visual experiences. At one extreme, you have a full correspondence between the visual stimuli in front of you and what you perceive. The opposite case is hallucination where perception has little or no relation to any visual stimuli. Ambiguity is somewhere in between, perception of these figures depends on both physical stimulus itself and on how they are interpreted in your head. Ambiguity is not limited to bistable perception, but in most cases there is a period of ambiguity and then a solution. For example when you see a person from a distance there is a period of uncertainty before you can tell who it is and you have a closer look to find out if it is really somebody you know. For truly bistable figures there will never be one interpretation that is more correct than the other, they are literally visual brainteasers.

Play it again

I did this project at University College London together with graduate student Fiammetta Ghedini from Italy. One of the questions about bistable perception we found most intriguing was to find the brain areas that make the bistable figures flip back and forth. We recorded brainn activity using a functional MRI scanner. Several other researchers have already looked at brain activity during bistable perception, but here we made some adjustments. First of all, there are hardly any brain imaging studies with many different bistable figures, most researchers restrict themselves to just a few well-known examples. Second, it is always a challenge to find out what to compare bistable perception with. With fMRI reseach it is necessary to choose a good control condition, something to compare your phenomenon of interest with. In our case we created unambiguous figures that looked the same as each interpretation of the bistable figures. Our study consisted of ten different bistable figures, and for each bistable figure we also had two stable figures.

We used both the bistable and stable figures in a way that has not been done in an fMRI study of bistable perception before. Take for instance the Indian-Eskimo (indian-eskimo-bistable.bmp, eskimo-stable.bmp, indian-stable.bmp) figure. Every time it looked like an Indian, the participant pressed one button and every time it appeared as an Eskimo the participant hit another button. Afterwards, a movie was shown of how the participant experienced the bistable figure. Here two stable versions of the bistable figure were shown changing back and forth. The replay movies also alternated with the same speed as the bistable figures: Indian-Eskimo-Indian-Eskimo. Every time the images changed in the movie, the participant would press the same buttons again. Of course everyone sees bistable figures differently. Therefore the replay movies were generated for each participant directly when he or she was laying in the brain scanner. What is the difference between looking at the bistable figures and the replay sequences? In the first case where the bistable figure is shown, there is one image but the experience of it changes back and forth. During the movie replay, there are two images alternating in front of the participants eyes in the same way as the illusion changed in his mind. This experiment gave us the possibility to compare how the brain reacts to bistable images that change back and forth in the mind compared to images that literally change back and forth in front of the eyes. In other words, we could look at the difference between internally and externally driven changes in perception.

A cube is a cube is a cube…

Fiammetta Ghedini and I did this project in the lab of Semir Zeki who has been a pioneer in the field of visual perception. In the beginning of our project we were surrounded by books with optical illusions and tutorials on how to get started with brain imaging research. To add to the confusion, prof. Zeki came into our office, waved his silver-topped walking stick in the air, and said “A cube is a cube is a cube, but a face is not a vase”, where after he left again. Just like the illusions we were studying, these words seemed like a riddle too. We tried to combine Zeki’s statement with research of how the visual brain works. There are regions of the brain that respond selectively to certain types of stimuli. For example, there is an area called the fusiform face area, which becomes active when you look at faces. The interesting things about visually selective areas like the fusiform face area is that they also get active when you look at an ambiguous figure like the Rubin vase and see the two faces in silhouette. It appears that processing specific areas such as the fusiform face area do not only respond when there is a face in front of you, but also when you have the perceptual experience of seeing a face. Something similar happens in a different brain area when you look at objects. So when you experience the Rubin figure change between vase and faces, brain activity most likely also fluctuates between two different areas.
Figures like the Necker cube are different compared to figures like the Rubin vase; a cube is a cube no matter how you turn it around. It remains a cube, just seen from a different perspective. We were interested in the difference between figures like the Rubin vase which changes between two categories and figures like the Necker cube which also flips but stay the same object. Therefore we included two different types of bistable figures in our experiment. We used the name “within-category figures” for geometrical figures like the Necker cube, which switch perspective but remain the same object. We also had another set of images such as the Rubin vase, which we called “between-category figures” because they change between two figure categories. To keep things simple and comparable, all our between-category figures could either be interpreted as a face or a full body figure. It is possible that brain activity is completely different for one type of figure compared to the other. This was the first fMRI experiment where brain activity during to perception of different types of bistable figures could be compared, so we were quite curious what would come out of it.

Symmetrical network

First, we compared perception of spontaneously reversing figures with the replay movies of externally changing figures. The goal here was to locate brain areas specifically involved in bistable perception. The perceptual alternations during the bistable condition were due to changes driven by the viewer, while alternations during movie replay condition depended on changes of the visual stimuli.

Our results showed that there is not just one area involved bistable perception, but several located in both left and right side of the brain. We were quite surprised to find activation in both hemispheres. There has been other brain imaging studies of bistable perception, but most researchers only found activity on the right side of the brain. We mainly found activation in frontal and parietal cortex. These are areas you would also find involved when you direct your attention towards something, without necessarily looking at it. This kind of attentions shift is similar to the situation where you are looking at your computer screen but in fact you are paying attention to something that is going on at a different place in the room.

There are probably several processes going on when you look at bistable figures. The brain has to register that it is possible to interpret the visual input in two ways, select one configuration and ignore the other, and then switch to the other interpretation. It is possible that the frontal and parietal areas spread across the brain are generally involved in evaluating and re-evaluating the figures. Our study shows that visual perception is an interpretive process and the brain seeks possible configurations of the figures. This is not a process that is exclusively related to bistable perception; we deal with ambiguity throughout daily life. There is not always one single correct way of perceiving a stimulus, but several likely ways. In general, internally driven perceptual changes seem to involve widely distributed brain areas. I think that the frontal and parietal areas shown here play an  important role when we interpret the world of visual patterns and different possibilities around us. It seems like there is a network of brain areas, which are active in the processes of constantly re-evaluating sensory stimuli and try out different solutions.

Categorization and rotation

Remember that we also created two types of figures? We wanted to see if there were differences in brain activity for the within-category figures compared to the between-category figures. The fMRI scans, allowed us to see which brain areas are specifically active when looking at different types of bistable figures. The results showed thant looking at within-category figures like the Necker cube is a different task that looking at between-category figures like the Indian-Eskimo. We expected that the between-category figures with faces and bodies would engage areas related to face and body perception, while the geometrical figures would be associated with activity in a brain area typically involved in seeing objects.

Areas related to face perception and surrounding regions were in fact specifically associated with perception of between-category figures. When we looked at the within-category figures, something surprising happened. We did not find activation in an area related to object recognition but instead a region much higher in the brain, namely the parietal lobe, an region located behind the top back part of you head. This region is involved in how you navigate in space. For instance you need the parietal lobe to know how far you need to extend you arm to reach the coffee cup in front of you. It is also an area known to be involved in mental rotation. When you try to imagine how an object looks when you turn it around, the parietal lobe is involved.
There is an influential theory about visual processing called the “Two stream hypothesis”. According to this theory, there are two main visual pathways where signals can continue after the primary visual cortex in the back of the head. One stream is called the “how pathway” or the “dorsal stream”, which ends in the parietal lobe and is involved in locating objects in space and directing actions. The other is called the “what pathway” or “ventral stream”, which goes to the temporal lobes and is linked with visual recognition. When people looked at between-category figures, there was clearly more brain activity along the ventral stream. It seems like people are constantly trying to categorize what they are looking at. It is possible that the brain interprets the geometrical within-category figures as being essentially the same objects, but seen from two different perspectives that flips back and forth. Therefore the geometrical figures are associated with activity along the “where” pathway. It seems like there is both general and more widespread brain activation related to bistable perception in general, and some brain activity specifically involved when seeing certain types of figures. It is possible that there is on-going interaction between local processing-specific areas and more regions that are more generally involved.

Ambiguity is everywhere

If you lay on your back and look up in the sky, at one moment there is a cloud that has a face and later a funny one in a dinosaur shape comes drifting by. On a dark and stormy night there might also be a shadow on the wall that looks like a hand with long bony fingers. There are a lot of situations where we deal with ambiguity. We need to categorize to be able to make sense of the world. But it seems like we also need to be open minded and not just settle for one visual interpretation and stick with it. Vision is to some extendtabout possibilities. The cube could be seen in a different perspective; the shadow on the wall could be someone sneaking up on you. We need to figure out what we are looking at, but at the same time we are also not perceptually stubborn. Being on guard and questioning the surroundings can be useful.

The results of this study show that there is both a more global network involved in bistable perception, and regions specific to the types of figures seen. Here we have explored how the brain processes two kinds of ambiguous stimuli. The geometrical within-category figures were associated with a brain region involved in the perception of space and mental rotation. The between-category figures that could be seen as either faces and bodies, evoked activation in brain areas involved in visual recognition and categorization. Ambiguity extends from simple bistable figures shown here to more open-ended ambiguities where there can be many possible interpretations. You may see someone with a mysterious look on his face walk down the street and wonder what that person is thinking about. Or find yourself standing in front of a piece of artwork where the motive cannot really be defined. You know from experience that it is not possible to see the same object from two different perspectives at the same time. Yet you are flexible enough to perceive both views changing back and forth. With more complex ambiguities there are not two, but many possible interpretations. Here, one interpretation might not excluded the other. Seeing a person being happy does not mean that same person cannot be confused as well.

It is quite amazing how we are able to experience our surroundings from so many perspectives and in so many different ways. Questioning what we see and seeking multiple possibilities seems to be part of our perceptual system on many levels. From simple drawings jumping between orientations to emotional expressions.