“The Colour Out of Space”, a short story written by H. P. Lovecraft, depicts the impact of a meteorite crash in a fictional town, leaving behind globules of color that fall outside the range of the known visible spectrum. To quote a sentence, “the colour, which resembled some of the bands in the meteor’s strange spectrum, was almost impossible to describe; and it was only by analogy that they called it colour at all.” Now, let’s imagine you have encountered this color that does not resemble any other color you have seen before. Would it really be impossible to describe? Or you might still try to describe it by comparing it to other sensory experiences such as a certain sound, taste or smell. This does not have to be taken as too unreal, considering that many people will agree on that orange is a “warmer” color than blue. Actually, describing a certain color as warm or cold is so common that we don’t see anything special about it. In everyday language, a sound can be “sweet”, and a taste can be “light”.

Seeing sounds and tasting colors

For some people however, this association across different senses (seeing, hearing, tasting, smelling and touching) is exceptionally more vivid and real. This is called synesthesia, a condition where stimulation of one sense causes unusual experiences in other senses. Seeing sounds and tasting colors might sound like some kind of fictional superpower, but it has actually been a topic of scientific studies.

Synesthesia has several subtypes depending on the senses involved. Auditory-visual synesthesia is a type of synesthesia in which heard sounds evoke an experience of shape and color. People with spatial sequence synesthesia see a sequence of numbers as points in space, which might come in handy when you are learning mathematics. The best-studied form of synesthesia is grapheme-color synesthesia, in which alphabets or numbers printed in black and white induce experience of color.

As an example, you can imagine a grapheme-color synesthete who perceive 2 and 5 as having distinct colors (e.g. green for 2, brown for 5). When asked to search for 2s among 5s, they are expected to be better at it than non-synesthetes, since they will experience the 2s as popping out [1]. It should be easier to find green numbers between brown numbers than to find 2s among 5s all printed in black only by the shape.

However, synesthesia can also interrupt with daily life. We can imagine a synesthete screaming in front of a McDonald´s: “No! The sign is so wrong. M has to be blue, not yellow!” They do have their own associations between graphemes and colors, and when real-life representations of graphemes do not match these associations, they go through some confusion. This is demonstrated with a paradigm called “synesthetic Stroop task”, which has shown that it is harder for a synesthete to process a grapheme written in a wrong color than in the correct one[2].

Above-mentioned evidences show that synesthetic experiences are automatic and involuntary. However, the true nature of the phenomenon is hard to observe and thus still being debated. Hupe et al. (2012) pointed out that many attempts to obtain objective information on the nature of synesthesia by studying brains yielded to unsystematic results or were not relevant to real color-sensitive areas[3]. This calls for a new approach for studying synesthetic brains and probably redefining the concept of synesthesia. In the following, we will be taking a look at some recent findings that attempt to deal with this matter.

The neural basis of synesthesia

Then, what is going on within the brains of synesthetes? For exploring the topic, neuroscientists have to take into account individual variability in perception, as each synesthete can have a unique set of associations. This variability between subjects along with the limited number of samples results in low statistical power.

Recently, researches are thus starting to investigate global difference between synesthete and nonsynesthete brains in contrast to earlier studies that focused on single brain areas. Because synesthesia is about binding of different senses, this seems indicative of a connection between different sensory-specific brain areas in synesthetes. The question is whether this connection is direct, or mediated by a convergence area. In a study observing auditory-visual synesthetes, people with synesthesia showed stronger connectivity of the left inferior parietal cortex with the auditory and vision specific areas but no stronger direct connections between the auditory and the visual areas, supporting the latter model [4].

Another research suggests that grapheme-color synesthesia is likely to be caused by increased connectivity in right retinosplenial cortex, which has links with memory, vision and emotion system [3]. Such an increase in connectivity will result in an increased exchange of information between areas.  When different senses are strongly connected, they are in turn less independent from each other. In synesthetic brains, boundaries between senses are thus weaker. For instance, vision in the brain of a visual-auditory synesthete is less distinct from the rest of the senses (especially from hearing, in this example). In summary, the neural basis of synesthesia is likely to be a global mechanism, rather than local.

Synesthesia in non-synesthetes

So far, we have talked about what synesthesia is and how it works. But when it is observed in only a proportion of people (1-5 % for grapheme-color synesthesia), why is it relevant to all of us? There is a phenomenon known as the “Kiki-Bouba effect” that answers this question. First observed in 1929, it refers to a non-arbitrary mapping between sounds and the visual shape of objects. Non-synesthetic participants were asked to decide which of the pointed or rounded shape is called “Kiki” or ”Bouba”, and the majority matched the pointy shape with Kiki and the other with Bouba.

A possible explanation is that the mouth makes a rounded shape when pronouncing “Bouba” and it makes us associate it with a round object. When you think about the fact that many synesthetic associations follow some systematic rules, this might be the key to exploring where synesthesia stems from. For example, there was a significant tendency for synesthetes to associate “o”, “i”, “u” with white more than with other colors. Whereas this is not so obvious as in Kiki-Bouba case, this association might also have a similar plausible explanation.

Furthermore, synesthesia-like condition was able to be induced in a nonsynesthetic population [5]. Subjective experience after reading books with colored letters could be predicted from the visual cortex activity. This shows that it is possible to get a synesthesia-like ability by training of cross-modal associations. Bor et al. also trained nonsynesthetic adults to learn grapheme-color associations [6]. After 9 weeks, they could pass several tests used for assessing synesthesia. This explains why genes seem to have only a moderate influence on synesthesia and indicates that neuronal plastictity plays an important role in it. Of course, it is yet to be further explored whether learned associations are as long-lasting and robust as in real synesthesia, and how much of the phenomenon learning can account for. Still, this finding raises the possibility of synesthesia being an exaggeration of normal cross-modal perception.

Implications of synesthesia research

Going back to the example of “the colour out of space”, let us think about practical benefits of synesthesia research. What if synesthetic associations enable us to describe not only a certain color, but visual experience as a whole? This means that we might be able to aid blind people to see through other senses. This technique called sensory substitution means to transform information acquired by one sensory modality into stimulation of another sensory modality. It can be viewed as an artificially acquired synesthesia, in that they both involve perceptual experiences induced by a stimulus that is qualitatively different from what normally gives rise to that experience [7]. To design sensory substitution devices in an optimal way, knowing about natural synesthesia will be very useful.

We can also relate synesthesia to second language acquisition process and how language interacts with perception. In 2016, Hamada et al. published a large collection of synesthetic color associations for Japanese kanji. It is a valuable resource for studying a letter system that is logographic, where the letter shapes are indicative of the meaning, unlike Western alphabets [8]. Eventually, it will lead us to the “hard problem of consciousness” in mind philosophy, which is a question of how to investigate the private, unobservable psychological experiences, of an individual. Though it would not be an essential solution to the problem, at least it gives us a clue how we might be able to imagine what it is like to be a bat, experiencing space with bio sonar.

Conclusion

  Synesthesia is a key to understanding of many complex aspects of what we experience. Integration and association of difference senses enrich our perceptual experiences, enabling us to find relevant names for objects and coming up with a metaphor that can be shared with other people. In addition to telling you how to properly design a character that looks, sounds or smells very evil, knowing about synesthesia can expand our realm of knowledge in language and consciousness, which has been one of the biggest mysteries of human mind.

References

[1] Ward, J. & Wright, T. 2014. Sensory substitution as an artificially acquired syneasthesia. Neuroscience & Biobehavioral Reviews, 41, 26-35

[2] Hamada, D., Yamamoto, H. & Saiki, J. 2016. Database of synesthetic color associations for Japanese kanji. Behavior Research Methods, 1-16.

[3] Neufeld, J., Sinke, C., Zedler, M., Dillo, W., Emrich, H.M., Bleich, S. & Szycik, G.R. 2012. Disinhibited feedback as a cause of synesthesia: evidence from a functional connectivity study on auditory-visual synesthetes. Neuropsychologia, 50(7), 1471-1477.

[4] Colizoli, O., Murre, J.M.J., Scholte, H.S., van Es, D.M., Knapen, T. & Rouw, R. 2015. Visual cortex activity predicts subjective experience after reading books with colored letters. Neuropsychologia, in press. http://dx.doi.org/10.1016/j.neuropsychologia.2015.07.006i

[5] Bor, D., Rothen, N., Schwartzman, D.J., Clayton, S. & Seth, A.k. 2014. Adults can be trained to acquire synesthetic experiences. Scientific Reports, 4, 7089.

[6] Ramachandran, V.S. & Hubbard, E.M. 2001. Synesthesia: a window into perception, thought and language. Journal of Consciousness Studies, 12. 3-34.

[7] Hubbard E.M. & Ramachandran, V.S. 2005. Neurocognitive mechanisms of synesthesia. Neuron, 48, 509-520.

[8] Hupe, J., Bordier, C. & Dojat, M. 2012. The neural bases of grapheme-color synesthesia are not localized in real color-sensitive areas. Cerebral Cortex, 22, 1622-1633.