Bridging senses: new developments in synaesthesia

Writing in the 1880s, Francis Galton was intrigued by synaesthesia, observing that it tended to cluster in families. This talk will describe how views of the molecular underpinnings of this fascinating trait have evolved over the years that followed these early descriptions. It will introduce the diverse methods that have been used to gain insights into synaesthesia genetics, ranging from twin studies and linkage analysis, to case-control screens and next-generation DNA sequencing, while highlighting the promise and pitfalls of each approach. Initial work in this area hoped for explanation in terms of a single gene (or perhaps a few genes) of large effect, but it has become clear that the genomic architecture underlying synaesthesia is complex and multifactorial. Nonetheless, while there is significant heterogeneity between families/cases in terms of the specific genes involved, the first clues from studies of rare gene variants appear to highlight shared pathways. Ongoing investigations in extended families, as well as large numbers of unrelated cases, should help clarify this point. Crucially, the identification of genetic factors robustly associated with synaesthesia can open up new avenues of research, giving novel entry points into neurobiological mechanisms that are important for the way we experience the world.

Professor Simon E Fisher, Max Planck Institute for Psycholinguistics, The Netherlands

Professor Simon E Fisher, Max Planck Institute for Psycholinguistics, The Netherlands

Professor Simon E Fisher is director of the Max Planck Institute for Psycholinguistics and Professor of Language and Genetics at the Donders Institute for Brain, Cognition & Behaviour in the Netherlands. Professor Fisher obtained his Natural Sciences degree from Cambridge University, and his Human Genetics DPhil from Oxford University. From 1996-2002, as a postdoctoral researcher at Oxford's Wellcome Trust Centre for Human Genetics, he carried out genomic investigations of human neurodevelopment, and was co-discoverer of FOXP2, the first gene implicated in a speech and language disorder. From 2002-2010, Professor Fisher was a Royal Society Research Fellow leading his own research group at the Wellcome Trust Centre. In 2010 he became head of the newly established Language and Genetics Department at the Max Planck Institute in Nijmegen. His research investigates unusual human traits by integrating data from genomics, psychology, neuroscience, developmental biology and evolutionary anthropology. Awards include the Francis Crick Medal and Lecture, and the Eric Kandel Young Neuroscientists Prize.

People with synaesthesia experience their senses in unusual ways. For example, they might see colours when reading numbers, or experience tastes in the mouth when they hear sounds. These experiences start early in childhood - so what is it like to be a child with synaesthesia? Simner will describe findings from her ERC-MULTISENSE research project on childhood synaesthesia, which tested over 7 thousand children between the ages of 5 and 12 years of age and gave a battery of tests to the children, to their parents and to their teachers. She’ll describe a number of her novel tools which can diagnose a range of synaesthesias including those that trigger colours (eg the 7 is red), tastes (eg 7 tastes bitter), and even personality traits (eg 7 is a bossy child). She’ll describe some of her findings related to the prevalence of synaesthesia in schools, and the impact of childhood synaesthesia on literacy, numeracy, learning, creativity, personality, happiness and health-related well-being. She’ll also evaluate how schools might better equip themselves to deal with the needs of synaesthetic children and how the identification of synaesthesia early in childhood might bring about beneficial changes in later-life well-being.

Professor Julia Simner, University of Sussex, UK

Professor Julia Simner, University of Sussex, UK

Professor Julia Simner is a neuropsychologist and researcher in the field of synaesthesia research and multi-sensory processing. Her education and career have taken her to the Universities of Oxford, Toronto, Edinburgh and Sussex, and she currently runs the Synesthesia and Sensory Integration lab at the University of Sussex. She is the Science Officer for the UK Synaesthesia Association, Director of the ERC-funded MULTISENSE project, and Director of The Savant Network. She is the author of the upcoming book A Very Short Introduction to Synaesthesia, and her book, The Oxford Handbook of Synesthesia was recently shortlisted for the British Medical Association’s Medical Book of the Year.

Grapheme-colour synaesthetic association is consistent over time. Although graphemes do not elicit the same colour across different synaesthetes, some regularities in synaesthetic experience have been reported. In this talk, the authors present their recent work on factors that affect synaesthetic colours for Japanese Kanji characters (logographic characters). In the study, they first explored the influence meaning has on synaesthetic colours for Kanji characters representing abstract meanings by examining synaesthetic colours for antonym pairs (i.e. characters with meanings opposed to each other) in Japanese synaesthetes. Results show that antonym character pairs that were learned in lower school grades elicited synaesthetic colours that were dissimilar from each other. This suggests that semantic relations are reflected in synaesthetic colours, at least in the early stages of learning abstract Kanji characters. Second, the effect that learning new sounds or meanings of graphemes has on synaesthetic colours for those graphemes was examined. Japanese synaesthetes were taught new sounds or new meanings for familiar Kanji characters. Results indicate that the pre- and post-learning consistency of synaesthetic colour choice was slightly, but significantly, lower for Kanji characters for which new sounds/meanings were taught than for those for which no novel information was taught. This suggests that synaesthetic colours can be modulated to reflect the synaesthete’s latest knowledge about graphemes. Implications of these findings will be discussed in relation to the model of synaesthetic grapheme-colour association by Asano and Yokosawa (2013), which takes into account the developmental process of grapheme learning.

Associate Professor Michiko Asano, Rikkyo University, Japan

Associate Professor Michiko Asano, Rikkyo University, Japan

Michiko Asano, is an Associate Professor in the Department of Psychology, College of Contemporary Psychology at Rikkyo University in Japan, where she has been teaching from 2014 (Assistant Professor, April 2014—March 2018; Associate Professor, April 2018—present). She received her PhD in Psychology from the University of Tokyo in 2009. Asano has conducted experimental psychological research on grapheme-colour synaesthesia, visual cognition, word recognition, cross-modal correspondences, and language development. Her specific interest is in the relationship between grapheme-colour synaesthesia and language processing. Her native language, Japanese, enables a unique examination of the influence of various linguistic properties on grapheme-colour synaesthesia because it uses three different scripts: Hiragana, Katakana, and Kanji.

In the years since synaesthesia was proposed to be an essentially psycholinguistic phenomenon (Simner, 2007), a variety of research has begun to investigate the complex relationship between language and grapheme-colour synaesthesia. By identifying the elements of language that are reflected in synaesthetes’ colour experiences, researchers gain valuable insight into the underlying structure and processing of words. This talk explores how grapheme-synaesthesia can be used as a tool to test and investigate psycholinguistic theories.

In particular, this talk focuses on two aspects of words: structure and meaning. The former investigates how synaesthetic colours reflect the morphological structure of words – for example, how the synaesthetic colours of rain and bow result in the colour(s) of rainbow. These correspondences between colour and morphology may also provide insight into the processing of other morphological processes, such as affixing (eg re- + read = reread). However, words also convey meaning, and in some cases these meanings are already intrinsically coloured, such as the green of leaf or the blue of sky. Contrasting such canonically coloured words with similarly spelled control words (eg fire vs fine) shows that the synaesthetic colours of words are also influenced by the conceptual, canonical colours inherent in those words. The conclusion addresses some substantial gaps remaining in the psycholinguistic study of synaesthesia, particularly fundamental questions about the everyday experience of synaesthetes using language that will provide essential context for future research.

Dr Jennifer Mankin, University of Sussex, UK

Dr Jennifer Mankin, University of Sussex, UK

Jennifer Mankin is a Teaching Fellow at the University of Sussex, UK, focusing on cognitive psychology and research methods. She began studying synaesthesia with Professor Julia Simner in the Synaesthesia and Sensory Integration Lab at the University of Edinburgh whilst earning her Master’s degree, and has continued to work with the lab in Brighton. Her recent research is primarily focused on the interaction between synaesthesia and language, and the role of semantic information in the development and everyday experience of synaesthesia. She is also very interested in developing a more complex understanding of synaesthetic experiences by incorporating synaesthetes’ feedback in the research process.

This talk will give an overview of differences in cognitive ability between synaesthetes and non-synaesthetes; it will discuss whether these differences can potentially serve as a unifying behavioural marker for different varieties of synaesthesia; and whether they speak to the issue of whether synaesthesia is categorical in nature or continuous with neurotypical cognition. Specifically, evidence will be presented that people with synaesthesia show enhanced episodic memory, more vivid mental imagery, and show certain enhancements in perception. The latter resemble those found in autism. These traits are often shared across different kinds of synaesthesia and can even show dose-like effects: the more kinds of synaesthesia a person has, the more extreme the scores. It will be argued that these differences may be part of the neurodevelopment ‘start-up kit’ for synaesthesia, but that possession of these traits alone does not make someone a synaesthete (not even partially). Instead, it is proposed that synaesthesia is a discrete category that forces us to consider that there are multiple ways of being ‘normal’ – a lumpy rather than continuous space of neurodiversity.

Professor Jamie Ward, University of Sussex, UK

Professor Jamie Ward, University of Sussex, UK

Jamie Ward is Professor of Cognitive Neuroscience at the University of Sussex. He has an MA in Natural Sciences from Cambridge and PhD in Psychology from Birmingham, and previously held a faculty position at UCL.  He is a Co-Director of Sussex Neuroscience and was Founding Editor of the journal, Cognitive Neuroscience.  He has a particular research interest in synaesthesia and, more generally, in the question of how information is integrated between the senses.  He Co-Directs a Leverhulme Doctoral Programme in ‘Sensation, Perception and Awareness’ at the University of Sussex.  In addition to his research, he is well-known as an author of leading textbooks (Students Guide to Cognitive Neuroscience; Students Guide to Social Neuroscience).

Synaesthesia is a mixing of the senses, eg letters elicit colour. Approximately 20% of people with autism spectrum disorder (ASD) have synaesthesia: the reason for this high co-occurrence is unclear, but it is likely that underlying neural mechanisms are at least partly shared. In several experiments it was investigated whether perceptual alterations in synaesthetes resemble those reported for ASD. Visual perception tests were conducted and autistic traits assessed in >60 synaesthetes and >40 non-synaesthetes. Synaesthetes revealed stronger autistic traits on subscales of the Autism-Spectrum Quotient and reported increased sensory sensitivity compared to controls (Glasgow Sensory Questionnaire). Interestingly, in non-synaesthetes, the degree of synaesthesia as expressed in a consistency score correlated positively with AQ scores, suggesting a relationship between these traits in non-synaesthetes. Similar to reports in the ASD literature, motion coherence thresholds of synaesthetes were elevated (p<.001) suggesting that synaesthetes are less sensitive to global motion patterns. Motion coherence thresholds correlated with the sensory sensitivity score in synaesthetes (p<.001). Similar to reports of enhanced detail processing in ASD, synaesthetes made less errors than controls on an Embedded Figures Task (p<.05). Altogether the results point to a bias towards detail processing in synaesthetes, which is also typical of visual perception in ASD. Atypical sensory processing may be a shared feature of synaesthesia and ASD. Implications of the findings will be discussed.

Dr Tessa van Leeuwen, Radboud University, The Netherlands

Dr Tessa van Leeuwen, Radboud University, The Netherlands

Dr Tessa van Leeuwen obtained her Master’s in Biology at Utrecht University, the Netherlands and conducted her PhD research at the Donders Institute of Radboud University, Nijmegen. During her PhD she studied the neural correlates of grapheme-colour synaesthesia. After defending her PhD thesis in 2011 she moved to the Max Planck Institute for Brain Research in Frankfurt am Main, Germany, as a postdoc to investigate oscillations in synaesthetes and schizophrenia patients using MEG. In 2014 she returned to the Donders Institute in Nijmegen with a personal grant to study the suggested relationship between synaesthesia and autism spectrum disorder. She investigates autistic traits in synaesthetes and visual perception in children with ASD and children at familial risk for synaesthesia.

Our ability to imagine a visual scene or to hear music in our minds is a common everyday experience. For some people these mental images are extremely clear and vivid, whereas others report no image-like experience and merely the awareness that they are thinking about the image content. Research suggests that people who have synaesthesia have particularly vivid mental imagery, and that this may be linked to the senses involved in the synaesthesia.  For instance, people who experience colours with smells may have more vivid imagery for smells. Other research has looked at whether synaesthesia can be experienced from a visual image formed in the mind. For example, whether imagining a letter can induce the colour experience usually induced upon seeing such a letter. This talk will provide an overview of the research to date concerning mental imagery and synaesthesia. It will highlight key areas for future research including a focus on individual differences and the determining factors of the synaesthetic experience. Overall, the research discussed will raise important questions about the potential impact of our perceptual experiences on the development and capabilities of our visual spatial cognition.

Dr Mary Jane Spiller, University of East London, UK

Dr Mary Jane Spiller, University of East London, UK

Dr Mary Jane Spiller is a Senior Lecturer in Psychology at the University of East London. Whilst studying for her BSc in Psychology with Sociology from the University of Bath, UK, she had a year long work placement at Harvard University researching mental imagery with Prof Stephen Kosslyn. Following this, she worked at the Institute of Psychiatry, King's College London, before completing her PhD at the University of East London. For her PhD Mary Jane researched synaesthesia and mental imagery. She has since gone on to further explore this, as well as other aspects of synaesthesia, multisensory perception and mental imagery.

Dr Hubbard will review a series of studies that his group has recently completed examining the impact of synesthesia on memory.  Although there is now a substantial literature, spanning from Luria to modern investigations, demonstrating the impact of synesthesia on long-term memory (LTM), studies of earlier stages of memory remain sparse.  They propose that LTM advantages may arise from advantages during encoding arising at earlier stages of memory.  To test this hypothesis, they investigated the impact of grapheme-color synesthesia on three different stage of memory, from early visual sensory memory (referred to as “iconic memory”), visual working memory (WM), and long-term memory.  In all three studies, they found a synesthetic memory advantage.  Critically this advantage was most prominent when memory was stressed by asking participants to remember large arrays.  These findings suggest that memory load may be important for understanding some of the discrepancies in the literature, as some of the failures to find differences may not have stressed memory enough to reveal a synesthetic memory advantage.  They then frame these findings in terms of classic dual-coding theories (Pavio, 1971; Pavio, 1991) and the “sensory memory” account of working memory (see, eg Postle and D’Esposito, 2015 and a recent debate in TICS Xu, 2017; Gayet, Paffen & Van der Stigchel, 2018; Scimeca, Kiyonaga & D’Esposito, 2018; reply by Xu, 2018) and extend these investigations to multisensory stimulation in non-synesthetes.

Professor Edward Hubbard, University of Wisconsin-Madison, USA

Professor Edward Hubbard, University of Wisconsin-Madison, USA

Professor Edward M Hubbard is an Assistant Professor at the University of Wisconsin-Madison, where he directs the Educational Neuroscience Lab. He has been investigating the perceptual and neural bases of synesthesia for nearly 20 years, and has more than 30 publications on synesthesia. His work has appeared in specialist journals such as The Journal of Consciousness Studies, Neuron. Nature Reviews Neuroscience, and Science and popular journals such as Scientific American.  He is a founding member of the American Synesthesia Association and co-hosted the ASA meetings in 2001 at UCSD and 2010 at Vanderbilt.  In 2011, he was the keynote speaker for both the American Synesthesia Association and the UK Synaesthesia Assocation meetings.  He is the co-editor, with Julia Simner, of the Oxford Handbook of Synesthesia, published in December 2013, which was recognized as a “highly commended” book by the British Medical Association in 2014.