Bridging senses: new developments in synaesthesia

Writing in the 1880s, Francis Galton was intrigued by synaesthesia, observing that it tended to cluster in families. This talk will describe how views of the molecular underpinnings of this fascinating trait have evolved over the years that followed these early descriptions. It will introduce the diverse methods that have been used to gain insights into synaesthesia genetics, ranging from twin studies and linkage analysis, to case-control screens and next-generation DNA sequencing, while highlighting the promise and pitfalls of each approach. Initial work in this area hoped for explanation in terms of a single gene (or perhaps a few genes) of large effect, but it has become clear that the genomic architecture underlying synaesthesia is complex and multifactorial. Nonetheless, while there is significant heterogeneity between families/cases in terms of the specific genes involved, the first clues from studies of rare gene variants appear to highlight shared pathways. Ongoing investigations in extended families, as well as large numbers of unrelated cases, should help clarify this point. Crucially, the identification of genetic factors robustly associated with synaesthesia can open up new avenues of research, giving novel entry points into neurobiological mechanisms that are important for the way we experience the world.

People with synaesthesia experience their senses in unusual ways. For example, they might see colours when reading numbers, or experience tastes in the mouth when they hear sounds. These experiences start early in childhood - so what is it like to be a child with synaesthesia? Simner will describe findings from her ERC-MULTISENSE research project on childhood synaesthesia, which tested over 7 thousand children between the ages of 5 and 12 years of age and gave a battery of tests to the children, to their parents and to their teachers. She’ll describe a number of her novel tools which can diagnose a range of synaesthesias including those that trigger colours (eg the 7 is red), tastes (eg 7 tastes bitter), and even personality traits (eg 7 is a bossy child). She’ll describe some of her findings related to the prevalence of synaesthesia in schools, and the impact of childhood synaesthesia on literacy, numeracy, learning, creativity, personality, happiness and health-related well-being. She’ll also evaluate how schools might better equip themselves to deal with the needs of synaesthetic children and how the identification of synaesthesia early in childhood might bring about beneficial changes in later-life well-being.

Grapheme-colour synaesthetic association is consistent over time. Although graphemes do not elicit the same colour across different synaesthetes, some regularities in synaesthetic experience have been reported. In this talk, the authors present their recent work on factors that affect synaesthetic colours for Japanese Kanji characters (logographic characters). In the study, they first explored the influence meaning has on synaesthetic colours for Kanji characters representing abstract meanings by examining synaesthetic colours for antonym pairs (i.e. characters with meanings opposed to each other) in Japanese synaesthetes. Results show that antonym character pairs that were learned in lower school grades elicited synaesthetic colours that were dissimilar from each other. This suggests that semantic relations are reflected in synaesthetic colours, at least in the early stages of learning abstract Kanji characters. Second, the effect that learning new sounds or meanings of graphemes has on synaesthetic colours for those graphemes was examined. Japanese synaesthetes were taught new sounds or new meanings for familiar Kanji characters. Results indicate that the pre- and post-learning consistency of synaesthetic colour choice was slightly, but significantly, lower for Kanji characters for which new sounds/meanings were taught than for those for which no novel information was taught. This suggests that synaesthetic colours can be modulated to reflect the synaesthete’s latest knowledge about graphemes. Implications of these findings will be discussed in relation to the model of synaesthetic grapheme-colour association by Asano and Yokosawa (2013), which takes into account the developmental process of grapheme learning.

In the years since synaesthesia was proposed to be an essentially psycholinguistic phenomenon (Simner, 2007), a variety of research has begun to investigate the complex relationship between language and grapheme-colour synaesthesia. By identifying the elements of language that are reflected in synaesthetes’ colour experiences, researchers gain valuable insight into the underlying structure and processing of words. This talk explores how grapheme-synaesthesia can be used as a tool to test and investigate psycholinguistic theories.

In particular, this talk focuses on two aspects of words: structure and meaning. The former investigates how synaesthetic colours reflect the morphological structure of words – for example, how the synaesthetic colours of rain and bow result in the colour(s) of rainbow. These correspondences between colour and morphology may also provide insight into the processing of other morphological processes, such as affixing (eg re- + read = reread). However, words also convey meaning, and in some cases these meanings are already intrinsically coloured, such as the green of leaf or the blue of sky. Contrasting such canonically coloured words with similarly spelled control words (eg fire vs fine) shows that the synaesthetic colours of words are also influenced by the conceptual, canonical colours inherent in those words. The conclusion addresses some substantial gaps remaining in the psycholinguistic study of synaesthesia, particularly fundamental questions about the everyday experience of synaesthetes using language that will provide essential context for future research.

This talk will give an overview of differences in cognitive ability between synaesthetes and non-synaesthetes; it will discuss whether these differences can potentially serve as a unifying behavioural marker for different varieties of synaesthesia; and whether they speak to the issue of whether synaesthesia is categorical in nature or continuous with neurotypical cognition. Specifically, evidence will be presented that people with synaesthesia show enhanced episodic memory, more vivid mental imagery, and show certain enhancements in perception. The latter resemble those found in autism. These traits are often shared across different kinds of synaesthesia and can even show dose-like effects: the more kinds of synaesthesia a person has, the more extreme the scores. It will be argued that these differences may be part of the neurodevelopment ‘start-up kit’ for synaesthesia, but that possession of these traits alone does not make someone a synaesthete (not even partially). Instead, it is proposed that synaesthesia is a discrete category that forces us to consider that there are multiple ways of being ‘normal’ – a lumpy rather than continuous space of neurodiversity.

Synaesthesia is a mixing of the senses, eg letters elicit colour. Approximately 20% of people with autism spectrum disorder (ASD) have synaesthesia: the reason for this high co-occurrence is unclear, but it is likely that underlying neural mechanisms are at least partly shared. In several experiments it was investigated whether perceptual alterations in synaesthetes resemble those reported for ASD. Visual perception tests were conducted and autistic traits assessed in >60 synaesthetes and >40 non-synaesthetes. Synaesthetes revealed stronger autistic traits on subscales of the Autism-Spectrum Quotient and reported increased sensory sensitivity compared to controls (Glasgow Sensory Questionnaire). Interestingly, in non-synaesthetes, the degree of synaesthesia as expressed in a consistency score correlated positively with AQ scores, suggesting a relationship between these traits in non-synaesthetes. Similar to reports in the ASD literature, motion coherence thresholds of synaesthetes were elevated (p<.001) suggesting that synaesthetes are less sensitive to global motion patterns. Motion coherence thresholds correlated with the sensory sensitivity score in synaesthetes (p<.001). Similar to reports of enhanced detail processing in ASD, synaesthetes made less errors than controls on an Embedded Figures Task (p<.05). Altogether the results point to a bias towards detail processing in synaesthetes, which is also typical of visual perception in ASD. Atypical sensory processing may be a shared feature of synaesthesia and ASD. Implications of the findings will be discussed.

Our ability to imagine a visual scene or to hear music in our minds is a common everyday experience. For some people these mental images are extremely clear and vivid, whereas others report no image-like experience and merely the awareness that they are thinking about the image content. Research suggests that people who have synaesthesia have particularly vivid mental imagery, and that this may be linked to the senses involved in the synaesthesia.  For instance, people who experience colours with smells may have more vivid imagery for smells. Other research has looked at whether synaesthesia can be experienced from a visual image formed in the mind. For example, whether imagining a letter can induce the colour experience usually induced upon seeing such a letter. This talk will provide an overview of the research to date concerning mental imagery and synaesthesia. It will highlight key areas for future research including a focus on individual differences and the determining factors of the synaesthetic experience. Overall, the research discussed will raise important questions about the potential impact of our perceptual experiences on the development and capabilities of our visual spatial cognition.

Dr Hubbard will review a series of studies that his group has recently completed examining the impact of synesthesia on memory.  Although there is now a substantial literature, spanning from Luria to modern investigations, demonstrating the impact of synesthesia on long-term memory (LTM), studies of earlier stages of memory remain sparse.  They propose that LTM advantages may arise from advantages during encoding arising at earlier stages of memory.  To test this hypothesis, they investigated the impact of grapheme-color synesthesia on three different stage of memory, from early visual sensory memory (referred to as “iconic memory”), visual working memory (WM), and long-term memory.  In all three studies, they found a synesthetic memory advantage.  Critically this advantage was most prominent when memory was stressed by asking participants to remember large arrays.  These findings suggest that memory load may be important for understanding some of the discrepancies in the literature, as some of the failures to find differences may not have stressed memory enough to reveal a synesthetic memory advantage.  They then frame these findings in terms of classic dual-coding theories (Pavio, 1971; Pavio, 1991) and the “sensory memory” account of working memory (see, eg Postle and D’Esposito, 2015 and a recent debate in TICS Xu, 2017; Gayet, Paffen & Van der Stigchel, 2018; Scimeca, Kiyonaga & D’Esposito, 2018; reply by Xu, 2018) and extend these investigations to multisensory stimulation in non-synesthetes.