APEX Award holder, 2019-2020
University of the West of England
Exploring Molecular Data with Immersive Technology: Interactive Sonification in Virtual Reality
What is your research project about?
In much of science, data is represented using limited, often two-dimensional, visualisations. However, we are sophisticated multi-sensory beings that process our surroundings though a complex fusion of information across multiple perceptual channels.
My research explores immersive and interactive sonification; the use of sound to complement visual representations of scientific data in virtual reality. Together with scientist Dr David Glowacki (University of Bristol) and composer Professor Joseph Hyde (Bath Spa University), we are developing techniques that render vast quantities of data produced by a real-time molecular simulation as sound. My work seeks to enable scientists to more easily comprehend and explore nanoscale molecular phenomena using cutting-edge multimodal interaction technology.
What attracted you to apply for the APEX Award? What were the challenges that influenced you to seek support?
As a researcher working at the intersection of sound art and computer science, I typically have to locate my research projects within one or other of these connected disciplines. However, the APEX Award was refreshing as I was able to emphasise the equal importance of both domains, collaborating with leading scientists and composers to produce outputs with impact in both science and art.
What impact has your research had so far and might it have in the future?
We are developing generalised sonification tools for interactive molecular dynamics in virtual reality. Our focus to date has been on the use of sound to convey energetic molecular interactions that are difficult to represent visually. Our initial studies looking at drug design and molecular docking suggest that sound makes these tasks easier for scientists. We are conducting studies now to see whether non-experts can also complete these tasks with the support of sound, which would not be possible with visualisations alone.
Collaborating with a composer brings music and compositional theory into our design process which enhances the sonification aesthetics, making representations more ergonomic and usable. Consequently, the sonification has artistic merit, representing new forms of molecular sound synthesis that will feature in a number or arts and science installations over the next year.