Groundbreaking New Technology Allows People To Listen to Music Through Touch

Various Music Notes Rainbow

The prototype consists of an audio-tactile algorithm that converts monophonic music into tangible stimuli based on vibration utilizing “tactile illusions.”

An audio-tactile algorithm created by University of Malaga scientists conveys melodic information through vibration.

A ground-breaking prototype developed by experts from the Department of Electronics at the University of Malaga and members of the R&D group “Electronics for Instrumentation and Systems,” will allow those with hearing loss to listen to music through the sense of touch.

It consists of an audio-tactile algorithm that transforms monophonic music into tangible stimuli based on vibration utilizing “tactile illusions.” According to the researchers, “It’s like ‘hacking’ the nervous system to receive a different response to the real stimulus sent.”

“What we want to achieve in the long term is for people who do not hear to be able to ‘listen’ to music”, assures researcher Paul Remache, the main author of this paper, who insists on the power of music to influence mood, as well as its possibilities as a therapy for mental disorders and treatment of pain.

The researchers predict that this will lead to a portable terminal that may be brought to a concert since this prototype will be easily transferable to technological devices like smartphones.

Researchers of the Department of Electronics have developed an audio-tactile algorithm that conveys melodic information through vibration. Credit: University of Malaga

Mapping music

This young researcher, working with professors Andrés Trujillo and Fernando Vidal from the UMA, developed an algorithm that can transform musical features and structures taken from MIDI files—Musical Instrument Digital Interface—into “vibrotactile stimuli.”

The first experiments proved that “tactile illusions” perceived through vibration provoke in people who do not hear a different emotional response –mostly positive– from that of the original music. Credit: University of Malaga

“It’s something similar to mapping music”, explains Remache, who adds that this is possible because this type of file not only can be played and generate sound, but also provide “symbolic representations”.

Controlling vibrations

Current models do not warrant the correspondence between the emotional response to music and the vibrotactile version of it. In view of this, these engineers of the UMA propose an arrangement of the “tactile illusions” in order to improve and extend the spectrum of musical features, adding dynamics to the vibration in the form of movement, changes of direction, and location.

“It is a challenging process since the perceptible frequency range of the skin is lower than that of the auditory system, which may cause the loss of some musical features”, they explain.

People with hearing loss will be able to listen to music through the sense of touch thanks to a pioneering prototype that has been devised by researchers of the Department of Electronics of the University of Malaga. Credit: University of Malaga

Different emotional response

The results of the first experiments, in which more than fifty volunteers participated, indicate that the arrangement of “tactile illusions” elicits more positive than negative emotions. They are also perceived as more agreeable and stimulating than the audio, provoking a different emotional response from that of the original music.

Smart instrumentation and application in healthcare

This first prototype was presented in the 11th International Workshop on Haptic & Audio Interaction Design (United Kingdom) –the biggest international event specialized in these areas of study– after being published in the scientific journal LNCS. At present, the researchers of the UMA are working on a second model and continuing with the experiments.

The research is the product of Paul Remache’s doctoral thesis and is part of the National Plan project “Smart instrumentations and application in healthcare.”

Reference: “Mapping Monophonic MIDI Tracks to Vibrotactile Stimuli Using Tactile Illusions” by Byron Remache-Vinueza, Andrés Trujillo-León, Maria-Alena Clim, Fabián Sarmiento-Ortiz, Liliana Topon-Visarrea, Alexander Refsum Jensenius and Fernando Vidal-Verdú, 18 August 2022, Lecture Notes in Computer Science.
DOI: 10.1007/978-3-031-15019-7_11


View Comments

  • I'm not sure I understand this. None conduction is ready a thing and harmonic frequencies down to the sub level already trigger vibrations you can feel so...

  • Even with bone conduction the vibrations move scilia in the cochlea, so if those are damaged or is the audatory nerve etc. (hearing in general) isn't working someone still won't hear.
    I'm guessing the mapping takes the frequencies of sound over the maybe larger range of frequencies the ear can hear and squishes or moves them to a range that can be felt instead, and then to vibrations. I think it's a neat idea, for the hearing impaired.

University of Malaga

Recent Posts

Decades-Old Mystery Solved? Astronomers May Have Uncovered How Galaxies Change Their Shape

By using artificial intelligence, researchers have potentially solved a long-standing question about the evolution of…

January 28, 2023

Mysteries Remain About Puzzling Bahamas Whiting Events

Researchers used satellite imagery to reveal where and when the discolored patches of water appeared…

January 28, 2023

Rotating Detonation Rocket Engine: NASA’s Revolutionary Propulsion Design for Deep Space Missions

As NASA takes its first steps toward establishing a long-term presence on the Moon’s surface,…

January 28, 2023

Important Advance in Quantum Science: Coherent Manipulation of Spin Qubits at Room Temperature

A team headed by Professor Wu Kaifeng from the Dalian Institute of Chemical Physics (DICP)…

January 28, 2023

Healthy Plant-Based Diets Could Lower Men’s Risk of Colon Cancer

A diet rich in whole grains, vegetables, and legumes and low in refined grains, fruit…

January 28, 2023

“Green” Biodegradable Medical Gowns Actually Produce Harmful Greenhouse Gases

According to new research published in the Journal of Cleaner Production, biodegradable medical gowns, which…

January 28, 2023