A smartphone held up in front of a person gazing to the right. The app GazeSpeak is seen on the phone screen"
(GazeSpeak, Enable Team, Microsoft Research)

Denne artikel findes på dansk: https://blog.bionyt.dk//ALS-sygdommen

The app GazeSpeak can help an ALS patient communicate – how does the app run ?
GazeSpeak runs on a smartphone and uses artificial intelligence to convert eye movements into speech, so a conversation partner can understand what is being said in real time.

The app runs on the listener’s device. They point their smartphone at the speaker as if they are taking a photo. A sticker on the back of the phone, visible to the speaker, shows a grid with letters grouped into four boxes corresponding to looking left, right, up and down. As the speaker gives different eye signals, GazeSpeak registers them as letters.

“For example, to say the word ‘task’ they first look down to select the group containing ‘t’, then up to select the group containing ‘a’, and so on,” says Xiaoyi Zhang, who developed GazeSpeak whilst he was an intern at Microsoft.

GazeSpeak selects the appropriate letter from each group by predicting the word the speaker wants to say based on the most common English words, similar to predictive text messaging. The speaker indicates they have finished a word by winking or looking straight ahead for two seconds. The system also takes into account added lists of words, like names or places that the speaker is likely to use. The top four word predictions are shown onscreen, and the top one is read aloud.

“We’re using computer vision to recognise the eye gestures, and AI to do the word prediction,” says Meredith Morris at Microsoft Research in Redmond, Washington.

The app is designed for people with motor disabilities like ALS, because eye movement can become the only way for people with these conditions to communicate. ALS progressively damages nerve cells, affecting a person’s ability to speak, swallow and eventually breathe. The eye muscles are often some of the last to be affected.

How many ways are there to communicate the people with ALS ?
There are currently limited options for people with ALS to communicate. The most common is to use boards displaying letters in different groups, with a person tracking the speaker’s eye movements as they select letters. But it can take a long time for someone to learn how to interpret these eye movements effectively. The app GazeSpeak can make the ALS patient communicate.
What are the benefits of using GazeSpeak for ALS patients?
GazeSpeak proved much faster to use in an experiment with 20 people trying both the app and the low-tech boards. Completing a sentence with GazeSpeak took 78 seconds on average, compared with 123 seconds using the boards. The people in the tests did not have ALS, but the team also got feedback on the technology from some people with ALS and their interpreters. One person who tried the device typed a test sentence in just 62 seconds and said he thought it would be even quicker in a real-life situation, as his interpreter can more easily predict what he is likely to say.
“I love the phone technology; I just think that would be so slick,” said one of the interpreters.
What kind of device do we need to use GazeSpeak-communication with ALS patients?
Other systems currently use software to track eye movements with infrared cameras. But these are often expensive and bulky, and infrared cameras don’t work very well in sunlight. The GazeSpeak app is portable and comparatively cheap, as it only requires an iOS device, like an iPhone or iPad, with the app installed.
When is the app GazeSpeak for communication with ALS patients released ?
Microsoft will present the app at the Conference on Human Factors in Computing Systems in Colorado in May.2017

Where could we download the app GazeSpeak for communication with ALS patients ?
The researchers say it will be available on the Apple App Store before the conference, and the source code will be made freely available so that other people can help to improve it.

Link: https://www.newscientist.com/article/2121579-microsoft-app-helps-people-with-als-speak-using-just-their-eyes/
(New Scientist 17 February 2017)

Scientist mentioned:Xiaoyi Zhang, 3rd year CS PhD student at University of Washington
http://www.xiaoyizhang.me/
xiaoyiz@cs.washington.edu

Tegn abonnement på

BioNyt Videnskabens Verden (www.bionyt.dk) er Danmarks ældste populærvidenskabelige tidsskrift for naturvidenskab. Det er det eneste blad af sin art i Danmark, som er helliget international forskning inden for livsvidenskaberne.

Bladet bringer aktuelle, spændende forskningsnyheder inden for biologi, medicin og andre naturvidenskabelige områder som f.eks. klimaændringer, nanoteknologi, partikelfysik, astronomi, seksualitet, biologiske våben, ecstasy, evolutionsbiologi, kloning, fedme, søvnforskning, muligheden for liv på mars, influenzaepidemier, livets opståen osv.

Artiklerne roses for at gøre vanskeligt stof forståeligt, uden at den videnskabelige holdbarhed tabes.