Microsoft’s 3D Soundscape Technology research helps visually impaired

The Windows Phone-powered technology brings audio cues through a bone-conducting headset.

Microsoft has joined forces with Guide Dogs UK to test a prototype version of its 3D Soundscape Technology. The aim: Help visually impaired people confidently and safely navigate their surroundings.

“At Microsoft, we have a long history of innovation in accessibility and we’re always looking for ways to empower people with disabilities by creating technologies that support the unique ways in which each person needs to interact,” explains Jenny Lay-Flurrie, senior director of accessibility at Microsoft and herself a person with a disability.

“We spent months with people with sight loss, as well as industry experts, to try to understand the difficulties they experience in various situations to build the right empathy of that experience and let that guide the design of our concept product.”

The technology combines an off-the-shelf Windows Phone handset (such as a Lumia) with a bone-conducting headset to deliver a multitude of useful audio cues to the inner ear of the wearer. This keeps the ear uncovered and ensures the wearer can still hear environmental noise and take part in conversations.

The 3D Soundscape technology utilizes the smartphone’s GPS and accelerometer sensors while picking up on prompts from specially installed Bluetooth and Wi-Fi-enabled beacons dotted along the route and inside buildings. Cloud based location and navigation data from Bing Maps provides enhanced contextual information such as shops, points of interest, and additional journey details.

Wearers will receive turn-by-turn directions to their destination, while the headset will send audible pings if they are straying from the chosen path or verging too close to the edge of the pavement.

The Windows Phone handset will also call out local points of interest such as the post office, coffee shop or newsagents, and even the odd slice of historical information.

The first trials for the initiative have been undertaken with the assistance of Future Cities Catapult, a UK-based urban design agency. They are currently helping test subjects make the journey between Reading and central London.

So, as well as safe and reliable walking directions, the technology also delivers real-time public transport information. Wearers will hear useful cues like “your bus stop is coming up in 25 feet” and “your train will be here in five minutes.”

These updates will become more powerful and useful as the system learns more about the user and their personal habits. For example, the cue: “You’re about to pass that pub you visited last Thursday,” could come in handy!

While the idea is still very much in the proof-of-concept phase, with many challenges to overcome, the hope is for 3D Soundscape technology to one day empower the hundreds of millions of visually impaired people around the world with an unobtrusive solution that makes towns and cities more accessible than ever before.

Sixty-two percent of participants reported amplified feelings of safety, confidence and resilience while out and about on their travels, allowing them to relax more into their journey. Many are keen to continue wearing the headset once the trial is complete.

“In the UK, there are about 180,000 people who rarely if ever get out and about. That’s a massive social issue in its own right,” said Richard Leaman, CEO of Guide Dogs.

“That was really the heart of our new strategy. Supporting blind people with 5,000 guide dogs is really not scratching the surface. Introducing new services, like the use of technology, is a vital ingredient to change,” he said.

“If our major cities in the UK had sensor-enriched zones where this capability worked, I think it would be an incredible achievement. Beyond that, I expect this to go global, as I’m sure major cities around the world will want the same. It’s like an awakening.”

While this is just a proof-of-concept for now, if you are a person with sight loss, “know that there’s support specifically for you,” Lay-Flurrie said.

“The Disability Answer Desk is in the UK, U.S. and more, and supports customers with disabilities using Microsoft technology. We want to help and enable people everywhere to be successful.”

To access the Disability Answer Desk, please click here.

You can find out more about the 3D Soundscape Technology trials over at Microsoft’s Independence Day microsite.