Podcast: Play in new window | Download
Your weekly dose of information that keeps you up to date on the latest developments in the field of technology designed to assist people with disabilities and special needs.
Microsoft’s Soundscape with Amos Miller, Product Lead | aka.ms/soundscape
Podcasts to Go: How to Play Podcasts on your iOS Device – AccessWorld® – June 2018 http://bit.ly/2KlrzzW
——————————
If you have an AT question, leave us a voice mail at: 317-721-7124 or email tech@eastersealscrossroads.org
Check out our web site: https://www.eastersealstech.com
Follow us on Twitter: @INDATAproject
Like us on Facebook: www.Facebook.com/INDATA
——-transcript follows ——
AMOS MILLER: Hi, I’m Amos Miller, product lead for Soundscape with Microsoft Research, and this is your Assistive Technology Update.
WADE WINGLER: Hi, this is Wade Wingler with the INDATA Project at Easter Seals Crossroads in Indiana with your Assistive Technology Update, a weekly dose of information that keeps you up-to-date on the latest developments in the field of technology designed to assist people with disabilities and special needs.
Welcome to episode number 370 of Assistive Technology Update. It’s scheduled to be released on June 29, 2018. Today we have a fascinating interview with James Miller, who is the product lead for Microsoft soundscape at Microsoft research. We are going to spend some time talking today about how their technology can help people who are blind or visually impaired navigate environment. We hope you check out our website at EasterSealsTech.com. Send us a note on Twitter at INDATA Project. Or call our listener line. We love to hear your questions and comments. You might even hear your voice on this or one of our other shows. The number is 317-721-7124
I’m guessing if you are listening to the show, you might already know about how to listen to podcasts. But I saw an article recently from Janet Ingber from AFB’s Access World Magazine that I thought was really good. It’s called “Podcasts to Go: how to play podcasts on your iOS device.” If you’re wanting to pass for now, that’s okay. But know that there are some folks who are still learning about podcasts, and this is a great way for you to help them figure out how to listen to podcasts. And if you are only listening on and on mobile device, this will show you how to do it on your iOS device. Janet does a great job explaining different pod catchers and how they work. She talks about the Saro app from Sarotech and how you can use that to listen to podcasts. She spent some time talking about apples podcast app and also the popular Downcast app. She does a nice job of getting into the accessibility on each one and what works and what doesn’t. She even mentions one call iCatcher. Some of them are free, some of them cost a little bit, but article does a nice overview of how to find podcasts, how to use your accessible technology to get to them, and she even talks about some of her personal experiences of being a podcast listener. I’m going to pop a link in our show notes over to access world Magazine’s post by Janet Ingber. The title is “Podcasts to Go: how to play podcasts on your iOS device.” Check our show notes.
***
[2:39] Interview with Amos Miller***
WADE WINGLER: We all care a lot about what’s happening in the world around us. As we go places, as we learn things, as we really live our lives, it’s important to know what’s in the environment. Recently we’ve had a series of conversation with folks at Microsoft about new and emerging technology they are developing to enhance the lives of people with disabilities. Today is no exception. I’m super excited that we’re going to be speaking today with Amos Miller, who is the product lead for Microsoft soundscape. I know you’ve heard about it. It’s been in the news quite a lot lately. We are excited to get some other details about that today.
Amos, thank you so much and welcome to the show.
AMOS MILLER: Thank you, Wade. It’s good to be here.
WADE WINGLER: Before we start talking about soundscape, tell me a little bit about yourself, your background and your journey to becoming the product lead at Microsoft for this exciting thing called soundscape.
AMOS MILLER: Okay. That’s an interesting journey. I am visually impaired myself. Most of my career has been in computer science and consulting in topics that are completely unrelated to my own situation with a disability. I grew up in Israel actually. I spent most of my professional career in the UK. After I joined Microsoft about 10 or 11 years ago, I had an opportunity to volunteer for the Guide Dogs Organization in the U.K. That’s where we started to explore the opportunity for technology to make a meaningful impact for mobility for blind people. One thing led to another, and we explored some ideas. One of them became soundscape. Over the years, that product that started over the evenings and weekends took shape and became a product that Microsoft was able to get behind and make available to the community.
WADE WINGLER: All right, Amos, we had a little sound trouble there and we actually have switched systems. But I think we are connected again. You’re there, right?
AMOS MILLER: I am here.
WADE WINGLER: Very good. Here we go. Why do we need something like soundscape? Who uses it?
AMOS MILLER: Soundscape is a technology that’s used largely by people who are visually impaired to enhance their awareness of their surroundings. The idea is that people who navigate or away find outdoors primarily will be using a cane, a dog, a sighted guide, whatever mobility aids they have. Sometimes still have a reasonable awareness of their surroundings if they are familiar with it, sometimes not. What soundscape tries to do is really enriched that awareness, help people get a sense for what’s around, where things are and thereby feel more confident to take decisions, to just wander, to go to places. One of the special aspects that we often try to look at and work on is the whole notion that if we think about technology purchase putting in our experience outdoors, and having a meaningful impact on our experience, the technology can’t interfere with that experience. The moment that you are walking down the road with your child, and the interaction with the technology takes your attention away from your child or your experience in the zoo or walking in the park, then the technology gets in the way and really detracts from the experience, even if it has some functional value.
The question is if we think about how technology is integrated into such experiences, how do you design something that enables you and enhances that experience, whatever that experience is, in a way that doesn’t take away from that.
WADE WINGLER: Like Goldilocks, right? Not too little, not too much, just right.
AMOS MILLER: And appropriate. Yes. We don’t really see it about quantity. It’s about manners and ways in which — if you are talking and having a conversation with someone, and a third person interrupts that conversation, there is a way to do that that is respectful of your other conversation. There is also the whole element of how you as a person process information from the cognitive perspective and how we as humans are extremely capable at dealing with multiple channels of information coming at us at the same time. For example, you could be sitting on the bus, having a conversation with your child, and somebody will be having a conversation behind you. All of a sudden, they’ll say words on a topic that are of interest to you. You are likely to notice that. Your brain is paying attention to stuff that you may not be attending to. Those capabilities, those innate human capabilities, are already there. We already have those. We are already use those very, very well. If technology can play well with those abilities, then we can integrate it in a better way to our experience.
WADE WINGLER: As you talk about that human experience, that totally resonates with me. Everything from described video where there is a small of time that’s being filled in with the right amount of information, or sitting at a family gathering where I try to listen to a couple of conversations and can kind of pull that off and drift back and forth between the conversations. That resonates with me. That makes sense.
AMOS MILLER: Exactly. The way you are going with your partner to the shops on a Sunday afternoon, you are walking down the street, you are both taking in what’s around you. You might also be having a conversation, whether it is relevant to the task at hand or something else. How does the technology come into play? How do you inform — if you are visually impaired and want to know the kinds of shops that you are walking past, but you don’t want that information to be something you have to attend to and stop the conversation. Like a driver with the GPS on the car, every time the GPS makes an announcement, everybody has to be quiet because they have to concentrate, right? That’s not an integrative experience. It pauses everything else for you to handle that. Those are the kinds of nuances we try to enable.
We had good evidence that that’s one of the detractor from people adopting these technologies. Since we’ve been doing a lot of studies around that topic and also soundscape, we’ve seen a lot of positive indications that there is something in that space to do and provide.
WADE WINGLER: Tell me about the basics. What’s the hardware and software look like with soundscape?
AMOS MILLER: Soundscape in its current incarnation is effectively an app on the iPhone. The trick and uniqueness of the soundscape is you wear stereo headphones, whether those bone connection headphones or earbuds, whatever headphones you are most comfortable with to where outside. You do need both ears, because what soundscape will do is it will present information in 3-D space around you. For example, you will be standing at a street corner, and Starbucks might be in front of you and to the right. You would hear the word Starbucks emanating from that direction. And if you turn to your right and face Starbucks, you will hear it centered. It’s a bit like 3-D images. You present a slightly different picture to each item. We do the same with audio, presenting a slightly different sound to be here which makes the announcement and sounds we play out sound like they are coming out from the direction where the business or restaurant or park bench or tree actually is. As you walk down the street, you would hear on either side of the street the shops and businesses that you would be walking past. If you approach an intersection, you hear the three-way intersection called out in the three directions, one to the left, one to the middle, and want to the right. It really gives you the sense of the layout of the space. These other kinds of experiences with the general awareness aspect of soundscape.
Been there comes the question, if soundscape is very much used for wayfinding and getting to places. Instead of turn by turn directions where you have to listen and follow instructions, the way that soundscape works is you would identify the place you are heading to, anyplace what we call a virtual audio beacon. Essentially it’s a sound that will sound like there isn’t audio beacon on that location, let’s say a café you are having to. As you walk down the street, you can always hear where that café is. You hear the sound emanating from that direction. You don’t have to walk straight towards it, but you always know right where it is. It might be coming from your left and you’re walking along the street, you know it’s on your left, so you know you need to find a place to turn left. Then you turn left, and all of a sudden it will sound like it is right in the middle and in front of you. You say okay, I’m walking straight towards it. It me hear a little bit to your right. That means you may need to cross the road. Basically it raises your awareness of everything around you and allows you to make your decision as you make your way there.
WADE WINGLER: I’m fascinated. As we talk about the hardware and software — it’s an iPhone, a set of headphones. Assuming that you are on some sort of data plan so that you have access to that, right?
AMOS MILLER: You need a data plan. Although once you’ve started a phone in the area, all the maps for the area are cached in your phone, so it’s not dependent on continued conductivity. It will work just fine if you lose connection.
WADE WINGLER: Good.
AMOS MILLER: We avoid the amount of back and forth between the phone and the network to manage — data consumption has not been an issue. The feedback we’ve gotten so far, it’s not been an issue.
WADE WINGLER: How does it do it? How is it discerning what is in the environment and building the auditory map to help you do the way fund? What’s going on in the background?
AMOS MILLER: The fundamental technology that we use is the 3-D audio technology that allows the ambient sound around you. At this stage, it’s pretty straightforward. We know where you are based on your location, based on your GPS location. We know and which way you are heading. There is a bit of nuance whether your phone is in your hand or pocket. It will work either way, but if it is not in your hand, it will work better when you are on the move because we can get the vector of your movement. So we know which way you are heading, we know where you are, and we pull the data from a mapping database called open street maps which is a crowd sourced mapping platform, to figure out what’s around you and do some understanding of what would be relevant for you in that moment in time at that space. And then generate the announcement appropriately in the space around you.
That’s technically how soundscape produces the experience. The 3-D audio really does give you a sense of where things are without you having to think about it. Instead of it saying to you at 3 o’clock or to your left or to the southwest, it’s just there. The benefit of that is that you don’t have to calculate that. That comes back to the point that if you are having a conversation with someone at the same time, the announcement is a very minimal announcement that you don’t have to process. You don’t have to make any effort to get that information while carrying on doing something else.
WADE WINGLER: As it is building these auditory experiences, is it relying solely on map data, or is it also looking at things that can visually process in the environment? Is it using the camera to determine things as well, or are we talking about map data entirely?
AMOS MILLER: This is map data entirely. You had spoken a few weeks ago to our friends at seeing AI, who are collaborating on this project with us.
WADE WINGLER: Yeah, I’m trying to connect the dots.
AMOS MILLER: They work really well together. One of the design ideas with soundscape is it is a background experience. Once you set it up, you don’t have to attend to it. You don’t have to have it as the primary app on your phone. It can just run in the background. This keeps going to the principle that this is an ambient experience. It’s an awareness experience. It’s not something you are doing. You will be doing something else. You can always pull up seeing AI and bring it up to detect a scene or look at a signpost. Seeing AI can completely integrate into the experience. You may also want to pull out one of your favorite step-by-step directions if you want to do so. Soundscape will continue to give that spatial awareness of your surroundings.
WADE WINGLER: That starting to paint the picture for me and understand how these tools might work together. It’s a fascinating grouping of technologies that Microsoft is putting together. As we think about soundscape, how does it learn? Can you train it on particular locations and might not know? Is it customizable?
AMOS MILLER: At this stage, open street maps is an open platform. Anyone can go at information, correct information, enrich the fidelity of information. It is community driven, and the wonderful thing about it is you may not want to do the map editing yourself, but you can highlight areas that don’t quite work for you. Often the community will come in and correct information and improve it. That’s the primary way that soundscape evolves. It takes a few days after you make an update on the map because it’s a generally used map by other apps as well. It takes a few days for it to get to the system and appear on soundscape.
There is a nuanced to that. Since we released soundscape in March, there’s been a lot of very interesting use cases and experiences. People have reported back mainly using the beacon for all sorts of things. This audio beacon I talked about earlier, where you can replace it anywhere you want to go. At the moment, it’s quite restricted to places that are already on the map. We’ve had a lot of interest in using the beacon and lots of other weird and wonderful ways. People are asking to depart from the app to place it anywhere. For example, you might be getting onto the beach and want to mark the point that you got onto the beach or enter the park or where your family sits for a picnic in the park. And how could you mark those points, those locations, so that you can place it beacon and come back to them? There are lots of good ideas coming back from the community that use this notion of this a very fluid awareness of where things are to almost personalize the space. These are areas we are looking into to build on to the basic map to make your experience of it more relevant to you.
WADE WINGLER: That makes sense. Practical questions. When, or is it available? And what’s the cost?
AMOS MILLER: The cost is free, as in the app is free. Anybody can use it as long as they have an iPhone. IPhones unfortunately are not free.
WADE WINGLER: True.
AMOS MILLER: At this stage, I would say that we only have a solution for the iPhone at the present. We are very well aware that there is a significant community out there who use androids. We don’t have soundscape for the androids at this stage. You will have the best experience using stereo headsets. Although I will say I had a wonderful experience only last week with a lady who was fully blind and scientifically deaf as well. She was not interested in using soundscape because it was something that she would not be able to interact with or deal with. But we noticed that she was actually able to hear the signal that the audio beacon was giving. The pitch worked for her. On the arm of her husband, they did a short route, about six or 700 feet to a place nearby. She was in the driving seat the whole way. She was saying, hey, it’s over there, let’s find a place to turn left. Now we are going straight there and they had a wonderful time. They were talking the way. They successfully reach the destination, and she burst into tears with emotion from having that experience. She never imagined she could do something like that and have that sense of independence when she was out and about.
I use the term “sense” because independence is not solo. She wasn’t doing all the guiding. She was on the arm with her husband, but she felt that she was in control and independent and that experience. That’s very meaningful to people.
WADE WINGLER: It’s been my experience that independence is relative. We are all trying to become more independent all the time. Every step in that direction is a good thing.
We are about out of time for the interview today, but if people wanted to learn more about soundscape or to follow the journey that you guys are on, what would you recommend in terms of contact information or a website?
AMOS MILLER: First of all, anybody that has an iPhone, please go and download it. Give it a try, tell us what you think. It’s soundscape on the iPhone. Just type soundscape, and it will be on the top of the list. You install it, there is a tutorial in the beginning to help you use it, to teach her how to use it. It’s a lot of fun. It takes a bit of learning. There is a feedback button that you can press and it gets to us, and we look at the feedback all the time and make adjustments. You can go to aka.ms/soundscape. That’s difficult to our website where there is lots of information, lots of FAQ’s where mobility instructors, people who are visually impaired, or really anyone can give it a try and learn how to use it.
WADE WINGLER: Amos Miller is the product lead for Microsoft soundscape and has been a great guest today. Thank you for being on the show.
AMOS MILLER: It’s a pleasure, Wade. Thanks for having me.
WADE WINGLER: Do you have a question about assistive technology? Do you have a suggestion for someone we should interview on Assistive Technology Update? Call our listener line at 317-721-7124, shoot us a note on Twitter @INDATAProject, or check us out on Facebook. Looking for a transcript or show notes from today’s show? Head on over to www.EasterSealstech.com. Assistive Technology Update is a proud member of the Accessibility Channel. Find other shows like this, plus much more, at AccessibilityChannel.com. The opinions expressed by our guests are their own and may or may not reflect those of the INDATA Project, Easter Seals Crossroads, or any of our supporting partners. That was your Assistance Technology Update. I’m Wade Wingler with the INDATA Project at Easter Seals Crossroads in Indiana.
***Transcript provided by TJ Cortopassi. For requests and inquiries, contact tjcortopassi@gmail.com***