ATU586 – DOT Inclusive Design Challenge Winners – 3rd Place – AVA app Project with Nicholas Giudice and Richard Corey of VEMI Lab at the University of Maine

——————————
If you have an AT question, leave us a voice mail at: 317-721-7124 or email tech@eastersealscrossroads.org
Check out our web site: http://www.eastersealstech.com
Follow us on Twitter: @INDATAproject
Like us on Facebook: www.Facebook.com/INDATA

—- Transcript Starts Here —–

Dr. Richard Corey:
Hi, I’m Richard Corey.

Dr. Nicholas Giudice:
And I’m Nicholas Giudice. We run the VEMI lab at the University of Maine and this is your Assistive Technology Update.

Josh Anderson:
Hello, and welcome to your Assistive Technology Update, a weekly dose of information that keeps you up to date on the latest developments in the field of technology designed to assist individuals with disabilities and special needs. I’m your host, Josh Anderson, with the End Data Project at Easterseals Crossroads in beautiful Indianapolis, Indiana. Welcome to episode 586 of Assistive Technology Update. It is scheduled to be released on August 19th, 2022. On today’s show, listeners, I am excited to start a whole series of shows celebrating the Department of Transportation’s Inclusive Design Challenge winners. If you can remember back over the course of the last few years, we’ve had many of the different semi-finalists on the show to talk about their programs and their projects that they had in for this Department of Transportation Inclusive Design Challenge.

Josh Anderson:
The idea behind the challenge was to ensure that autonomous vehicles are accessible for all who may want to use them, and there were a lot of great ideas there. Well, here at the end of July, they announced the three winners, and we’re very excited to have those three winners on the show these next three weeks. So we’ll start that all today, and we’re excited to welcome special guests Dr. Richard Corey and Dr. Nicholas Giudice. They’re from the VEMI lab at the University of Maine and theirs was the AVA App Project. According to the US Department of Transportation website, this team developed AVA, the Autonomous Vehicle Assistant, an innovative ride-hailing and localization smartphone application designed to seamlessly assist passengers with visual impairment and older adults during pre-journey planning, travel to pick-up locations, and vehicle entry.

Josh Anderson:
AVA uses innovative human-machine interfaces and technologies, such as GPS and computer vision, to help users find and ultimately arrive at the ADS-DV safely. The initial rollout of AVA’s training modules can be fully deployed and utilized via users’ existing smartphones, representing a cost-effective and timely solution to the problem of trust in automated vehicles. Today’s interview was previously recorded and originally aired in episode 513 on March 26th of 2021, but we had to go ahead and dust it off and play it again to congratulate them on their third-place win of the Department of Transportation’s Inclusive Design Challenge. Be sure to tune in next week, where our guest will be Dan Davies from AbleLink Tech, talking about their WayFinder ADS, which took second place in the Inclusive Design Challenge, and then we will wrap that up on September 2nd with Dr. Bradley Duerstock from Purdue University and their EASI RIDER Project, or the Efficient, Accessible, and Safe Interaction in a Real Integrated Design Environment for Riders with Disabilities Project, as they were the first-place winners of the Department of Transportation Inclusive Design Challenge.

Josh Anderson:
From all of us here at Assistive Technology Update and from Easterseals Crossroads Assistive Technology Program and the INDATA Project, we want to congratulate all of the winners and really all of the semi-finalists for being part of such a great program and ensuring that, as we move towards a world that includes autonomous vehicles, those vehicles are made accessible from the start. Transportation has always been a giant barrier for individuals with disabilities, pretty much all over America, and I’m sure probably all over the world. So the creation of the autonomous vehicle already opens up a whole new world of transportation, of getting around, of getting to jobs, to the store, to the doctor, and to other places that one might need to go. So making sure that, from the very beginning, accessibility is built-in, is just a wonderful and very important thing.

Josh Anderson:
So, shout out to all of the participants, huge kudos to the winners, and really, just congratulations to the Department of Transportation for being smart enough and forward-thinking enough to have such a challenge to make sure that autonomous vehicles are accessible to all riders. Again, don’t forget to listen for the next two weeks to hear the second-place and first-place winners, but for now, let’s go ahead and get on with the show. Folks on today’s show, I’m really excited to introduce Richard Corey and Nicholas Giudice. They’re working on the AVA App Project, which will help open the door, no pun intended, to autonomous travel for individuals with disabilities. Today, we’re going to learn a little bit more about them and about this exciting technology that they’re working on. Nicholas, Richard, welcome to the show.

Dr. Richard Corey:
Hey, thanks for having us.

Dr. Nicholas Giudice:
Thanks, Josh.

Josh Anderson:
Yeah, guys. I’m really excited to talk about this technology and how it can really help folks, but before we do that, could you tell our listeners a little bit about yourselves?

Dr. Richard Corey:
Yeah, sure. We’re from the VEMI lab at the University of Maine. This is a laboratory that’s focusing on human technology interfaces. We’ve been in existence for 12, almost 13 years now. Nick and I have been collaborative partners on this for that time, and we almost jokingly say that we have this academic marriage. But yeah, we’re studying the way in which humans interface with the technology in front of us and we do a lot of work into wayfinding, especially blind, visually impaired assisted technology. We’ve been doing a lot of work into virtual and augmented reality and how to use some of that technology to assist with even simple tasks, like riding a bike, for example, in the fog. We’ve also started to move into biotechnology as well.

Dr. Nicholas Giudice:
Just as a little background, so we bring an interesting background to our studies. My background is experimental psychology and Rick has a background in lots of things, including interactive design and collaboration, and we’re in a computer science department, broadly defined. So we’re really interested in saying, “Well, how can we take knowledge of human understanding and human interactions and lead to better technology and information access?” We come very strongly from the human side and also from our own first-person experiences. I’m congenitally blind and so I bring a lot of my own phenomenology in my own use of technology, frustrations with technology, frustrations with information access, into the types of things that we design to be multisensory. As Rick mentioned, bio-inspired designs. How can we make technology be more like how our brain works using all of our senses instead of just visual design?

Dr. Nicholas Giudice:
And so, that’s what gets us into this autonomous vehicle stuff we’ve been now working in for a couple years, and this newest project is one leg of that, where this is an amazing opportunity for blind folks, for older individuals, in terms of increasing transportation, accessibility, independence, dealing with a lot of the challenges that are out there and just getting from place to place. But these vehicles are not currently accessible and that’s not on the design table, in terms of the engineers are thinking about how to keep these things on the road, understandably, not for them not to crash, and thinking about the sensors and the control algorithms, but not the human factors, and certainly not the accessibility aspects of the human factors.

Dr. Nicholas Giudice:
And so, we’re really interested in this end-to-end process. It’s not just making the car work once you’re in it, but you have to figure out where it is, and how to find it, and how to get to it safely, which is what the AVA Project is about. So it’s a multistage process and we’re trying to think about it from beginning to end so this new technology can be truly accessible for underrepresented drivers and people that would hugely benefit from this technology.

Dr. Richard Corey:
I think we need to say “yet”.

Dr. Nicholas Giudice:
Yeah, exactly.

Dr. Richard Corey:
They’re not thinking about it yet, because we frequently have talks with the people in the autonomous vehicle industry and it’s on the agenda, but it’s not priority yet.

Dr. Nicholas Giudice:
Well, and the DOT posting this challenge, to Rick’s point, shows that they are beginning to think about it, at least at the policy administrative level.

Josh Anderson:
And you guys brought up a ton of great points there. I’m just going to dissect it a little bit. So just because you just said it, tell us a little bit about that US Department of Transportation Inclusive Design Challenge. We’ve had some other folks on that are involved in it, but tell us a little bit about that.

Dr. Nicholas Giudice:
It’s a really neat mechanism. They’ve set this up as groups that are competing for a prize, so it’s not set up as a traditional research grant, which is lots of formal aspects and everyone’s in competition with each other. But this is set up as here’s a real problem. This is a growing problem that’s being understood by DOT and other governmental agencies, and even beginning with the car manufacturers. How can we get people to think about this, to work together, to build synergies and leverage expertise to solve some of these problems? They used this idea of coopatition, so we’re cooperating. It is still a competition in some ways, because we are competing for these prizes, but there’s a real effort by the DOT to get the 10 groups that got this initial prize to work together. I say hats off to them. It’s a very innovative mechanism.

Josh Anderson:
Oh, yeah.

Dr. Richard Corey:
Yeah. You have to understand how unusual this is, for a government agency to be giving out a prize this way. Both Nick and I are absolutely amazed and thrilled. What a great way to get inclusive design out into the forefront, not just for autonomous vehicles, but in general, to say, “Hey, listen. We really need to start talking about this and we need to start investing in it.” And for them to give out these prizes and have this kind of collaborative structure that they have set up, and we’re talking to other people in the IDC prize phase two, I couldn’t be more impressed. Because sometimes, you don’t really get the governments working this efficiently, so it’s kind of nice.

Dr. Nicholas Giudice:
And it’s not just academics. That’s what happens a lot of times when you have an NSF or NIH prize or award, which we have other grants like that. But this is actually getting people out of academia, which is really important, because as many listeners may know, you get academics doing things, and you write a paper and do a conference presentation, then that’s it. What’s really driven Rick and I is that we want the work that we do at the VEMI lab to get out and actually make a difference, and to get out to people, and to be something that transcends any academic silos. I think this mechanism is really encouraging that.

Josh Anderson:
Oh, definitely. Sometimes just getting all the different groups in the same room is amazing, because you never know what everyone else is working on. So being able to feed off each other, really, that collaboration can get some great things done.

Dr. Richard Corey:
Yeah. It’s been really interesting, especially when we talk to people that are currently working in the industry. We’ve had talks with people in Washington who are thinking about policy on this. You realize that everybody’s got a different slant at it. It changes the way in which you start to think about, how is this going to work, especially, in particular, about just autonomous vehicles and how is that going to work down the road? What does it look like? Where are people going? Yeah, in thinking about inclusivity, it’s a much broader term right now, because there’s ideas out there of using fully autonomous vehicles to get your kids to school. It’s like, “Oh. Well, there’s a different design problem. How do you have a four-year-old travel in the car by itself?” I don’t know.

Josh Anderson:
So, guys, just getting back, tell me all about the AVA App Project. You mentioned it a little bit, but let’s dig a little bit deeper into it.

Dr. Richard Corey:
I’ll start. Nick will likely interrupt or jump in because that’s how we work. The AVA app is an application that, right now, is looking at helping people get from the location they’re at using a ride-sharing fully autonomous vehicle to the vehicle.

Dr. Nicholas Giudice:
Let’s just stop for a second. This is important, because almost all models, projections, are saying that these vehicles will not necessarily be personally owned, but will be following this ride-share, Uber, Lyft type model.

Dr. Richard Corey:
That’s a good point. Yeah. No matter what we’re looking at, there’s there seems to be a big, big push towards this ride-sharing model. Actually, I’m going to switch this around. Let’s look at the current models of ride-sharing out there with Uber and Lyft. You have a human driver that is going to pull up, and park, and they can communicate with a person and say, “Hey, I’m over here,” or when they’re parking their vehicle, they can be aware of obstacles, or trash cans, or I don’t know, snow banks. We’re up in Maine.

Dr. Nicholas Giudice:
Or you can text them. I’ll often text them and say, “Hey, I can’t see, so when you get here, you need to look for me and call out and say, “I’m over here.”

Dr. Richard Corey:
Yeah. Yeah.

Dr. Nicholas Giudice:
Showing me the picture of the car and the license plate isn’t going to do much.

Dr. Richard Corey:
Yeah, it’s not going to help. We are looking at this going, “Okay, that’s the current model, and we currently have this human-to-human interaction. How do we look at human-vehicle interaction, human-vehicle collaboration,” which is a term that Nick and I have coined. “How do we look at this, because we know that these AI, these robotic vehicles will be picking us up?” And in some cases, it will just be a single human getting in the back as a passenger. So now you go back to the same problem. Does the vehicle think in terms of don’t park to the door you’re going to enter next to the trash cans, or the snowbank, or next to an icy patch that you can see. And then the question is how do you know? Because there’s no talk back.

Dr. Richard Corey:
There’s nobody saying to you through a window, “Hey, I’m over here.” So, what do you do in that case? What sort of technology is going to be available in the vehicle, on your phone, on your person, to get there? That’s really the heart of what AVA is. How do you get from A to B safely just through technology? Because we’re at this junction in society where we’re overriding manual control to an AI. Nobody’s saying it’s a good or a bad thing, we’re just saying it’s happening. How can we use this technology to connect?

Dr. Nicholas Giudice:
And this is a general problem. Part of our vision of inclusivity is that it gets that inclusive design, universal design. How can something that might help a blind person or an older adult also be relevant for a 20-year-old that has normal vision? So, the issue for defining these things, a lot of people experience this, back when we went to concerts or what have you, you come out and you’re trying to find your Uber and there’s 25 cars. How do you know what one’s yours? This is going to be even worse when everything is an AI autonomous vehicle, there’s no human. They’re going to all often look similar within fleets because that’s easier for the technology and various sensors. A lot of people are going to experience this problem, but without the human in the loop, as Rick said, it’s particularly difficult for this last-meter problem, which may be a few meters. How do you actually get me when I’m on the sidewalk to get to the actual car when we don’t currently have a way of communicating with the human, me, and the AI driver?

Dr. Nicholas Giudice:
And so, what we’re working on within this app are ways that the car and the human can communicate more and techniques that we can use to help. Imagine when you’re using the app, you have your smartphone and it is connecting and talking with the desired vehicle. How can we provide other cues to guide you in that would be very similar to what you might otherwise use?

Josh Anderson:
So what are some of those cues that you’re working on?

Dr. Richard Corey:
The simplest one right now is knowing that all the technology’s pretty much gone electronic. The simplest one is honk the horn, just so you can at least get a spatialized angle for where the vehicle is. The other things we’re looking at is using computer vision on the phones, using the cameras on our phone to help identify vehicles. The door, we’re even down to the handle, if there’s going to be a handle, there seems to be some argument over that, but looking down to how to get there. But basically, Nick and I have been working on this for 12 years and the idea of how do we use current technology, simple consumer technology to get people there so you don’t have to buy these $100,000.00 whatever items to get through it? We just want to use the phones, current standard, maybe future standards, of where phones are. What’s in that technology that we can use and what wireless technology can we use to start to identify both the combination of GPS, and then maybe we can get something a little shorter? I don’t know, wifi signal, or maybe if they improve Bluetooth. Yeah.

Dr. Nicholas Giudice:
This is where working with the OEM car manufacturers will be helpful in this type of coopatition, in the IDC grant and other formal connections with car companies, saying, “We’re developing stuff. We need to be able to connect into your central system so we can talk more seamlessly with the vehicle.” I think that’s what will start happening. Part of it’s targeting, as Rick said, beeping the horn, but you’ll also be doing this through an app, so the thing will know Josh and it could say, “Josh over here.” We can actually have it speaking, and from a standpoint of hearing, that’s very useful because it provides a spatialized location because you’re hearing it coming from a specific place in space. Our auditory system’s very good at that. That’s the beauty of having a human driver doing that, because you know where that thing is.

Dr. Nicholas Giudice:
Rick mentioned using visual overlays and what we think of as compensatory augmentations or visual augmented reality. You could say, “Well, why is that useful? Especially if your focus is on blind people and older adults?” Most blindness is not total. The vast majority of blindness is people have residual vision that’s usable. A lot of us assistive technologies don’t account for that. People are very interested in developing stuff that’s purely non-visual. It’s essentially ignoring 90% to 95% of the distribution, the population of people that are legally blind or have visual impairments. So, in many cases, if we can make something that highlights the edge of something, imagine seeing a train platform that has these high contrast edges, that’s hugely important. It reduces falls and attracts people’s attention.

Dr. Nicholas Giudice:
So if we can use the phone and some computer vision to detect the outline of the car, highlight the handle to draw people’s attention, this is really useful for just making efficient navigation that limits the awkwardness. Even though you’re at the car, you don’t want to be feeling around trying to find the handle to get into it. We want to make these things as graceful and seamless as possible. And so, the more we can use these different senses and different technologies using the commercial hardware, as Rick said, we really feel this is hugely important.

Dr. Nicholas Giudice:
We can couple that with other non-visual senses too. We do a lot of work with vibrations and using vibration on the phone. We can also use that for a guidance mode, so when the phone’s pointing the right way, it’s going to be vibrating. We can use different auditory cues, so combining all this multisensory information into our interfaces, this is really our bread and butter and it’s something we’ve just always found so interesting. Why does so much technology rely on one sensory mode? That’s not at all how we actually experience the world.

Dr. Richard Corey:
It’s funny that Nick brought that up. One of the projects we did years ago was just that question alone. Why do we rely so heavily on vision? We made an entire VR simulation with the idea of it all goes black and then you have to navigate without vision in a virtual reality simulator, which seems like a complete counter to the idea of you have this head-mounted display that is visual, but then we just basically break it down to just audio cues only.

Josh Anderson:
I’d like to try that out. I bet that’d actually be kind of fun or at least give you a better idea of those folks that you’re working with and what they’re dealing with all the time.

Dr. Richard Corey:
Josh, you have no idea how badly I want you to try that out, and as soon as the darn pandemic is over, come and have some fun.

Josh Anderson:
We’ll do it. Yes, yes, yes. We definitely will. I really like the way that you guys are making not a one size fits all. Like you said, they’ve got all different kinds of cues and different kinds of ways. Because I know, even when they first started talking about autonomous vehicles, all of us were very excited because transportation’s just such a barrier for folks with all different kinds of disabilities, just getting around and being independent and having to rely on other folks. Depending on where you live, sometimes there’s transportation available, and it may or may not be good. And then if you live other places, there isn’t even anything that’s available, so you’re always relying on others. I love that you guys are thinking of this part that may have gotten completely overlooked.

Dr. Richard Corey:
That’s funny you bring that up, because I’m not 100% sure that it has been overlooked. I think that the autonomous vehicle companies, they’re very aware there’s an entire demographic that they haven’t even touched yet. It’s not just an accessibility issue. You’re looking at older adults. How many people have had their licenses taken away for one reason or another, or just age-related, or they shrunk down below the steering wheel or something like that?

Dr. Nicholas Giudice:
Well, that’s the aspect of inclusion, I think. Thinking about accessibility as being tied to disability, I think, is a limited way to think about it. I think we talk about it in terms of information access. Information can be lots of different types of information, lots of different scenarios for lots of different people. Maine has the oldest population in the country, so it’s a great place for us to be studying this. One of the things that we have thought about and don’t necessarily have data to support this, but this is something that we are beginning to look into, as Rick said, older adults, it’s a real challenge. Taking a license away is a big deal. If you look at the statistics for accidents and injury, it’s huge in people over 70, and particularly, 80. Older adults have really adopted ride-sharing in a lot of ways because it’s a way to get around, you don’t have to worry about driving, you could talk to someone.

Josh Anderson:
You get your freedom back.

Dr. Nicholas Giudice:
Yeah, exactly.

Dr. Richard Corey:
Yeah.

Dr. Nicholas Giudice:
And so, there may be a scenario here where older adults are actually some of the early adopters for autonomous vehicles because they see it as something that can really solve a problem. Normally, when you think about technology and high-tech technology, it’s the early adopters of the 20-somethings and teens. I think this has a potential to have a very large demographic of our population. One of the fastest growing demographics is aging, corresponding with vision loss. I think that these groups really get the potential here, if we can make it work.

Josh Anderson:
Well, guys, if our listeners would want to find out more about you, about the AVA Project, or about any of that stuff, what’s the best way for them to do that?

Dr. Richard Corey:
The VEMI lab at the University of Maine. Pretty simple. Look it up at the University of Maine. Either do a search online for the VEMI lab, V-E-M-I, or we’re on Twitter, we’re on Instagram. I don’t know. We’re all over the place.

Josh Anderson:
You want to give the actual website?

Dr. Richard Corey:
Oh, what is it now? It’s vemilab.org?

Dr. Nicholas Giudice:
Oh, they can go there. Yeah, vemilab.org, or you can go through the University of Maine.

Dr. Richard Corey:
Yeah, the University of Maine.

Dr. Nicholas Giudice:
Umaine.edu/vemi. There’s also a specific website for Autonomous Vehicle Research Group which if you search on, you’ll find work on this project and other stuff that we’re doing in the autonomous realm.

Dr. Richard Corey:
Which is under the VEMI lab.

Dr. Nicholas Giudice:
If there are people here that are interested or part of the demographics that we’re interested in, reach out, because we’re going to be doing some surveys. We’re going to be doing some stuff where we really want to get a broader range of people. And so, the more people that we can get, you have a big base of listeners, a lot of them are in the demographics, particularly that we’re interested in. If we can get them to be part of the grassroots to make this work, that’d be great.

Josh Anderson:
All right. Excellent guys. We’ll put links to those over in the show notes so that folks can easily find those. Well, Richard, Nicholas, thank you so much for coming on the show today, telling us all about the AVA Project, but also just making sure that we’re thinking about these things as we make new technologies and make sure that we make everything inclusive for all. I’m glad you guys are involved in that US Department of Transportation Inclusive Design Challenge, along with some other folks we’ve talked to here on the show. I think some really great things are going to come out of that. I really do like that inclusive design is being included in the early process, as opposed to something that folks are just trying to add on later.

Dr. Richard Corey:
Fully agree with that.

Dr. Nicholas Giudice:
Thanks, Josh.

Dr. Richard Corey:
Thank you.

Dr. Nicholas Giudice:
It was fun to be here.

Josh Anderson:
Do you have a question about assistive technology? Do you have a suggestion for someone we should interview on an Assistive Technology Update? If so, call our listener line at 317-721-7124. Send us an email at tech@eastersealscrossroads.org or shoot us a note on Twitter, @indataproject. Our captions and transcripts for the show are sponsored by the Indiana Telephone Relay Access Corporation or INTRAC. You can find out more about INTRAC at relayindiana.com. A special thanks to Nicole Prieto for scheduling our amazing guests and making a mess of my schedule. Today’s show was produced, edited, hosted, and fraught over by yours truly. The opinions expressed by our guests are their own and may or may not reflect those of the INDATA Project, Easterseals Crossroads, our supporting partners, or this host. This was your Assistive Technology Update and I’m Josh Anderson with the INDATA Project at Easterseals Crossroads in beautiful Indianapolis, Indiana. We look forward to seeing you next time. Bye-bye.