ATU723 – IMPACTech Podcast Crossover with Dr. Mary Goldberg, Brandon Biggs and John McInerney Part 2

Hi, welcome back to a very special part two of our interview with the folks from Audiom, Brandon Biggs and John McInerney and our partners over at the Impact Tech Podcast, Dr. Mary Goldberg and Dr. Michele Zarella and I’m Josh Anderson and this is your Assistive Technology Update.

Hello and welcome to your Assistive Technology Update, a weekly dose of information that keeps you up to date on the latest developments in the field of technology designed to assist individuals with disabilities and special needs. I’m your host Josh Anderson with the INDATA Project at Easterseals Crossroads in beautiful Indianapolis, Indiana. Welcome to episode 723 of Assistive Technology Update, it is scheduled to be released on April 4th, 2025. Listeners, welcome back to part two of our special crossover episode with our friends from the Impact Tech Podcast which is part of the IMPACT Center at the University of Pittsburgh Department of Rehabilitation, Science and Technology along with our very special guests, Brandon Biggs and John McInerney. If you didn’t listen to our show last week, make sure that you definitely go back and give it a listen. Also, make sure that you listen to the Impact Tech Podcast, we’ve got a link down in the show notes where you can go and check that out as well.

We’re going to go ahead and pick up part of the interview from last week where Brandon is describing some of the things that Audiom is able to do and not able to do and then we will get on with the rest of the interview directly after that. Again, if you didn’t listen last week, make sure to go back and check that out, you can also check out the Impact Tech Podcast. But right now, let’s go ahead and get on with the show.

Can use your phone as well to navigate around. So, similarly to the computer, there’s a panel of buttons that show up on your touchscreen and you can double tap on the left button to move left one meter or double tap and hold to repeatedly move around one meter in one direction but it does not sync with your current location indoors. It will sync with your location outdoors but GPS doesn’t really work inside and GPS is the only real free method for us to use. If you do want to have Audiom sync with an indoor experience, you need to ask a place like GoodMaps to sync with the Audiom experience and that’s … They do have a GoodMaps version that you can use to navigate around the hotel with computer vision and so something like that could allow you to sync your location to Audiom but they haven’t integrated with us yet, they need more people to ask for it. So, if we want to be able to do that, that’s the next step there.

Thank you. So, I do appreciate the fact that using Audiom would take the place of a sighted guide to shepherd me around the hotel the first time so that I could orient myself, that would be extremely helpful in venues like that. Thank you.

Yeah, absolutely. It can either take a guide, the place of a sighted person orienting you to the hotel or a tactile map depending on your preference but it’s a digital tool that you don’t need to know how to read braille to use. You can use your braille display to read braille coming through your screen reader so, if you’re deaf-blind, it’s definitely possible to use it as well. But it does take a little bit of training to use because blind people just aren’t used to reading maps, I think they view less than one map a year on average and so it’s a thing you need to train your brain to be able to do like sighted people have.

Brandon, how has Audiom changed since its first iteration?

Oh, man.

Too big of a question?

Yeah. So, basically, the fundamental experience hasn’t changed, what’s changed is that we’ve allowed a lot of more advanced functionality and we’ve changed the user experience a lot. I guess the biggest way that it’s changed is we are fully WCAG AAA compliant and what that means is that you can access every aspect of Audiom through text, completely through text. Every feature on the map, every element is fully accessible through text, there’s no areas where sound is important or required or vision is required. We’ve also added on the visual experience for sighted users, we added on that touch screen experience, we’ve added on a way for people to follow routes on outdoor maps and campus maps, specific ones that’s been enabled on, and we’ve allowed you to be able to rotate your character around.

So, originally, we were using what’s called a grid mode experience where you were locked to the north orientation but now you can switch to what’s called first person mode and use your left and right arrow keys to spin around towards different objects and whatnot, just like you’re really moving around the space. And then you can use the up arrow to move forward and the back arrow to move backward rather than using the arrow keys to move around as if you’re feeling a tactile map. So, those are a few different changes that we’ve made since the beginning but there’s a much, much more, we even have a map drawing tool that you can create your own maps non-visually as well. We’ve also started doing other diagrams, not just maps, engineering diagrams or circuit boards, anatomical diagrams, blueprints, architectural diagrams. So, those are other things that we’ve started working on.

That’s awesome. So, you’re able to design your own house with Audiom now?

That’s the goal, yeah.

Nice, nice. I just know that’s where you started, saying that was the [inaudible 00:06:18] so I love that you’ve been able to build that all in there.

What has driven those changes that you’ve made to the product over the years? What’s motivated them?

Lots and lots and lots of user testing. We’ve ran around nine academic studies and co-designs to get what people think of particular interface elements on over 100 blind participants and over 30 sighted participants. And so, some of the co-designs that we’ve done, that’s where we started, we started with co-designs. And co-designs are really where you ask people to design an interface and give feedback on features and functionality they want while you’re still in early stages so they can give a lot of strong feedback on an interface that might not be perfect. And so, you’re giving early access to totally blind folks or low vision users to give their feedback and so that’s driven a lot of the interface conventions and elements of what we’ve added in and why we’ve prioritized specific things.

And then the reason why we started working with sighted participants is because we wanted to demonstrate equivalent purpose and that’s where we were starting to go more into the systematic approach of demonstrating equal access. And the reason why that’s important is because the Americans with Disabilities Act and other legislation around the world use the Web Content Accessibility Guidelines as a core framework and one of the very first items in there is that any non-text content needs to have a text alternative that serves the equivalent purpose but nobody’s defined that. And so, in our most recent study, we had sighted participants look at a visual map and look at an audio map and, each one, they answered 15 questions, they were separate maps and then we compared the accuracy of their responses and there were things like what state bordering California has the highest number of COVID cases.

And so, we were able to demonstrate there was no significant difference between audio and the visual map. And so, now, future work, we’re probably going to expand that out to more participants and validate it with more difficult questions, that kind of thing.

Brandon, I know this is going to be very hard to answer but do you have a favorite map you’ve been able to work on?

Well, I definitely say my favorite maps are the ones that have the most sounds. So, my two favorite ones right now are our demo map, of course, which is of the Magical Bridge playground in Palo Alto, California. And so, if you go onto audoim.net for personal use or xrnavigation.io/audiom-demo, they’re both going to show the same map and it’s going to give you a interactive tutorial where you can learn how to use Audiom and that is an actual real live location and it has lots and lots of sounds and it’s very detailed. And then we’ve just did a project with Vision Ireland to make a map of one of their centers and it has a microcosm of all public transit that you can think of. So, they’ve got an airplane, a part of an airplane inside that center and they’ve got a bus station in that center and they’ve got a subway in that center and a couple of buses and so that map has all kinds of really interesting sounds and you can go around it and explore all those different types of public transit on that map and it’s really, really cool. So, I would say those are my two favorite maps at the moment.

So, Brandon, I have a question. So, as an older adult, I have not been into gaming for blind individuals and, Audiom, a large part of that is based on the gaming principles that you described early on. So, what advice would you give to me as an older adult to train myself to become proficient in using the Audiom technology? Would you say, well, go get these games A, B, C, D and start using them or follow the tutorial that you have on your website? What would you recommend to someone like myself?

Everyone needs to do the tutorial even if you know how to use games because Audiom has very unique conventions that you don’t really find in many other tools and it combines a lot of the gaming conventions that people are used to. So, it will take you about two minutes to go through, two to three minutes to go through the interactive tutorial and I recommend that for everyone. Really, the best way to learn is to find a map that you’re motivated to view. So, whether that’s if you’re going to CSUN or another conference like the National Federation of the Blind Conference or something like that, you really need to have a map that you’re motivated to learn and that you’d sit down, take the time to really understand the conventions. And once you get faster at using Audiom, then you can start building that mental map, it becomes much, much quicker to virtually explore and understand what the map is showing.

I’ve been looking at Audiom maps for the last three or four years now and I can look at a map and really quickly understand what’s going on and learn where the booths are in the exhibit hall. I can find my booth without asking anybody, I can tell you that my booth is at the very top of the grand ballroom on the left, third booth down, that kind of thing and it’s real easy for me to get that information very quickly into Audiom and so you just need to practice with it. But it’s definitely an area where we need to do more work on learning how to best communicate these types of conventions for older adults because that is definitely a super, super important group that is not represented in our development team and so we need to spend a lot more time and effort making sure that it’s usable and understandable to folks who aren’t part of our development team.

Our development team is mostly young blind folks, totally blind folks so that’s a different group of people. So, we’ve done a lot of co-designs with older folks and it does work. It’s not pleasant at the beginning but it definitely is possible if you spend some time and do it and so it takes time.

So, John and Brandon, given this perceived … I guess I’m insinuating some potential generational divide both from your experiences with lived experience with visual impairment but also as innovators, what kind of challenge or opportunity does that present for the future as software specifically, I think, in this general application area continues to evolve?

Well, any new interface, I think, requires a lot of new training and getting used to. So, the smartphone required a lot of new conventions to get used to. Even if you’re learning how to read tactile maps and braille, that is a real big learning curve. I think Audiom is something that takes a lot less learning curve, you can do it on your own and you don’t need a teacher of the blind to teach you how to do that, whereas, you do need somebody to teach you to learn how to read tactile graphics. So, it’s one of these things where, I think, something like Audiom, with more co-design and iterating on the tutorial and techniques and just observing older individuals, we can make the interface much easier to use and give them the information quicker that they’re really interested in but it is, again, just going to be a lot more iteration.

I would agree, Brandon, and I’ve done a little work teaching older folks on voiceover on an iPhone, things like that and it can be pretty daunting if you’ve recently lost your sight through something like macular degeneration or some disease that impacts older adults. So, the guidance I would give people is you need to be patient and you need to be persistent and, as Brandon said, you need to find something or a map that you really want to learn. I’ve been thinking about examples, we just recently had a national election and, on all the news broadcasts, they show blue and red states and how many electoral votes each one has, that would be a great map for an older adult because you can get trends within the 48 contiguous states as to what’s blue, what’s red, for example. We know, and I hate to talk politics, that the two coasts tend to be blue but, other than that, I can’t tell you what the trend looks like through the US and something like that would be a great learning tool for an older adult, I think anyway.

We actually made a live election result map this last November and our problem is publicity because we’re a really small company, we don’t have as many people following us. And so, when we managed to get out that live election result map, it actually took us a while to figure out how to get access to the data and we managed to get it, right on election day, working. And we never really actually had it working until the election results started coming in and so we had to make a few little changes when that happened. But next year or next four years or whatever, whenever we end up doing another live election result map, it’ll definitely be up. But if you are interested in getting election results for your county for local elections, that’s also something that can be created in Audiom and we can work with your local precincts to make sure that that’s fully usable.

That is super cool. Brandon, John, this question is for both of you just because as a creator and, John, as somebody who’s been involved with a lot of bringing things to market, what’s some advice you could offer to someone out there who maybe has a great idea or sees a need that’s not met or just wants to make a piece of assistive technology or something that can help folks? What’s some advice you could give them from some lessons maybe you’ve learned along the way?

If you are disabled and you’re solving a problem for yourself similar to what we’re doing with Audiom, then you definitely need to run with that and build out the minimum prototype of that interface that works for you or even have a demo that works for you. And then this is where academia is really helpful, you can get credit building that interface for a master’s degree or a PhD degree in something like human-centered computing or rehabilitation research for more physical engineering products. And so, that’s how a lot of people in this space get started and get the credentials that they need to then go out and get grant funding and talk to investors. It’s really, really hard to get investors if you don’t have a working prototype in the assistive technology space and then it’s even more difficult to get customers and whatnot if you don’t have a product. So, if you’re disabled and solving a problem for yourself, I would definitely reach out to places like the IMPACT Center or different incubators that are out there for early startups and then maybe look into getting your degree in academia so you can get some of these grants that we’re talking about.

If you’re not disabled or you’re solving a problem for somebody that’s outside of your own personal use, you have to get lots of user feedback to make sure that you’re solving a problem. Because a lot of people will come into this space and say, “Hey, I’ve got a really great idea, let’s give blind people sight again using AI,” and there’s potential there but it’s not a silver bullet. AI is really great for … These meta glasses that everybody’s been talking about in the blindness community, I have them myself and they’re really great for reading large long signs in museums, they are horrible, horrible at identifying objects. So, even though the identifying objects is what they were demoed as, do not trust that AI because it is about 60% wrong. So, anyway, that’s been my experience.

I would agree with Brandon on the user input factor because that is essential because, even taking something simple like websites, a website can be WCAG 2.1 AA compliant and meet all the criteria but, from a usability standpoint with the screen reader, it can be terrible. So, there’s two aspects, there’s the compliance aspect and there’s the usability aspect and the usability aspect is key. Another example that I learned is that, if you are using sign language, ASL, and just showing the signs you know that you use with your fingers, it’s just part of the equation. Because I learned that a lot of people that are hearing impaired that use ASL, the gestures, the whole body gestures are just as important. A simple example that I learned is you can sign how and it can mean two things, how are you and, if you ask the question how from a visual standpoint, you raise your eyebrows. So, someone doing ASL from a user standpoint, that’s another example where you really need user input, as Brandon said, to solve whatever accessibility issue you’re trying to address.

There’s a famous example here at Georgia Tech where a professor was asked back in, I think it was the late ’90s or early two 2000s, he built a lot of technology around wearables and AI, image recognition, computer vision and he was asked, “Oh, could you work on this sign language interpreting thing?” He’s like, “Yeah, it’ll take me about a year, I think I’ll be able to get an automatic sign language recognition tool,” and it’s over 20 years later and they still haven’t cracked the code. So, it’s really hard and part of the reason I think why they had difficulties because they don’t really have deaf people on their team or native sign language speakers on their team. And so, if you’re building for a deaf audience, you need CODAs or deaf people on your team that can actually guide the research, I think that’s where you’re going to get the biggest bang for your buck and it’s really you need to have a disability-led, user-led team, that’s just critical.

Brandon, you mentioned this earlier but could you tell the listeners again how they can find out more about Audiom?

If you’re blind, you can go to audiom, audio M, dot net and use Audiom for your own personal use and we have a couple of maps on there and we have what we call our advocacy champion program where we tell you how you can work with any institution that has a map on their website and contact them and advocate for a map. And then if you’re an institution with a map, please send us an email or go to our website at xrnavigation.io and you can definitely learn a lot more information on the website about Audiom and the process. So, we really work with any type of map and our vision is that every map is usable to everyone and we’re building the platform to make that happen.

Awesome. Mary, did you have anything else?

Thank you. Yeah, no, we’ve really loved being part of this conversation and getting to know Brandon and see his product evolve over the years, it’s been a real pleasure at the IMPACT Center and work with John as well on our advisory board. We can be reached at idea2impact.org. Our, I’ll call it an incubator but we really have … We say, generally, it’s a three-stage training program that Brandon has participated in two times over, it does provide small awards to innovators like Brandon working in the AT space that are looking to do some further refinement on their product but really focus on what Brandon nailed on the head which is making sure that they really, really understand their problem and make sure that there’s a nice fit for the market as well. So, yeah, if you’re interested in learning more, please visit us there and, of course, check out IMPACT Deck as well. Thanks so much for the opportunity today.

Oh, thank you so much for the opportunity. Mary, John, Brandon, even silent partner Michele that’s on here that no one can hear but does all the work behind the scenes, thank you all so much for being part of this today, for letting me be a part of it, it’s been an absolute pleasure and, hopefully, we all get to do it again sometime.

Thank you for having me.

It’d be great, Josh. Thank you.

Do you have a question about assistive technology? Do you have a suggestion for someone we should interview on an Assistive Technology Update? If so, call our listener line at (317) 721-7124, send us an email at tech@eastersealscrossroads.org or shoot us a note on Twitter @INDATAproject. Our captions and transcripts for the show are sponsored by the Indiana Telephone Relay Access Corporation or InTRAC. You can find out more about InTRAC at relayindiana.com. A special thanks to Nicole Prieto for scheduling our amazing guests and making a mess of my schedule.

Today’s show was produced, edited, hosted and fraught over by yours truly. The opinions expressed by our guests are their own and may or may not reflect those of the INDATA Project, Easterseals Crossroads, our supporting partners or this host. This was your Assistive Technology Update and I’m Josh Anderson with the INDATA Project at Easterseals Crossroads in beautiful Indianapolis, Indiana. We look forward to seeing you next time, bye-bye.