ATU708 – AI and the Future of Accessibility with Dr. Gregg Vanderheiden

Hi, this is Gregg Vanderheiden. And this is your Assistive Technology update.

Hello and welcome to your Assistive Technology Update, a weekly dose of information that keeps you up to date on the latest developments in the field of technology designed to assist individuals with disabilities and special needs. I’m your host, Josh Anderson with the INDATA Project at Easterseals Crossroads in beautiful Indianapolis, Indiana. Welcome to episode 708 of Assistive Technology Update. It is scheduled to be released on December 20th, 2024. And today we welcome back Dr. Greg Vanderheiden to the show. Dr. Greg Vanderheiden. Thank you for coming back and welcome back to the show.

Thank you very much.

Well, we did your introductions last time, so we will just go ahead and leave that kind of back there. I want to get into this exciting topic. So I guess let’s start off with maybe artificial intelligence. What is it and what exactly can it do right now?

Well, artificial intelligence is funny because it’s kind of a misnomer. We use it for any time that something other than a human being is making decisions. I think the term was actually coined in the ’50s or even earlier. And we’ve had AI for a really long time. But recently we have this generative ai, which basically just took everything on the internet, if you would, massive amounts of input and by just using probability. In other words, if you had these three words what’s the most probable next word to follow it kind of thing. They found it could answer questions and it could do all sorts of things. And it looked like it could understand language and things. And maybe it does, but maybe in the same way a child learns in that if you’ve ever had a child and you said, “Well, let’s go to the store.” And he’ll go, “No daddy. That’s wrong.” And you’d say, “What’s wrong?” And he has no idea what the rules of grammar are, but if you use the wrong grammar. Then he will correct you or she will correct you.

And this is because they learn by example, if you would. And that’s kind of what we’re seeing with the generative AI and it’s been astounding. Now they’re going through and they’re trying to put all sorts of rules and stuff on it to try and keep it from making… It looks like it’s understanding when it’s really just kind of parroting back. So if you put a lot of garbage into it, it will come out with stuff that doesn’t make sense. If you ask it questions about stuff that it doesn’t really have a lot of content that relates to, it will make it up. It kind of feels like your Uncle Joe or whatever that is really, really knowledgeable, but when he doesn’t know something he’s just as sure about it as when he does know it. A.

Nd so you end up with you ask him a question and he’ll give you an answer and you aren’t really sure if she should trust the answer or not. And if you ask him esoteric questions he will take a shot at it, but he will speak with the same confidence as the stuff that he actually really knows. And you can’t tell by staring at him which one it is. And so a lot of people worry about AI, talk about hallucinations and stuff. But there’s a lot of stuff that it already does really well today. And there’s more that it’ll be doing tomorrow. So the dangers are not so much in it, but how we use it. Their end is the rub. But around accessibility, there’s a lot of things that it can be used today and in the near future and in the far future that are going to really revolutionize accessibility.

Where are some places where you see AI being used in the accessibility space now?

Well, first of all, the one thing about it is because it’s based on probability and what it sees, what it sees a lot of, when you ask it what’s most probable, if you say, “How tall is a man?” It’s going to tell you. It’s going to give you the most probable answer. But you know that there are men that are very short and men that are very tall. And if you say, “Are men shorter than women?” If you think of the mean, that would be true. But if you said, “Is a man shorter than a woman?” It could also say, “No, the woman will be shorter.” Well, you have no idea because you said, “A man,” you didn’t say, “The average of men.” But it’s going to give you the most probable answer. And most probably any arbitrary woman is going to be shorter than any arbitrary man. If you didn’t realize that was the question you asked, you got the answer back and you just went off. So that’s a problem you have, but you can use it to your advantage.

Now, one of the things they said, “Well, we just have to have more information about people at the extremes.” And that’s not going to solve the problem. Because you can add all the data you want about the people at the extremes, but if there are more people in the middle more probable answers still going to be in the middle. In other words, it’ll tend to average things back. So we have to, when we’re looking at accessibility, try to not use it in ways where you’re saying, “What’s most probably going to be true for a person?” To predict what’s going to be true for a person with a disability. But you can use it a different way. For example, the normal way a person communicates is somewhat fluid and somewhat verbose. So for example, if I wanted a screwdriver I would say, “Hey, could you hand me a screwdriver?” However, if you’re underwater where you can’t communicate so quickly and I gave you a grease pencil and a slate and you needed a screwdriver, what would you write on that slate? Are you going to write screwdriver and show it to the guy?

You speak Telegraphically. Why do we call it telegraphically? Because you used to pay by the word. So when did you speak? You spoke in the least number of words you possibly could. However, so I’m a communication aid and I have to point out the words as I’m talking. And I’m just going to be slow at pointing out the words. And so how should I talk? If I am normal, if you would use the word, or typical or standard or anything that was logical you would speak telegraphically. However, a person’s intelligence, if I talk to you in telegraphic sentences you would lower your estimate of my intelligence immediately. And so what happens is if you are a communicator you have a choice. You can either speak slowly and spell everything out in which case people are not going to be around, because it’s going to leave because it takes too long. They’re not going to listen to you, blah, blah, blah. Or you can speak in short in which case they will think that you are not very bright. And so you got no way to win.

So what if we take AI and say, “Whatever I point to I’m going to take those words and I’m going to feed them into an AI.” But in front of it I’m going to say, “How would a person of college age say this politely?” And then I say, “Want food.” And it says, “Hi, could I please have some food?” And so now I’m using AI’s ability to say, “What is the most probable way that a person would say something like this to take my telegraphic and make it into acceptable language to be used in polite society?” I could even say, “Want food frustrated.” And it would say, “Hey, I’m really hungry. When’s the food going to be ready?” Or you could say, “Food angry.” And they could say it in angry. So they can with just a couple words shape stuff.

And you could even say to the AI, “By the way, give me three versions of everything that I ask for. So you give me three versions of a polite blah, blah, blah.” And then what I can do is it’ll give me three different versions. And I could pick the one that most matches what I really want it to say or the way I want to say it. So this is one of the ways that we could use AI today. We can use it to look at content and say, “What’s wrong with this?” You can use it to scan a whole bunch of things. We have tools for automatically looking at websites to see what things are gone. But it’s not very good at picking out things that are not black and white, like the contrast or whether something is there or not there. But if you say there’s alt text there, is it any good? And it can’t judge that.

But with AI now you can with a lot of coaching. These are some of the ways that we can use it today. We can use it in making things accessible in the early stages by having it look at things and pick out problems that you can then repair. You can have it suggest repairs. Now, it’s not necessarily going to be better than the best accessibility expert, but all the people in all the companies who are designing all the products who’ve gotten some training are not the best. And there are not very many of the best people in accessibility. So it can be a real boon. So that’s one of the things.

The thing that I think that can happen next that I would like to see is it can start to relieve some of these things. So let’s say web pages. Today if you have headers you have to mark them up as headers. Headers are obvious. If they weren’t visibly obvious then people who don’t use assistive technologies wouldn’t know that they were headers. So we can have an AI today trivially pick out what the headers in a document are if the 50th percentile person could figure it out. So if we make pages that the person midway between the brightest and least can figure it out. The AI is going to be better at it than that. So why don’t we just build into the browsers that when they look at a page they just take the visual presentation. And if they aren’t marked up as headers they treat them as headers in their document object model. Which is what at uses for accessibility.

Why can’t you have it so that when you have a picture of text? All of our devices now will give us a text version of software, so why can’t we make it do that? So there’s a whole bunch of places where we could have the browser do things that we now require authors to do. And then the authors wouldn’t have to worry about doing them or not doing them or using a technology that’s hard to do them. Because the browser could just look at it and figure it out for itself and create the document object model which is the digital version that all the at uses to figure out what’s on the page. It can also make it that you could have a plugin to the browser that just says, “Make all the headings bolder,” or, “Make the links stand out,” or, “Make this or that.” And all this stuff that is hard for some people to see could be much easier for them to see.

And again, so much that we have authors doing might be able to be done by the browser itself relieving gazillions of hours of work on the part of authors. So that’s number two.

Nice.

And I think that’s something that’s very, very close to what we could be doing.

Where do you see artificial intelligence going?

I think if you take that same thing that we could actually make it so that there’s very little that companies need to do. And the products would be more accessible to more people and all products would be accessible. So let me unwrap that. Today we have the latest set of guidelines coming out. I have 292 rules that apply to all different kinds of things. Webpages, software documents, et cetera. But with all those rules, if you actually go and look at webpages there’s a very small percentage that meet even half of them. And very few that meet all of them. And if you read the guidelines they say right at the top, “If you do all of these things your product will be the minimum accessibility, but it still won’t be accessible to a lot of people.” People with cognitive language and learning disabilities, for example. People who are deaf/blind. People who are blind but also have a cognitive disability. People who have cerebral palsy and who are blind. I mean all of these people start to have problems because there aren’t tools to make things accessible.

What if we could create something called an info bot. I’ll call it an info bot. And it’s free, it’s open source. We create one monster. It’s in the web. It sits in the cloud. And we teach it by examples and stuff to the point where it can understand any interface that the 50th percentile human can. In other words, if you can make a product that half of the population can figure out how to use, and presumably most products you want to do that, this info bot could figure it out. Not only that, but you’re going to show it to the info bot before you release it anyway. And if it has any problems, you could teach it whatever it didn’t know. Then it can be aimed at any interface. It could be the TV set with all these streaming programs that we can’t figure out how to use anymore. It could be the thermostat, it can be the microwave, it can be the cook pot that you just bought. The smart pot that you got bought that you can’t figure out how to use. Because you have low vision and they decided to print dark red start button is dark red on black. And you can’t even see it. But you just aim it at anything and it can understand it.

All right, now on the back end of that we are going to have what I call individual user interface generators. And what does it do? It generates an interface for that individual. So if this individual is bright blind and does hierarchical interfaces it’s going to give them the interface that would be on that product when it was manufactured if everybody were blind. So it’s not like a screen reader, which is an audio interface to a visual interface to the functionality. It would be if nobody could see what would a computer look like? What would a thermostat look like? What would all these things look like if the person were bright and could handle hierarchical? That’s the interfaces that this person gets.

Now their uncle or aunt who is older and can’t remember things very well and is very confused by technology and also can’t see now would get a completely different interface. It would be one that was tuned to them. But they would get that interface for everything they had that they ran into. So when they aimed at the thermostat, they would get an interface that they could understand for using the thermostat. When they aimed it at the microwave they would get one that they could understand. When they aimed it at the cook pot they would get one that they could understand.

Now whereas the other person might get a nice big hierarchical one with lots of controls, this one might get one where it says, “What do you want to do?” And then okay. And so, “How long do you want to cook it? Or do you want it done rare?” Just operating off of the stuff on the front of the buttons. But instead of giving you all the buttons at once it would walk you through the process and it would punch all the buttons. Of course it had seen the instruction manual, because when the manufacturer made it they show the info bot the manual so it would know the manual for that product too. In this case, now we have not just some people, but as long as we can create an individual user interface generator for that type of an individual or for that specific individual they will have an interface to everything, any product, whether or not the manufacturer didn’t have to do anything except make it so that it was understandable by half the population. And presumably they will do that. So I can now aim this thing.

A side benefit of this is that, have you ever gone to a hotel and there’s a thermostat on the wall and you went over to it? And then you stared at it and said, “Now how on earth do I get this thing to work?” And again, you are at the top of the digital affinity pile, not the bottom end. There are people I know that just never change the temperature even if it’s the wrong one, because they have absolutely no idea how to do it. And so they just live with it however it is. You’d aim it at that thermostat and you’d know it. When you went home and aimed it you’d get the same one when you went to the hotel, you’d get the… Since the thermostats all do the same thing you’d get the same interface.

Now if the one at home’s got more features, you’d get the same interface, but there would be some more options. But when you went to the hotel the same way you operate the one at home would be the same way here. Just you wouldn’t have the options you don’t have. Microwaves, I once had somebody who was headed up a foundation and he said their microwave got old. They gave it to Goodwill. They went down to the store and they tried to buy one. And they looked and they looked and they looked and they couldn’t figure out how to use any of the ones at the store. They went back to Goodwill, bought their old microwave back and took it home, because they knew how to use that one. And this is somebody who had three foundations that he had set up that he was funding. So this is a CEO kind of guy who’s still at the top of his… But he didn’t have good digital affinity. He just was very bright and accomplished. He wouldn’t have that problem because he’d get a new one. He could look at it and he would get the same interface he did before.

My favorite is that the streaming services keep changing and they change arbitrarily. So that not only when I go from one to the next can I figure out where to find things, because each one does it differently. But even the one that I knew and I knew where things were, recently they changed the interface and suddenly nothing goes where it was and I couldn’t find it. And I used to know where to find, “Continue watching.” And I could not find it to save my soul. So we ended up just sort of picking whatever we could see, which may be the purpose of it. Because these things are not optimized for use. They’re optimized for revenue. So if you confuse people enough that they always just pick whatever’s at the top, ta-da, you win.

But at any rate, I would love it that when I aim this thing I would get an interface I could understand. And when I aimed at each different streaming interface it would give me the same organization. And if they changed it, they can change it all they want. There’s something when you age that’s called crystallized intelligence. And as we age, our intelligence will crystallize. When we’re really young little kids, they just absorb. They can learn anything. They learn language for God’s sakes. Just as we get older we may have a lot of intelligence and wisdom, but we lose our flexibility. And I’ve had people who I’ve worked with who are very old, who were cooks. They were this, they were that. And you could talk to them about what their expertise was and they were sharp as a tack. But if you tried to teach them something new they could not learn it, because their intelligence had crystallized. That is they had it but it couldn’t learn.

Well think of people like this who are now trying to use these interfaces we’re giving them, like the television stuff. And then we change it and they can’t do it. We have people that we’re working with that one of them was a computer scientist, one of them wrote compilers, one of them was a physics professor. And none of the three of them could figure out how to get onto Zoom. Every week they have a Zoom call. They used to have a family Zoom calls. And somebody in their family has to call them up 10 minutes to 15 minutes before the call and on the phone slowly walk them through getting onto the Zoom call. And they have to do that each week. And they’ve had to do it each week for two years. These are very, very bright technical people, but they can’t remember, they can’t learn. And we keep changing the interfaces on things.

And so this is something that I think that we can do is that AI will give us the ability to be able to give people interfaces that are stable and that are understandable. And now we can have an interface for cognitive language and learning. One of the problems we have in creating interfaces for them is they’re so different that what’s right for one isn’t necessarily right for another. And for some people the level of cognition actually changes from day to day or even in the course of a day. So if you have an interface that is for each individual instead of trying to make one that works for all of them because you can’t, now we can for the first time have interfaces that really work for populations that we don’t reach out today. And it’s an interface to all of the different products that they encounter. So that’s I think some of the promise for AI in the future.

There’s nothing that prevents them, for example, saying, “Well, I’d like to try a new interface.” And if they can use it, then they can switch. It’s their individual thing. But if it doesn’t work. And we’re going to be having interfaces in the future where sometimes you’ll be able to just talk to it. So you just talk to it and it will look at the interface and it’ll do what it needs to do. Some cases though, it won’t always be verbal. It may be verbal one direction and not the other way. For example, I may talk to my television to get to the show I want, but I don’t want it to talk back and tell me what’s showing. I mean, I want to see the show. So it’s going to be an interesting combination of command.

But one thing we found though is that everybody thinks, “Oh, we’ll just be able to talk to them.” But people don’t know what to ask. And this is a problem. And it’s good if you already know in advance what you want and you can say it and you know how to say it in the right way. Some people can search, some people can’t. But if we can figure out how to make it so that they can be able to see it and talk to it, figure it out. But they also, we want to have the ability to browse. And so that’s not just going to be verbal, “This is what I want. Do it.” We have to have browsing. And so there’s going to be some real interesting challenges here.

Definitely some challenges, but also some really just amazing things. And I love seeing all the doors that artificial intelligence has already begun to open up, not just with access, but being built into some tools that maybe are assistive technology or can be used as assistive technology even if they’re not made for that. I really see some of the costs coming down already for certain things. And I really hope that that continues as these new tools are developed and come out in the future too.

Be careful of the cost. Right now, all of these engines and AIs are all losing money hand over fist as they try and jockey for dominance and get you all dependent upon them. It’s like the heroine’s always free it for the first dose. But there is a reckoning coming and it won’t always be free. And when it isn’t free there’s always going to be distributive justice. Those people who have the money to afford… If there’s a free version and a paid version the paid version’s going to be the better one. And the people who have it are going to have the advantage. And so this is, I’m a little worried about.

One thing that’s really critical about all this and that is that even though in the future it may be good or better than what we’re doing today, it’s critical that we don’t stop doing what we’re doing today for accessibility. Because we might have something better in the future. We need to continue to do everything we’re doing today to make things accessible until such time as these new things are fully up and proven to be working and acceptable by the people who have disabilities. We did a site with Vint Cerf on the future, and it’s futureofinterface.org.

Once again, thank you so much for coming on the show and just telling us just all this great stuff about artificial intelligence.

Thank you.

Do you have a question about assistive technology? Do you have a suggestion for someone we should interview on assistive technology update? If so, call our listener line at 317 721 7124. Send us an email at tech@eastersealscrossroads.org. Or shoot us a note on Twitter @INDATAproject. Our captions and transcripts for the show are sponsored by the Indiana Telephone Relay Access Corporation or INTRAC. You can find out more about INTRAC at relayindiana.com. A special thanks to Nicole Prieto for scheduling our amazing guests and making a mess of my schedule. Today’s show is produced, edited, hosted, and fraught over by yours truly. The opinions expressed by our guests are their own and may or may not reflect those of the INDATA Project, Easterseals Crossroads, or supporting partners, or this host. This was your assistive technology update.

I’m Josh Anderson with the INDATA Project at Easterseals Crossroads in beautiful Indianapolis, Indiana. We look forward to seeing you next time. Bye-bye.