ATU692 – PAVE with Prof. Dr. Alireza Darvishy and Felix Schmitt-Koopman

Hi. I am Alireza Darvishy, professor in the area of digital accessibility at Zurich University of Applied Sciences in Switzerland. This is your Assisted Technology Update.

Hello. Welcome to your Assistive Technology Update, a weekly dose of information that keeps you up-to-date on the latest developments in the field of technology designed to assist individuals with disabilities and special needs. I’m your host, Josh Anderson, with the INDATA Project at Easterseals Crossroads in beautiful Indianapolis, Indiana. Welcome to episode 692 of Assistive Technology Update. It is scheduled to be released on August 30th, 2024. Today we’re super excited to have folks from the ICT Accessibility Lab at Zurich University of Applied Science in Switzerland here. They’re going to tell us about a tool they made called PAVE, and how it can help individuals make their PDFs more accessible. As always, we thank you so much for listening. Now let’s go ahead and get on with the show.

Listeners, all of the assistive technology and other tools we talk about on this show don’t do a whole lot of good if the items that they’re looking to access aren’t accessible to them. Today we’re super excited to learn all about a super cool tool called PAVE, and how it’s able to make PDFs more accessible. We cannot wait to learn all about it.

Alireza, Felix, welcome to the show.

Hi.

Thank you.

Thank you.

Yeah. I am really excited to talk about this really great tool. Before we get into talking about PAVE and all the great ways that it can help, could you tell our listeners a little bit about yourselves?

Yes. Maybe I can start. I am Alireza Darvishy, a professor in the area of digital accessibility at Zurich University of Applied Sciences in Switzerland. We carry out research and development, and teaching in the area of digital accessibility. This lab, my ICT Accessibility Lab, was founded for about 20 years ago. I was the founder of this lab. We are active in teaching research. Our research and development is mainly in the area of education, accessible education and accessible mobility. We have a number of PhD students doing their PhD and research projects on different topics in the area of accessible education and also accessible mobility.

Awesome. And Felix, what about you?

Yeah. Hi, I’m Felix Schmitt. I’m a third-year PhD student at the University of Zurich. My PhD project is exactly about PDF accessibility, and especially the mathematical formulas. How can we make mathematical formulas accessible?

Maybe I can add something. By myself, I have a vestibular visual impairment. I studied computer science. I was one of the two PhD students in Switzerland having a visual impairment and pursuing a computer science degree. My motivation at that time was also to establish a center, and do research and teaching in the area of digital accessibility.

That is awesome. I love how you have the personal story connecting you to it. I know all the listeners of this show know the importance of digital accessibility, so we’re so glad that you started it there.

I know that one of the tools that has come out of your accessible lab and this whole center is a tool called PAVE. What exactly is it?

PAVE, maybe I start a little bit and I will give over to Felix to explain more. PAVE stands for PDF accessibility validation engine. It was created originally 10 years ago, in 2014. Two of my students created this tool because, as a visually impaired researcher, I had to read a lot of PDF documents, mainly research papers. Most of them were not accessible. As probably you know, to create accessible PDF documents is very difficult, time-consuming, expensive. At that time, the idea was to create something which is web-based that everybody can use it, and should be intuitive and simple.

The students, 10 years ago, created this tool, the original version of the tool. You can upload your PDF document, and the software analyzes your PDF document, and corrects some accessibility issues. The rest will be corrected through collaboration between user and the software. That was the original idea of PAVE. This project won an accessibility award in Paris in 2014. It was a competition called Quote For A Cause, and it got the accessibility award. That was the original idea of PAVE.

Through these last 10 years, this tool had been used very often because it was free. It has, of course, some limitation. Maybe I can ask Felix to maybe describe more, the limitation and the new approach that we have persuaded in the last two years, three years.

Yeah. Perhaps, what I can add is the old tool really follows it is creating a report, an accessibility report. It is analyzing the PDF. Then some things, it can fix automatically. The other things, the user has to fix. There were some limitation, especially in the back end with the sizes of the PDFs, with special kind of PDF objects, and so on.

What we are doing now is to create a new version out of it that follows a little bit different approach. We no longer have this report that the user has to go through the issues and has to fix them. Instead, it has to annotate and see more graphically, so it follows eight steps through a process. It’s guided through this process to fix the stuff, and is not directly working with accessibility details in PDFs, and so on. Less expert knowledge is required, that’s the idea to make it more intuitive.

One point is also important. I’m sure, Felix, you will more describe and elaborate more. This, we utilize AI, specifically deep learning for identify structure and different elements in the PDF document.

Yeah. That’s also one reason why we split it up and changed the complete process, how it is happening, so we can easily integrate the models. Each model can make the prediction, and the user can correct these predictions. We not have the issue that we have errors that accumulate, if you have multiple models in a row.

We developed two models yet. One for page object detection. That means for different regions in a different document, where is the header, where is the table, and so on. The other one is the recognition of mathematical formulas. The translation of the image of a formula into a text.

Maybe I can add, Josh, that many scientific papers have maybe mathematical formulas or scientific graphs. These are really not accessible for a visually impaired researcher or student. With this project, we tried to really identify mathematical formulas and provide an alternative for visually impaired or blind users.

Oh, nice. Very nice. I guess, from a user standpoint, it can do some amazing things. I love that the AI is able to pull out the information and interpret it in that kind of way.

From a user experience, what does it look like if I’m the person that opens up the tool and starts using it? What’s my experience look like?

Maybe I can tell you this tool is really not out for end users, visually impaired users. This tool is used for people who create accessible PDF for visually impaired people.

Okay.

I think maybe you can explain more, Felix.

Yes.

You carried out the user study.

Yeah. First of all, as Alireza said, making a PDF accessible should be done by the author or somebody else. It cannot be done by the visually impaired himself. This is an important point.

There, it chose … Yeah. It’s always very time-consuming, it needs some knowledge, and so on. Especially those who needs the awareness. That is, I would say, the most critical part why most authors are not making their PDFs accessible, because they are not aware of it. They have, most of the time, also not the knowledge.

If you’re using now, the new version of PAVE, the second version, it’ll guide you through eight steps. In the first step, you have to define the regions in a document. Where is the title, where are paragraphs, where are figures, and so on. This is supported by an AI model, so you can let it detect, and then you only have to make some corrections to it, if there’s some errors in it. Then the second step is defining the logical reading order. In which order shall it be read by a screen reader? Then the third step is the heading levels, so they can define the complete heading structure of a document.

Then step four are the tables, where you have to define each table cell in a document. Because else, the screen reader is not possible to read tables correctly. The same for the list, that’s step five. Step six is then you have to create an alternative text for figures. In step seven, you have to do the same but for mathematical formulas. Where also, our novel math editor plays in. If the AI model is to recognize it, so it supports yours, you not really need to know how you should create valid alternative text for mathematical formulas. You only have to compare it and make some correction. It works everything visually.

Nice.

Then it creates automatically, out of it, the alternative text for it. In the last step, you can check the results, and also update some meta information. Then you’re already through the process.

Nice. I love the way that it walks the individual through. Because as you said, a lot of folks don’t even think about that accessibility. Even if they do, they may not have any clue how to make them accessible. I love that it’s able to walk them through all those different steps to make sure that all the different pieces are actually accessible to someone using assistive technology.

One important item is here that, actually, that the author of the document shouldn’t really need a lot of time to create an accessible PDF document. It was also one of the main concern in the previous version, or in other tools. We tried to make the tool in a way that the author of the document doesn’t need really a long time to correct the accessible issues.

I think maybe, Felix, you can tell Josh about the user study you carried out?

Yeah.

Yeah. We also carried out a user study with 19 participants. Some had some experience with PDF accessibility authors, or had not. What it showed, especially this tech quality. If you analyze the text at the end, so the accessibility of the PDF, it shows that it’s really much better as if they are doing with Adobe. It shows also that the complete process, because it’s so guided, they have to go through these steps, and so on. It showed that it’s super helpful for novice, but also for experienced users, that they are really doing all the steps and they always know what they should do. This is also very important because they not felt so much lost, like with other tools.

Yeah, that’s very true. But how do we really get that information out to people? What’s a way that you try to get this information out so that folks know the tool is there and are able to access it? Or maybe are able to think of accessibility, as they create the stuff?

Yes. I think maybe I can say something. We try through awareness training, or including this issue in the curriculum of students in teaching. We try to raise awareness about that. Also, many publishers have some legal requirement. For instance, there is an EU Accessibility Act which comes into force next year. PDF accessibility is also one of the requirement. Publishers, like Springer, or other publishers, should actually also try to make their documents, publications accessible.

That doesn’t happen from today to tomorrow, it needs time. That’s a process. But I think little by little, there is awareness among publisher or in the scientific community, also. The conference organizers try to publish on their website that accessibility, that the document authors should provide their documents in an accessible format.

I think, in summary, there are some legal requirement, but also through awareness in teaching or in different other events. We try to raise awareness about the topic.

Maybe you have something to add, Felix?

Yeah. Awareness, that’s I would say a big issue. At the end, also for this tool, that the people know it. It’s super difficult. But yeah, we try to do something with trainings, and so on. There’s a lot of work to do still in this field so that people know about PDF accessibility.

One of my personal wish is that, in the future, more researchers with disabilities are active at universities worldwide. I think assistive technologies provide a way, and also awareness. Not only technologies can maybe help to promote researchers with disabilities, I think, because these guys understand really better the needs. If they have the technical expertise, I think they are good people to be at universities.

What I can tell you, most of the research ideas, they come from my personal life because in the area of education or in the area of mobility that I face. I have these challenges every day.

Well, Alireza, Felix, I got to ask. There at the ICT Accessibility Lab at Zurich University, with other kind of work do you do?

Yes. We have teaching. We have lectures called Digital Assistive Technologies. The people, the students get to know to different assistive technologies. Also, they can carry out mini projects.

Oh.

They can bring an idea and develop a prototype of the assistive technologies.

Nice.

New assistive technologies. That’s a good experience for them to think, to carry out interviews with person with disabilities, and start designing new assistive technologies. Or we have another lecture in the area of implementing accessible websites and accessible apps.

We have, as I mentioned at the beginning, with have two thematic areas. Accessibility education and accessible mobility. Accessible education, PAVE falls within this area. We have also developed other small tools in the area of accessible mobility. We developed an app called Touch Explorer. This app can be downloaded from App Store. That’s an app that present a map within the app. The visually impaired user can touch different parts of the maps presented in the app, and the person or the user hears different sounds. For instance, every day sounds when the user moves on a lake, for instance. Or he or she will hear a sound of the street when his finger or her finger is on a street. Or he can or she can hear birds, the sound of birds, when the forest. The user can explore a little bit the environment. That’s a map that he can zoom in, zoom out, and explore an area. That’s available and can be downloaded.

We have research projects also, in the area of accessible mobility. We use computer vision technologies to build new generation of assistive technologies to support visually impaired users to avoid obstacles, for instance, on the street. We have another project which tries to utilize state-of-art technology to support mobility of other kind of barriers for visually impaired users. This big event is called CYBATHLON. C-Y-B-A-T-H-L-O-N. That’s kind of Olympiads for assistive technologies, assistive technology developers. It takes place every four years in Switzerland. All people, all teams come from all over the world and they compete against each other. For instance, we have a project in the area of, as I mentioned, accessible mobility. And also, we try to be the state-of-the-art technology to support people with visual impairment.

And also, we have one of the project we currently tackling now, research projects, is utilizing AI deep learning with methods to create alternative text or description for scientific graphs. Maybe Felix, can you explain more about this project?

Yeah. Similar to mathematical formulas, it is for most authors, very difficult to describe a graph to create alternative text for alterative text, or also for graphics. We are also developing some AI models to recognizing the graph, so also to support with intuitive user interface to allow a user to simply create alternative text that is really useful at the end for the screen reader user.

Awesome. I guess I should have asked what don’t you do there, because that is-

Yeah. Maybe last project, which one of the PhD students dealing right now, is about ethical aspects of AI for students with disabilities. As you probably know, many universities also will utilize or use AI for teaching, for learning. There are some consideration for students with disabilities, some so-called ethical biases. One of my employees, PhD student also, trying to make research in this area.

Nice. We can’t wait to learn a whole lot more about that. If our listeners do want to find out more, what’s a good way for them to do that?

I think our website is accessibility.zhaw.ch is our website. But in addition, is our PAVE website. The current version of the website is www.pave, P-A-V-E, -PDF.org. The website’s in English and in German. I think Felix will provide also the link for the new prototype, if somebody’s interested to try out and give us feedback. We have some other resources we can just provide, you can put it in the show notes.

I recently have a book. It’s called Research and Teaching Barrier Free. This book contains recommendations for universities to how they can make university or research and teaching, or teaching environment, accessible for researchers and lecturers with disabilities. Maybe that would be also good resource for people who are interested.

Perfect. We will make sure to put all that down in the show notes so our listeners can check it all out.

Well, thank you both so much for coming on today. For telling us not just about PAVE and how it can help folks just make the content they’re trying to put out there in PDF form a whole lot more accessible. But also, all the amazing other things that you do there at the ICB Accessibility Lab, and all the other things that you’re both involved in. Again, thank you so much for coming on the show. We’ll put all that information down in the show notes so people can check it out. We look forward to maybe having you back sometime and maybe learning even more.

Thank you. Thanks for inviting us.

Thank you.

Do you have a question about assistive technology? Do you have a suggestion for someone we should interview on Assistive Technology Update? If so, call our Listener Line at 317-721-7124. Send us an email at tech@eastersealscrossroads.org. Or shoot us a note on Twitter at @indataproject.

Our captions and transcripts for the show are sponsored by the Indiana Telephone Relay Access Corporation, or INTRAC. You can find out more about INTRACT at relayindiana.com. A special thanks to Nikol Prieto, for scheduling our amazing guests and making a mess of my schedule. Today’s show was produced, edited, hosted, and fraught over by yours truly. The opinions expressed by our guests are their own and may or may not reflect those of the INDATA Project, Easterseals Crossroads, our supporting partners, or this host.

This was your Assistive Technology Update. I’m Josh Anderson with the INDATA Project at Easterseals Crossroads in beautiful Indianapolis, Indiana. We look forward to seeing you next time. Bye-bye.