Podcast: Play in new window | Download
Your weekly dose of information that keeps you up to date on the latest developments in the field of technology designed to assist people with disabilities and special needs.
Show notes: BrightSign Glove – Hadeel Ayoub & Phoenix Marcon CEO of Re-Voice | www.sensidy.com
RESNA NewsBrief http://bit.ly/2w6DGgK
——————————
If you have an AT question, leave us a voice mail at: 317-721-7124 or email tech@eastersealscrossroads.org
Check out our web site: https://www.eastersealstech.com
Follow us on Twitter: @INDATAproject
Like us on Facebook: www.Facebook.com/INDATA
——-transcript follows ——
HADEEL AYOUB: Hi, this Hadeel Ayoub, and I’m the CTO of Revoice.
PHOENIX MARCON: Hi, this is Phoenix Marcon, and I’m the CEO of Revoice, and this is your Assistive Technology Update.
WADE WINGLER: Hi, this is Wade Wingler with the INDATA Project at Easter Seals crossroads in Indiana with your Assistive Technology Update, a weekly dose of information that keeps you up-to-date on the latest developments in the field of technology designed to assist people with disabilities and special needs.
Welcome to episode number 325 of Assistive Technology Update, scheduled to be released on August 18, 2017.
Today I have a conversation with Hadeel Ayoub and Phoenix Marcon who are the CIO and CEO respectively of Revoice. They are doing a thing called the Bright Sign Glove, an interesting and emerging technology designed to take sign language and convert it to written and spoken text.
We hope you’ll check out our website at www.eastersealstech.com, sent us a note on Twitter at INDATA Project, or give us a call on our listener line. We love to hear your feedback, questions, comments, all those things. The number is 317-721-7124.
HADEEL AYOUB: Hi, this Hadeel Ayoub, and I’m the CTO of Revoice.
PHOENIX MARCON: Hi, this is Phoenix Marcon, and I’m the CEO of Revoice, and this is your Assistive Technology Update.
WADE WINGLER: Hi, this is Wade Wingler with the INDATA Project at Easter Seals crossroads in Indiana with your Assistive Technology Update, a weekly dose of information that keeps you up-to-date on the latest developments in the field of technology designed to assist people with disabilities and special needs.
Welcome to episode number 325 of Assistive Technology Update, scheduled to be released on August 18, 2017.
Today I have a conversation with Hadeel Ayoub and Phoenix Marcon who are the CIO and CEO respectively of Revoice. They are doing a thing called the Bright Sign Glove, an interesting and emerging technology designed to take sign language and convert it to written and spoken text.
We hope you’ll check out our website at www.eastersealstech.com, sent us a note on Twitter at INDATA Project, or give us a call on our listener line. We love to hear your feedback, questions, comments, all those things. The number is 317-721-7124.
***
Have you gotten a text message from RESNA, the Rehab Engineering and Assistive Technology Society of North America? They are warning that a scammer has sent “phishing” text messages claiming to be resonant leadership. These have included requests for money about ATP recertification fees, outstanding conference fees, or being the subject of an ethical complaint. The warning says RESNA doesn’t use text messages to inform anyone of outstanding fees or complaint investigations, so if you get that kind of message from RESNA, do not click on the link. You can go to our link in our show notes to learn more about this warning.
***
I’m not a signing which expert. I never have been. I did take a lot of signing which classes, but I have to admit my skills were never all that good. The more time that goes that I don’t get a chance to work on those skills, the more they fade. That doesn’t change the fact that I’m always fascinated in technology that is designed to increase communication option for people who are deaf or hard of hearing or people who use ASL for other reasons.
You can imagine how excited I was when I saw a press release about a thing called Bright Sign, which is a glove that – well, I’m not going to spoil it. It really has a lot to do with sign language and technology and I think there are some pretty nerdy technology things here that I really want to dive into with my guests today.
So I was so excited when both the CIO and CEO of a company called Revoice agreed to be on the show today. I am so excited to have Hadeel Ayoub and Phoenix Marcon on the show. Hadeel, Phoenix, thank you for being with us today.
HADEEL AYOUB: Thank you for the introduction.
PHOENIX MARCON: Thank you.
WADE WINGLER: Before we jump into the technical stuff that I’m excited about, I want to hear a little bit about the two of you and how you each became interested in this kind of work, and then also some of the origin story for the product. I know there are some fascinating and interesting stuff there. Hadeel, can I start with you? Tell me about yourself and your background and how you got to where you are today with this.
HADEEL AYOUB: I’m doing research in assistive technology field as part of my PhD degree to look for a niche to research in the field and find a way to apply technology that exists. I was specifically interested in communication. I have relatives who have autism, and I see them trying to communicate in sign language in public and always having to have someone to translate their everyday communication, because in school there is always someone to understand, or at home but what about in public. I thought, for fun, let me try and reassemble a glove that I already had that was drawing in the air making sketches and make it output letters for sign language and see where it goes. When the success rate was amazing, I thought this could be a great PhD project. Let me pursue it further and see where I go with it. That’s what happened. I started developing it further, adding more features to it, recruiting more participants, listening to the feedback of the users to their needs and how it can make their lives better and easier. From there I started prototype one, and now I’m at 13, all feeding back the participants’ uses and testing into the prototype design loop.
WADE WINGLER: I’m going to stop you there because I’m going to ask Phoenix to do the same thing. I think there is a fascinating convergence coming. Phoenix, same question: tell me about your background and how you got to the point where you are with this now.
PHOENIX MARCON: My background is a little less complicated. I’ve been a serial entrepreneur with small turnkey businesses for the last 15 years. About three years ago, I just had this vision of these gloves. I’m asked that question all the time: have you known someone who is deaf? I have not. I started sketching it out with my stick figure type drawing. It seems something that was a viable. If we are sending the general public into space now, why can’t we have clothes for the deaf?
I got together with a patent attorney, and we found a patent a couple of years ago. In that patent search, after we filed it, we came up with Hadeel. We found her in the UK. I read her blog, and she seemed to have the same vision I did, which was the ability to get this to kids. She had said something in her blog that sticks to me to this day, which was ” I hope one day the schools will pick this up.”
We shared that vision. I reached out to her and invited her to become partners. Her glove, her schematics were exactly like mine. Our patents read the same. Basically joined forces, created Revoice, and merged patterns; created another patent which we like to call the “super patent.” That’s where we are at.
WADE WINGLER: You’re telling me that the tagline for Revoice needs to be “Great minds think alike.”
PHOENIX MARCON: Yeah.
HADEEL AYOUB: When it’s for a greater cause, I think there is an attraction. You attract the same vibes you give out. I want to help people and he wants to help people. We share a certain goal. I think the universe got us together.
PHOENIX MARCON: One of my favorite quotes in my life is by George Elliott. He said, “What are we here for if it is not to make life less difficult for others?” When I had this vision, so to speak, of these gloves, I just felt that was something I was called to do. It was fascinating because I was going into something that was way over my head. Then Hadeel came into my life and changed it forever.
WADE WINGLER: You guys fascinate me because I spent the first 20 years of my career doing technical work in the field of assistive technology, and then I went back to business school for an MBA not too long ago. I’m seeing the two skills you guys have. As a look at this from a business perspective, you had the opportunity to compete and become rivals cop but you didn’t. Tell me about the fact that you guys decided to find these synergies. How did that change the trajectory of your work?
PHOENIX MARCON: It’s not about us and it’s not about Revoice. It’s about the kids. I have developed a business model that’s patented itself which allows us the ability to give away these gloves for free to anybody 18 years or younger. That’s what stems in for both of us. You are absolutely right: we basically, in each of our perspectives, eliminated the competition.
HADEEL AYOUB: Yeah. It’s the same for me. I always wanted it to be offered for free to the end-users and people who need it. That’s why I want schools to pick it up or clinics or medical insurance companies to give them out as benefits for the people who need them. For me, it was never about gaining anything. It was about providing it to the end user for free.
PHOENIX MARCON: What’s even more fascinating is Hadeel revealed to me earlier this year. She said, “If I had never sent her the email, the gloves would’ve never seen the light of day.” Because she doesn’t have that business experience and because it was pretty much her PhD project. Imagine that. Just one email changed the course of potentially 70 million deaf people in the world.
WADE WINGLER: I know this will resonate with my audience because there is a lot of altruism in the field of assistive technology. It sounds like a lot of great stuff is lining up to do something special.
A clarifying question for our audience. We’ve talked about people who are deaf, about kids. Is that the target audience and is that the only target audience for the project: deaf kids?
PHOENIX MARCON: It is primarily the target audience, but these gloves are so advanced. What you seen any videos or online for these gloves, it doesn’t even compare to how our patent is written. Just think of military applications for special forces where they could sign to themselves and communicate to their team miles away in complete silence, essentially. You could be a non-deaf person and no sign language and travel to a foreign country and speak in the language in the world using the gloves. There are other applications, but our main focus right now is the kids.
WADE WINGLER: Do you know the story of braille, how it came to be?
PHOENIX MARCON: Not too much.
WADE WINGLER: Braille was originally used by the French military so that spies could read directions and instructions in the dark. Then it became the primary written communication for people who are blind or visually impaired. It’s interesting that you have a military slant on that because that’s how braille was born.
PHOENIX MARCON: I have a military background so I think that came into it.
WADE WINGLER: There you go.
HADEEL AYOUB: We were approached by military asking us to develop a glove specifically for their soldiers to use on the battlefield. This was one of the people who approached us for a completely different application. We got approached by the gaming industry, which I turned down because they have enough virtual reality gloves and gear. They don’t need our glove. It has game applications, music applications because you can play music notes with the different sensors. There are a lot of different applications. Our main focus is on children who use nonverbal ways of communicating. They could be speech disabled, deaf, artistic, nonverbal autism, anybody who uses sign language as their way of communication.
WADE WINGLER: That makes a ton of sense. That leads me to the next question. Talk to me about the technology. Based on my technical experience, there must be some gyroscopes or something like that going on here. Let’s get nerdy. I promised that in the introduction. Tell me about the technology and how it works.
HADEEL AYOUB: I’ve mentioned that there are a lot of different versions of my glove; however, they all have the same sensors. It’s just the output that is written differently. There is definitely a gyroscope to give dynamic movement coordinates to the computer to understand where exactly the hand is in 3-D space. There are bend sensors that give the computer indication of each finger’s position in space: is it completely flexed, bent half way, fall away? This is basically the sensors. Nothing complicated. It’s the software that is complicated. The sensors are very basic.
WADE WINGLER: From the hardware perspective, is this a self-contained device? Is a device dependent? Is it connected to a smart phone? What’s happening with the processing side?
HADEEL AYOUB: It’s both. There is one version that is completely stand-alone. All the hardware is contained within the hardware of the glove. The sensors, the screen that displays the text that is signed, a speaker that outputs the words that are assigned. It’s completely stand-alone with a charging battery and everything. It looks like a normal glove. Everything is hidden within the textile so you don’t feel any of the hardware.
There is another version that connects to a smart device. That’s for translating and training. If you want to train your own version of sign language or have special words that you made up in sign language, you need to be connected to a device in order to train the new words and if you want output in a different language.
WADE WINGLER: That leads me to another – I guess a technical question but not a computer technical question. Talk to me more about the kinds and levels of sign language. There is spoken English, ASL, more primitive gesturing stuff out there. Talk to me about how you handle those differences in the kind of sign language.
HADEEL AYOUB: All my testing has been done with American Sign Language. It’s proof of concept until we go into production. That one library of sign language works in all of its variations. It’s like a model scheme and where I can base all the different assignment with libraries. For every market, we are going to have to program a sign language library that that market uses. For autism, they use a completely different kind of sign language which is called mechaton, and it’s action words. For that, I have also developed a different scheme because I have worked with children with autism in the first round of testing. I have to program just for them.
For right now, it works with American sign language. Later on it can work with any library of sonically depending on the market needs.
WADE WINGLER: This may be further down the road, but there is regional sign language basic communities. For an example, I’m in Indianapolis and I know there are some Indianapolis signs and also situational sign language where people have signed names and things like that. Is it on your radar to be able to include those kinds of things? Would that be built into the product or would people add their own as they go?
HADEEL AYOUB: Again, it’s both. They will come built in as standard library assignment which and then there is the option to train the glove with any number of words you want. We had a at one of the conferences where people didn’t know how to use time they would but were able to train the glove for their made up sign language and it still out but whatever they wanted it to say. A person would put a sign that he made up that that I love dogs. Every time he made that sign, the glove would say I love dogs. It’s very important to personalize the glove to every person is used because, as you said, there are variations. You and I can sign the same words yet the coordinates and sensors pick it up in a different way.
The glove learns from the user. The more you use it, the smarter it becomes an understanding or hand movements. That’s where the machine learning comes in and training comes in. Essentially the glove comes equipped with both: a standard library and the ability of training for your own use.
Did that answer your question?
WADE WINGLER: It does. Talking about machine learning, artificial intelligence makes a ton of sense, especially in this situation, because there is so much that is individualized and specific to regions or users or even conversations and days.
I guess it leads me to another question. In terms of your sensors, they have to be pretty discrete, right? Sometimes the sign language is subtle and sometimes sign language is fast. When I’m observing people who do sign language – again, I’m limited in that area – there are times when I miss stuff because they are going so fast on the signs are little and subtle. How do you deal with those things?
HADEEL AYOUB: This came up with testing. They were signing and playing and building and coloring at the same time so the glove just went crazy trying to process the whole time what was being signed. A design solution was to have a button to tell the glove, “I’m starting to sign now,” and then tell the glove, “Hey, I just stopped signing. Can you please process.” This worked amazingly because they cut the errors to half. With the next round of testing, the glove was so much better in output because he knew exactly when to process and when to stop processing, even if the signing was fast. That was one way of solving it. It’s a process. We learn from the testing and go back.
Then we got another user who said, “I hate the buttons. I don’t want the buttons. Think of something else.” So then we created gestures to tell the glove when to start and when to end. There is gesture to start, sleep, end, stuff like that.
WADE WINGLER: I guess for a lot of users, it would be analogous to action buttons they are familiar with, like Siri, or Alexa, or those kinds of things were it listens all the time, but now pay attention.
HADEEL AYOUB: Exactly.
WADE WINGLER: Another linguistic question. This seems to be a one-way solution. We are talking about using gestures or ASL to create text to speech so that somebody can read or hear what’s going on. That’s part of the communication process. Do you have any plans or vision for the other side of the communication process where a speaking person might be able to communicate with somebody who is deaf or hard of hearing using this technology?
HADEEL AYOUB: Absolutely. The device-dependent glove will have this feature because it has to be processed in a computer, not on a glove. Something like this already exists. They just need to plug into the software. This is one of the plans, definitely.
WADE WINGLER: What are your users and beta testers saying about the glove? You’ve given a few pics into that, but what kind of reaction are you getting from people as they experience this?
HADEEL AYOUB: It’s those kinds of reactions that kept us going. You know how bumpy it can be to go into production of something that is unheard of. The reactions we get from families and from the kids is amazing. The look on their faces when they realized the other person can understand what they are saying was really fulfilling and satisfying to us.
I’ve got so many parents waiting for this to become a product so they can purchase it for their children. We’ve promised a lot of gloves once we have them.
Even if it solved 50 percent of the communication problem, it does solve 50 percent of a problem so it’s good enough. We are not trying to make it perfect before we give it out because in the state it is aiming, it still helps. That’s the point. We want it to make people’s lives easier, and then we can develop it from there and upgrade just like Apple gives you an update every couple of weeks.
WADE WINGLER: I agree. I think assistive technology is always done best when it is an iterative process. That’s what you’re doing.
Phoenix, I know you are more the business partner in this situation. What is the current product’s status? When will it be available? How much will it cost? What does that look like for you guys?
PHOENIX MARCON: As with any startup, specifically tech startup, right now we are in the seeking the funding phase. We have turned down quite a bit of funding from various avenues over the years because the individuals that have wanted to fund us, they want to turn it into a full retail product, which basically defies our business model. We can’t say too little Jimmy, “Hey little Jimmy, give us 10 grand so you can start making friends.” It goes against what we believe in. We are just looking for that right partner that’s willing to invest in us in this social product, so to speak, and still get a good return on that. That’s where we are right now. We’ve got some good leads. Once we secure that hopefully within the next 60 to 90 days, then we should start going into manufacturing.
WADE WINGLER: In my clear in my assumption that your business model is about giving away the product to the people who are deaf or hard of hearing and then using those gaming or military commercial aspects to offset your cost to do that?
PHOENIX MARCON: Right now the model is anyone under 18, we have a system in place to where they would essentially get it for free, and anybody over 18 would have a charge. We are talking probably around the $2,000 mark, which is pretty minimal considering – have you heard of cochlear implants?
WADE WINGLER: You bet.
PHOENIX MARCON: I have a deaf friend who has been deaf since she was two. She’s been rejected by her insurance company five times who says she doesn’t qualify — which I find fascinating. On the fifth time, she said fine, I’ll take you to court – anyway, two weeks later she got a letter stating she qualified. It’s $175,000. It’s a process I don’t agree with. We feel $2,000-$2,500 is a relatively good cost for an adult.
WADE WINGLER: What is your timeframe? I’ve seen some headlines recently that said you are looking like something in 2018?
PHOENIX MARCON: Correct.
WADE WINGLER: If people wanted to learn more about what you are doing with the Bright Sign glove at Revoice, what contact information would you like to provide?
PHOENIX MARCON: Right now they can go to our company called Sensidy.com. That our Revoice site. It’s periodically updated with new information.
WADE WINGLER: I’ll pop that link in the show notes so people can go directly to that. Before we wrap up – and we are about out of time – what does success look like? Five years down the road when this has gone smashingly well, what does it look like?
HADEEL AYOUB: For me, attending all the conferences specifically dedicated to people with disabilities. I would like to see these translators replaced with our gloves. It’ll save time, money, resources, and it is an investment in itself. Instead of recruiting 15 translators in every conference, they’ll just have two or three gloves and it will take care of the whole event. I’m talking globally. Replacing all the translators, individual translators, conference translators, replacing them with our gloves.
WADE WINGLER: Phoenix?
PHOENIX MARCON: To add on to that, just for your audience’s information, we had scheduled an interview last month with a deaf speaking individual. He sent an invoice, which was no problem, for a translator. That was the first time I’ve actually seen an invoice for a translator from a translation company. For a two-hour meeting, it was $379. It’s unfortunate. I feel like it’s one of those businesses that’s taking advantage of the hearing-challenged. But to answer your question, my personal goal is that 1 million kids have these gloves.
WADE WINGLER: There you go.
PHOENIX MARCON: That’s the five-year plan.
HADEEL AYOUB: One day can we hope to give a voice to those who can’t speak. We want our glove to be an extension of the senses, something they have with them at all time that gives them voice and freedom to interact in society without having someone be with them the whole time.
PHOENIX MARCON: I don’t know if you’ve seen our tagline, but it’s simple. It’s HELP, Hear Every Living Person.
WADE WINGLER: That’s excellent. I hear my lines lighting up by the interpreters who were worried about their jobs at this point. I do think there is some new ones and things about communication that I think are important to get here. I think you are making a step in the right direction. I think what you are doing with Bright sign glove and the work you’re doing, I think it’s one of the things we will look back in a few years and say this was the first thing that moved us in a direction that was good for everybody. I appreciate your being on the show today.
Hadeel Ayoub is the Chief Technology Officer for Revoice, and Phoenix Marcon is the CEO. Thank you so much for being on the show.
HADEEL AYOUB: Thank you.
PHOENIX MARCON: Thank you so much.
WADE WINGLER: Do you have a question about assistive technology? Do you have a suggestion for someone we should interview on Assistive Technology Update? Call our listener line at 317-721-7124, shoot us a note on Twitter @INDATAProject, or check us out on Facebook. Looking for a transcript or show notes from today’s show? Head on over to www.EasterSealstech.com. Assistive Technology Update is a proud member of the Accessibility Channel. Find more shows like this plus much more over at AccessibilityChannel.com. That was your Assistance Technology Update. I’m Wade Wingler with the INDATA Project at Easter Seals Crossroads in Indiana.
***Transcript provided by TJ Cortopassi. For requests and inquiries, contact tjcortopassi@gmail.com***
***
Have you gotten a text message from RESNA, the Rehab Engineering and Assistive Technology Society of North America? They are warning that a scammer has sent “phishing” text messages claiming to be resonant leadership. These have included requests for money about ATP recertification fees, outstanding conference fees, or being the subject of an ethical complaint. The warning says RESNA doesn’t use text messages to inform anyone of outstanding fees or complaint investigations, so if you get that kind of message from RESNA, do not click on the link. You can go to our link in our show notes to learn more about this warning.
***
I’m not a signing which expert. I never have been. I did take a lot of signing which classes, but I have to admit my skills were never all that good. The more time that goes that I don’t get a chance to work on those skills, the more they fade. That doesn’t change the fact that I’m always fascinated in technology that is designed to increase communication option for people who are deaf or hard of hearing or people who use ASL for other reasons.
You can imagine how excited I was when I saw a press release about a thing called Bright Sign, which is a glove that – well, I’m not going to spoil it. It really has a lot to do with sign language and technology and I think there are some pretty nerdy technology things here that I really want to dive into with my guests today.
So I was so excited when both the CIO and CEO of a company called Revoice agreed to be on the show today. I am so excited to have Hadeel Ayoub and Phoenix Marcon on the show. Hadeel, Phoenix, thank you for being with us today.
HADEEL AYOUB: Thank you for the introduction.
PHOENIX MARCON: Thank you.
WADE WINGLER: Before we jump into the technical stuff that I’m excited about, I want to hear a little bit about the two of you and how you each became interested in this kind of work, and then also some of the origin story for the product. I know there are some fascinating and interesting stuff there. Hadeel, can I start with you? Tell me about yourself and your background and how you got to where you are today with this.
HADEEL AYOUB: I’m doing research in assistive technology field as part of my PhD degree to look for a niche to research in the field and find a way to apply technology that exists. I was specifically interested in communication. I have relatives who have autism, and I see them trying to communicate in sign language in public and always having to have someone to translate their everyday communication, because in school there is always someone to understand, or at home but what about in public. I thought, for fun, let me try and reassemble a glove that I already had that was drawing in the air making sketches and make it output letters for sign language and see where it goes. When the success rate was amazing, I thought this could be a great PhD project. Let me pursue it further and see where I go with it. That’s what happened. I started developing it further, adding more features to it, recruiting more participants, listening to the feedback of the users to their needs and how it can make their lives better and easier. From there I started prototype one, and now I’m at 13, all feeding back the participants’ uses and testing into the prototype design loop.
WADE WINGLER: I’m going to stop you there because I’m going to ask Phoenix to do the same thing. I think there is a fascinating convergence coming. Phoenix, same question: tell me about your background and how you got to the point where you are with this now.
PHOENIX MARCON: My background is a little less complicated. I’ve been a serial entrepreneur with small turnkey businesses for the last 15 years. About three years ago, I just had this vision of these gloves. I’m asked that question all the time: have you known someone who is deaf? I have not. I started sketching it out with my stick figure type drawing. It seems something that was a viable. If we are sending the general public into space now, why can’t we have clothes for the deaf?
I got together with a patent attorney, and we found a patent a couple of years ago. In that patent search, after we filed it, we came up with Hadeel. We found her in the UK. I read her blog, and she seemed to have the same vision I did, which was the ability to get this to kids. She had said something in her blog that sticks to me to this day, which was ” I hope one day the schools will pick this up.”
We shared that vision. I reached out to her and invited her to become partners. Her glove, her schematics were exactly like mine. Our patents read the same. Basically joined forces, created Revoice, and merged patterns; created another patent which we like to call the “super patent.” That’s where we are at.
WADE WINGLER: You’re telling me that the tagline for Revoice needs to be “Great minds think alike.”
PHOENIX MARCON: Yeah.
HADEEL AYOUB: When it’s for a greater cause, I think there is an attraction. You attract the same vibes you give out. I want to help people and he wants to help people. We share a certain goal. I think the universe got us together.
PHOENIX MARCON: One of my favorite quotes in my life is by George Elliott. He said, “What are we here for if it is not to make life less difficult for others?” When I had this vision, so to speak, of these gloves, I just felt that was something I was called to do. It was fascinating because I was going into something that was way over my head. Then Hadeel came into my life and changed it forever.
WADE WINGLER: You guys fascinate me because I spent the first 20 years of my career doing technical work in the field of assistive technology, and then I went back to business school for an MBA not too long ago. I’m seeing the two skills you guys have. As a look at this from a business perspective, you had the opportunity to compete and become rivals cop but you didn’t. Tell me about the fact that you guys decided to find these synergies. How did that change the trajectory of your work?
PHOENIX MARCON: It’s not about us and it’s not about Revoice. It’s about the kids. I have developed a business model that’s patented itself which allows us the ability to give away these gloves for free to anybody 18 years or younger. That’s what stems in for both of us. You are absolutely right: we basically, in each of our perspectives, eliminated the competition.
HADEEL AYOUB: Yeah. It’s the same for me. I always wanted it to be offered for free to the end-users and people who need it. That’s why I want schools to pick it up or clinics or medical insurance companies to give them out as benefits for the people who need them. For me, it was never about gaining anything. It was about providing it to the end user for free.
PHOENIX MARCON: What’s even more fascinating is Hadeel revealed to me earlier this year. She said, “If I had never sent her the email, the gloves would’ve never seen the light of day.” Because she doesn’t have that business experience and because it was pretty much her PhD project. Imagine that. Just one email changed the course of potentially 70 million deaf people in the world.
WADE WINGLER: I know this will resonate with my audience because there is a lot of altruism in the field of assistive technology. It sounds like a lot of great stuff is lining up to do something special.
A clarifying question for our audience. We’ve talked about people who are deaf, about kids. Is that the target audience and is that the only target audience for the project: deaf kids?
PHOENIX MARCON: It is primarily the target audience, but these gloves are so advanced. What you seen any videos or online for these gloves, it doesn’t even compare to how our patent is written. Just think of military applications for special forces where they could sign to themselves and communicate to their team miles away in complete silence, essentially. You could be a non-deaf person and no sign language and travel to a foreign country and speak in the language in the world using the gloves. There are other applications, but our main focus right now is the kids.
WADE WINGLER: Do you know the story of braille, how it came to be?
PHOENIX MARCON: Not too much.
WADE WINGLER: Braille was originally used by the French military so that spies could read directions and instructions in the dark. Then it became the primary written communication for people who are blind or visually impaired. It’s interesting that you have a military slant on that because that’s how braille was born.
PHOENIX MARCON: I have a military background so I think that came into it.
WADE WINGLER: There you go.
HADEEL AYOUB: We were approached by military asking us to develop a glove specifically for their soldiers to use on the battlefield. This was one of the people who approached us for a completely different application. We got approached by the gaming industry, which I turned down because they have enough virtual reality gloves and gear. They don’t need our glove. It has game applications, music applications because you can play music notes with the different sensors. There are a lot of different applications. Our main focus is on children who use nonverbal ways of communicating. They could be speech disabled, deaf, artistic, nonverbal autism, anybody who uses sign language as their way of communication.
WADE WINGLER: That makes a ton of sense. That leads me to the next question. Talk to me about the technology. Based on my technical experience, there must be some gyroscopes or something like that going on here. Let’s get nerdy. I promised that in the introduction. Tell me about the technology and how it works.
HADEEL AYOUB: I’ve mentioned that there are a lot of different versions of my glove; however, they all have the same sensors. It’s just the output that is written differently. There is definitely a gyroscope to give dynamic movement coordinates to the computer to understand where exactly the hand is in 3-D space. There are bend sensors that give the computer indication of each finger’s position in space: is it completely flexed, bent half way, fall away? This is basically the sensors. Nothing complicated. It’s the software that is complicated. The sensors are very basic.
WADE WINGLER: From the hardware perspective, is this a self-contained device? Is a device dependent? Is it connected to a smart phone? What’s happening with the processing side?
HADEEL AYOUB: It’s both. There is one version that is completely stand-alone. All the hardware is contained within the hardware of the glove. The sensors, the screen that displays the text that is signed, a speaker that outputs the words that are assigned. It’s completely stand-alone with a charging battery and everything. It looks like a normal glove. Everything is hidden within the textile so you don’t feel any of the hardware.
There is another version that connects to a smart device. That’s for translating and training. If you want to train your own version of sign language or have special words that you made up in sign language, you need to be connected to a device in order to train the new words and if you want output in a different language.
WADE WINGLER: That leads me to another – I guess a technical question but not a computer technical question. Talk to me more about the kinds and levels of sign language. There is spoken English, ASL, more primitive gesturing stuff out there. Talk to me about how you handle those differences in the kind of sign language.
HADEEL AYOUB: All my testing has been done with American Sign Language. It’s proof of concept until we go into production. That one library of sign language works in all of its variations. It’s like a model scheme and where I can base all the different assignment with libraries. For every market, we are going to have to program a sign language library that that market uses. For autism, they use a completely different kind of sign language which is called mechaton, and it’s action words. For that, I have also developed a different scheme because I have worked with children with autism in the first round of testing. I have to program just for them.
For right now, it works with American sign language. Later on it can work with any library of sonically depending on the market needs.
WADE WINGLER: This may be further down the road, but there is regional sign language basic communities. For an example, I’m in Indianapolis and I know there are some Indianapolis signs and also situational sign language where people have signed names and things like that. Is it on your radar to be able to include those kinds of things? Would that be built into the product or would people add their own as they go?
HADEEL AYOUB: Again, it’s both. They will come built in as standard library assignment which and then there is the option to train the glove with any number of words you want. We had a at one of the conferences where people didn’t know how to use time they would but were able to train the glove for their made up sign language and it still out but whatever they wanted it to say. A person would put a sign that he made up that that I love dogs. Every time he made that sign, the glove would say I love dogs. It’s very important to personalize the glove to every person is used because, as you said, there are variations. You and I can sign the same words yet the coordinates and sensors pick it up in a different way.
The glove learns from the user. The more you use it, the smarter it becomes an understanding or hand movements. That’s where the machine learning comes in and training comes in. Essentially the glove comes equipped with both: a standard library and the ability of training for your own use.
Did that answer your question?
WADE WINGLER: It does. Talking about machine learning, artificial intelligence makes a ton of sense, especially in this situation, because there is so much that is individualized and specific to regions or users or even conversations and days.
I guess it leads me to another question. In terms of your sensors, they have to be pretty discrete, right? Sometimes the sign language is subtle and sometimes sign language is fast. When I’m observing people who do sign language – again, I’m limited in that area – there are times when I miss stuff because they are going so fast on the signs are little and subtle. How do you deal with those things?
HADEEL AYOUB: This came up with testing. They were signing and playing and building and coloring at the same time so the glove just went crazy trying to process the whole time what was being signed. A design solution was to have a button to tell the glove, “I’m starting to sign now,” and then tell the glove, “Hey, I just stopped signing. Can you please process.” This worked amazingly because they cut the errors to half. With the next round of testing, the glove was so much better in output because he knew exactly when to process and when to stop processing, even if the signing was fast. That was one way of solving it. It’s a process. We learn from the testing and go back.
Then we got another user who said, “I hate the buttons. I don’t want the buttons. Think of something else.” So then we created gestures to tell the glove when to start and when to end. There is gesture to start, sleep, end, stuff like that.
WADE WINGLER: I guess for a lot of users, it would be analogous to action buttons they are familiar with, like Siri, or Alexa, or those kinds of things were it listens all the time, but now pay attention.
HADEEL AYOUB: Exactly.
WADE WINGLER: Another linguistic question. This seems to be a one-way solution. We are talking about using gestures or ASL to create text to speech so that somebody can read or hear what’s going on. That’s part of the communication process. Do you have any plans or vision for the other side of the communication process where a speaking person might be able to communicate with somebody who is deaf or hard of hearing using this technology?
HADEEL AYOUB: Absolutely. The device-dependent glove will have this feature because it has to be processed in a computer, not on a glove. Something like this already exists. They just need to plug into the software. This is one of the plans, definitely.
WADE WINGLER: What are your users and beta testers saying about the glove? You’ve given a few pics into that, but what kind of reaction are you getting from people as they experience this?
HADEEL AYOUB: It’s those kinds of reactions that kept us going. You know how bumpy it can be to go into production of something that is unheard of. The reactions we get from families and from the kids is amazing. The look on their faces when they realized the other person can understand what they are saying was really fulfilling and satisfying to us.
I’ve got so many parents waiting for this to become a product so they can purchase it for their children. We’ve promised a lot of gloves once we have them.
Even if it solved 50 percent of the communication problem, it does solve 50 percent of a problem so it’s good enough. We are not trying to make it perfect before we give it out because in the state it is aiming, it still helps. That’s the point. We want it to make people’s lives easier, and then we can develop it from there and upgrade just like Apple gives you an update every couple of weeks.
WADE WINGLER: I agree. I think assistive technology is always done best when it is an iterative process. That’s what you’re doing.
Phoenix, I know you are more the business partner in this situation. What is the current product’s status? When will it be available? How much will it cost? What does that look like for you guys?
PHOENIX MARCON: As with any startup, specifically tech startup, right now we are in the seeking the funding phase. We have turned down quite a bit of funding from various avenues over the years because the individuals that have wanted to fund us, they want to turn it into a full retail product, which basically defies our business model. We can’t say too little Jimmy, “Hey little Jimmy, give us 10 grand so you can start making friends.” It goes against what we believe in. We are just looking for that right partner that’s willing to invest in us in this social product, so to speak, and still get a good return on that. That’s where we are right now. We’ve got some good leads. Once we secure that hopefully within the next 60 to 90 days, then we should start going into manufacturing.
WADE WINGLER: In my clear in my assumption that your business model is about giving away the product to the people who are deaf or hard of hearing and then using those gaming or military commercial aspects to offset your cost to do that?
PHOENIX MARCON: Right now the model is anyone under 18, we have a system in place to where they would essentially get it for free, and anybody over 18 would have a charge. We are talking probably around the $2,000 mark, which is pretty minimal considering – have you heard of cochlear implants?
WADE WINGLER: You bet.
PHOENIX MARCON: I have a deaf friend who has been deaf since she was two. She’s been rejected by her insurance company five times who says she doesn’t qualify — which I find fascinating. On the fifth time, she said fine, I’ll take you to court – anyway, two weeks later she got a letter stating she qualified. It’s $175,000. It’s a process I don’t agree with. We feel $2,000-$2,500 is a relatively good cost for an adult.
WADE WINGLER: What is your timeframe? I’ve seen some headlines recently that said you are looking like something in 2018?
PHOENIX MARCON: Correct.
WADE WINGLER: If people wanted to learn more about what you are doing with the Bright Sign glove at Revoice, what contact information would you like to provide?
PHOENIX MARCON: Right now they can go to our company called Sensidy.com. That our Revoice site. It’s periodically updated with new information.
WADE WINGLER: I’ll pop that link in the show notes so people can go directly to that. Before we wrap up – and we are about out of time – what does success look like? Five years down the road when this has gone smashingly well, what does it look like?
HADEEL AYOUB: For me, attending all the conferences specifically dedicated to people with disabilities. I would like to see these translators replaced with our gloves. It’ll save time, money, resources, and it is an investment in itself. Instead of recruiting 15 translators in every conference, they’ll just have two or three gloves and it will take care of the whole event. I’m talking globally. Replacing all the translators, individual translators, conference translators, replacing them with our gloves.
WADE WINGLER: Phoenix?
PHOENIX MARCON: To add on to that, just for your audience’s information, we had scheduled an interview last month with a deaf speaking individual. He sent an invoice, which was no problem, for a translator. That was the first time I’ve actually seen an invoice for a translator from a translation company. For a two-hour meeting, it was $379. It’s unfortunate. I feel like it’s one of those businesses that’s taking advantage of the hearing-challenged. But to answer your question, my personal goal is that 1 million kids have these gloves.
WADE WINGLER: There you go.
PHOENIX MARCON: That’s the five-year plan.
HADEEL AYOUB: One day can we hope to give a voice to those who can’t speak. We want our glove to be an extension of the senses, something they have with them at all time that gives them voice and freedom to interact in society without having someone be with them the whole time.
PHOENIX MARCON: I don’t know if you’ve seen our tagline, but it’s simple. It’s HELP, Hear Every Living Person.
WADE WINGLER: That’s excellent. I hear my lines lighting up by the interpreters who were worried about their jobs at this point. I do think there is some new ones and things about communication that I think are important to get here. I think you are making a step in the right direction. I think what you are doing with Bright sign glove and the work you’re doing, I think it’s one of the things we will look back in a few years and say this was the first thing that moved us in a direction that was good for everybody. I appreciate your being on the show today.
Hadeel Ayoub is the Chief Technology Officer for Revoice, and Phoenix Marcon is the CEO. Thank you so much for being on the show.
HADEEL AYOUB: Thank you.
PHOENIX MARCON: Thank you so much.
WADE WINGLER: Do you have a question about assistive technology? Do you have a suggestion for someone we should interview on Assistive Technology Update? Call our listener line at 317-721-7124, shoot us a note on Twitter @INDATAProject, or check us out on Facebook. Looking for a transcript or show notes from today’s show? Head on over to www.EasterSealstech.com. Assistive Technology Update is a proud member of the Accessibility Channel. Find more shows like this plus much more over at AccessibilityChannel.com. That was your Assistance Technology Update. I’m Wade Wingler with the INDATA Project at Easter Seals Crossroads in Indiana.
***Transcript provided by TJ Cortopassi. For requests and inquiries, contact tjcortopassi@gmail.com***