ATU670 – Floreo with Vijay Ravindran

Hi, this is Vijay Ravindran and I’m the CEO and founder of Floreo, and this is your Assistive Technology Update.

Hello and welcome to your Assistive Technology Update, a weekly dose of information that keeps you up to date on the latest developments in the field of technology designed to assist individuals with disabilities and special needs. I’m your host, Josh Anderson with the INDATA Project at Easter Seals Crossroads in beautiful Indianapolis, Indiana. Welcome to episode 670 of Assistive Technology Update, it is scheduled to be released on March 29th 2024. On today’s show, we’re looking forward to learning all about Floreo and how they’re using virtual reality to help individuals with a myriad of needs. We have a quick story about a new patch being developed that can help individuals use artificial intelligence and machine learning to assist with vocalization. We also welcome back Amy Barry from BridgingApps with an app worth mentioning. Don’t forget listeners, we always love hearing from you, so if you have a question, a comment, perhaps a suggestion for a guest that we could have on to interview, we want to hear them all. You can call our listener line at 317-721-7124 or shoot us an email at tech@eastersealscrossroads.org. Also, feel free to give us a like, subscribe or leave a comment on whatever service it is that you’re using to listen to us. As always, listeners, we thank you so much for taking time out of your day to listen to our show. So let’s go ahead and get on with it.

It’s our first story today comes from IEEE Spectrum. It’s titled, “Elastic Patch Tech Helps Vocally Impaired Speak, Non-Invasive AI Assisted Device converts Neck Movements to Words and Phrases.” It’s written by Michael Nolan and it talks about a new device coming out of the Biomedical Engineering Lab at UCLA. The device itself is a small kind of fabric patch, a small square that is actually worn on the neck of an individual who perhaps has difficulty using their vocal tracts to speak. And what this device does is it sits there and it measures the muscle movements of your neck as the individual would try to form words and sentences. It then sends that information to another device, which uses artificial intelligence and machine learning to translate those movements into speech. So essentially it can help folks who perhaps have had, either their vocal cords removed, or perhaps some other kind of ailment, paralysis, and other things that would limit their ability to speak.

Let’s say here in the story that this is a prototype AI model and its performance at least at this time is very limited. It can only really kind of detect five different sentences, and those are, “Hi, how are you doing today?” “Hope your experiments are going well.” “Merry Christmas.” “I love you,” and “I don’t trust you.” Kind of a wild array of different things that are there. But also this is very, very new. It’s very non-invasive, so if we kind of think a lot of folks that maybe have had a tracheotomy, or some other procedure, or have had their vocal cords removed, we kind of think of the old-school kind of microphone that you hold up there and it would sit there and kind of give you a voice. So this is actually something that could maybe replace or augment that, and actually be just placed on the neck like a big square sticker that then sends that information.

As I said, it’s very new, kind of research of very, very new device, so that’s probably why it’s so limited, but as with a lot of artificial intelligence and machine learning these days, I’m sure that will grow quite quickly before it does come out to the public. I will put a link to this over in the show notes so that you can really go and read as it digs a little bit more into how the device works, how it was created, and some other information of interest just on how it might be able to help folks.

But really, as with the most assistive technology and these kind of medical devices, it just gives folks another choice and maybe another option. So who knows when this actually does come to market, what it will look like if it will be anywhere near the initial concept or the initial offering of this type of device. But it’s just very cool, and a great new way to use artificial intelligence and machine learning to help individuals with disabilities be able to access the world a little bit more, and be able to communicate with others. Again, we will put a link to this down in the show notes so that you can go and check out more about it for yourself.

Listeners, up next, we are very excited to welcome back Amy Barry from BridgingApps to the show with an app worth mentioning. Take it Away, Amy.

This is Amy Barry with BridgingApps, and this is an app worth mentioning. This week’s featured app is DoorDash. Delivery Services such as DoorDash play a major role in people’s lives, specifically those with physical disabilities who encounter difficulties shopping in person. These services provide a convenient solution, enabling them to acquire essential items from the comfort of their homes. Online shopping platforms are designed to be user-friendly and often incorporate accessibility features, facilitating seamless navigation and order placement for those with mobility or vision impairments. This assistance fosters independence and enhances the quality of life for individuals by ensuring they have a reliable access to groceries and other necessities. We’ve found that DoorDash offers a wide variety of options from your favorite restaurants and stores, making it easy to get freshly prepared meals and groceries delivered to your doorstep.

You can also get over-the-counter medicines, drinks, snacks, flowers, and more delivered. You can choose from getting your items on demand, same day delivery, and you can even schedule deliveries, so that you get your items when it’s most convenient. A great feature for individuals with disabilities is real-time tracking, so they can see when their order will arrive and be able to prepare for the pickup. DoorDash also offers no contact delivery. Your food will be left in a safe place and you’ll be alerted that it’s ready for you to pick up at your doorstep. DoorDash is available for iOS and Android. The app is free to download. For more information on this app and others like it visit Bridgingapps.org

Listeners, virtual reality and its uses are in the news quite often these days and today we are very excited to have an individual from Floreo on to talk about how this program can utilize virtual reality to make a positive impact on the lives of individuals with disabilities. Vijay, welcome to the show.

Thank you for having me.

Yeah, I’m excited to get talking about everything that Floreo really has to offer, but before we do that, could you tell our listeners a little bit about yourself?

Yeah, sure. I’m a software engineer by background, early engineer at Amazon, joining in 1998, was there for seven years during the early days of Amazon’s rise in retail online, and then most recently was at the Washington Post, working on technology to help journalists. And I started Floreo inspired by my son, I’m the dad of a child on the autism spectrum and saw VR as a modality that could possibly help neurodiverse kids.

Awesome. That brings me to kind of my next question is what is Floreo?

Floreo is a virtual reality therapy system that takes virtual reality content that’s clinically designed to help with social life and independent skills development, delivers it in mobile virtual reality headsets, so that a learner, a person who might benefit from those types of trainings, can experience those scenes. All the while, there’s a companion coaching application that a supervising adult, which can range from a highly trained professional like a therapist, to special educators, to even parents or other caregivers, to be able to snap around, see what the learner is seeing, be able to manipulate characters to create social dynamics and provide live coaching. And so, the Floreo system helps neurodiverse learners gain new skills.

Nice. And I am sure that there’s probably way too many to name here in one podcast, but what are some of the situations that are available for individuals to explore?

So the Floreo system has over 200 lessons today and growing, so it is a long list. They’re listed on our website at FloreoVR.com, but broadly speaking, it’s a wide range. So we have modules aimed at younger kids in earlier developmental phases to practice things like how to raise your hand in the classroom or how to make small talk in the cafeteria to safety areas like crossing the street, law enforcement encounters, handling bullying scenarios, to content for older learners who might be looking at independence, how to shop in the grocery store, how to think about time management or teamwork in a job setting with colleagues. And so virtual reality affords an environment that can enable practice in an immersive environment that can convey skills that we’ve been able to show, can generalize and can lead to faster skill acquisition.

Nice. And you said that there’s another side of it where the teacher, the coach, the therapist, the other person not using the virtual reality can see what’s going on. Is there a way for them to track progress or kind of see if the individual is, “Mastering skills” isn’t the right really word I’m looking for there, but is able to progress through the scenarios, I guess?

Yeah, the key part of the coaching role and the coaching application is that that individual is able to see everything that the learner is seeing in virtual reality on an iPad or a computer-based screen. And so, from that coaching app, they can see what the learner is seeing, they can provide live coaching, depending on the lesson, they might be required to press buttons to kind of guide the interaction of that learner with virtual avatars that are interacting with them. And they’re also tracking progress, and when needed, qualitatively scoring whether the learner was able to perform a certain action or not, which then is able to be available if it’s going into a report or be able to deem whether the learner has achieved mastery.

Nice. And I’m sure that that just has to work better for some individuals than actually going out and being on a city bus, or actually out in public or some of them, you mentioned bullying, not quite a situation you really want to have to learn that skill while you’re in that sort of experience, I suppose.

Yeah. I think the way we tend to think about it is that there are a lot of situations where the cost of failing can be high. So think of crossing the street; if you get it wrong and get hit by a car, that’s a severe situation, so you want to do what you can to not have that happen. There’s also that if you can start in VR, it can be more gentle, so you can graduate up to some of those real situations with a better sense of confidence, which can lead to a more comfortable situation. And so, the VR environment is very much one that can be a launchpad for success in the real world.

Oh, definitely, definitely. And I have to ask, what kinds of headsets does Floreo work on? Does it have a one headset it’s preferred on, or I guess, what kind of virtual reality device do I need in order to be able to use it?

So today, we support iPhones that can be placed into virtual reality goggles that typically run between 15 to $30 retail. And then we do also support the Meta Quest Virtual Reality Headset family. Those are today available to enterprise customers, and we’re working to make that available to families as well soon. And later this year, you’ll see Floreo on several other virtual reality headset platforms.

Nice. So there’s definitely some choice there. You don’t have to, if you’ve already got an iPhone or a device like that, then with just a little bit of kind of inexpensive peripheral, you could easily kind of get into the whole system. So that’s really cool. I’m sure you have tons of these, but can you tell me a story about how Floreo was able to make a positive impact in their life?

Absolutely. I think one of the earliest stories that really captured me was a child who, when they were very young, was hit by a car in a parking lot and it led to the child having great aversion to being out in the community. So the family couldn’t drive anywhere to take the child, because the child just did not want to get out a car, given what had happened. And the schools district that we worked with, we worked with Chester County Intermediate Unit in Pennsylvania, was able to work with the individual using our street crossing lessons that gave them a familiarization with cars passing by and not being in danger if you’re on the sidewalk safely, and eventually led to that child being successfully able to navigate to travel with their family into parking lots. And so, that was an early win that struck me.

I’ll give you one other, and that was that we had a family reach out during the pandemic, and it was a really terrible situation where the parents had actually both passed away due to COVID, and so a grandparent was raising the child. And this was before the vaccines, so the grandparent was not going out into the community and the child was not making it to any therapy sessions. And so, the Floreo system was a way that that grandparent could step into providing some form of skill acquisition with that child during this period where visiting the clinical center was not an option.

Man, I never even thought about that. Still being able to practice those skills, and do those things and see the results without actually having to go anywhere. I guess that also kind of raises a question for me. Does the, I keep wanting to say, “Provider,” I know that’s not quite the right word, but the teacher, the coach, the individual kind of on the other side, do they have to be right there in the room with the individual, or is this something that can be done remotely or how does that work?

I’m so glad you asked. Yeah, so that’s one of the key facets of our system that it’s really helped gain adoption, which is that, that coaching application that I described earlier can really run in two ways. It can be running in person, in the same room on the same WI-FI network, which is how many schools and parents use it, and clinical centers that have in-clinic therapy. But we have a growing cadre of innovative care practices that are able to use the system as part of a telehealth product. And the coaching application can also run remotely, and coupled with video conferencing, enable a remote technician or BCBA or therapist to be able to conduct a VR session to teach a skill. From the child’s perspective or the learner’s perspective, they’re getting the same experience that they would in person because they have the headset on, they’re seeing the same experience. The coach, or the supervising adult is able to use the coaching application just like they would if they were in person. And then really the only difference is that they rely on video conferencing to be able to physically see the learner and be able to communicate with them and provide that live coaching.

That’s great. That makes it so much more accessible, especially for folks who traveling to a therapy session, I know can present its own kind of barrier. So that’s so nice, because I know at least in some of the programs kind of around here, not so much in RAT, but sometimes just getting folks to therapy sessions in any kind of regular timeframe is very, very challenging. So that gives a great option for folks to be able to continue treatment and continue just getting folks to where they need to be, and practicing and using those skills without making them actually come to an office every single time. So very cool. I love that it does kind of have that option.

Vijay, this doesn’t sound like something that’s completely done. It sounds like something that’s going to continue to grow, and you’re going to continue to have more and more really cool things for folks to learn and master and stuff like that, so can you tell us kind of what’s on the horizon? Are you working on anything or excited for, say, anything coming down the pipeline?

Yeah, I mean the list is a mile long because I think once therapists, teachers and parents understand the power of virtual reality to teach skills, that they start to think about the areas where traditional therapy techniques aren’t working as well as we’d like, that there’s certain skills that are hard to teach. So we have a lot of things coming up soon. Doctors and dentist offices is one area that we’re excited to build content to help prepare neurodiverse learners for experiencing those situations and being better prepared. We are also looking at a lot of the in-the-community scenarios beyond that, so just a lot of different things on the way. 95% of the catalog is really driven by our customers using what we have and then telling us what they need above and beyond that. And that’s been a really great way to make sure that we’re building something that’s very useful.

Oh, for sure. And I feel like especially with this kind of thing, I mean, what’s the use in having any training or skills building program on there if people aren’t going to use it? So yeah, taking the consumer feedback and the individual’s feedback on what they really need sounds like the best way to do it, and it sounds like what you guys are really doing.

So we know that for a product like Floreo to be practical for therapists in areas such as ABA or speech therapy, that the product needs to not only be seen as effective from a research standpoint, and clinically accepted as something that can be beneficial in the care program for the child, but also needs to have the nuts and bolts of reimbursement codes, so that it can fit into the economic ecosystem for these clinic practices. So that’s an area that we, as a company, have had to focus on because it’s a very important and critical part of the whole puzzle.

And so we were really proud in 2022, we applied for with the American Medical Association and received a CPT-III tracking code for virtual reality to be added to the practice expense for mediated therapies that range from occupational speech, physical to include also ABA. And so, that code went into effect last year in 2023, and it’s available to be used, and we have some large providers now starting to utilize the code, and discuss with payers its acceptance.

Meanwhile, the next phase of our strategy was that we also need to develop the right regulatory approvals to help support all of the reimbursement scenarios we need. And so, we started down the path last year on applying for FDA approval as a medical device. And so, the first step was we ran a study with a care organization called Cordica in Southern California. And using that data primarily, we were able to apply for and receive something called FDA Breakthrough Device Designation. It’s a designation for novel technologies that treat chronic conditions that are in the public interest for new innovation. And so, the FDA has signaled that they looked at our research, they gave us this Breakthrough Device Designation, they also admitted us into a new program to help us with working out reimbursement scenarios. And we’re now preparing to execute what’s called a pivotal trial in April. That trial will be producing the data that we intend to then use to apply for an FDA market authorization, so that we have a label and the ability to say that we’re a therapy that treats a medical condition.

Oh, that’s awesome. That’s going to open up access to so many more individuals, as well as maybe acceptance for some providers to at least kind of try and see how it can help their folks. So that is awesome. But if our listeners want to find out more, what’s a good way for them to do that?

Yeah, we’re standing by at Floreo F-L-O-R-E-O VR.com. And so we have ways for parents, educators and therapists, healthcare providers to talk to us and onboard onto using Floreo. We also have some special programs for school districts to leverage the Meta Quest headset platform. We have a generous donation from Meta that allows us to donate hardware in school scenarios. For families, we have Medicaid waiver reimbursements now enabled in several states, and those have come from parents and families advocating directly with their service coordinators. Floreo can often be qualifying under assisted technology budgets that are self-directed funds that families have. But our main business today is that we’re working with about 80 healthcare provider customers and about 50 plus school districts and growing. And so, we’re excited to hear from all of you.

Awesome. We will put a link down there in the show notes so folks can kind of check it out, find out more, reach out and learn a little bit more as they might need. Well, Vijay, thank you so much for coming on the show today, for telling us about Floreo and really just the cool things. I don’t know, ever since virtual reality started catching on a little bit, this is the way I’ve hoped I would see it kind go is just kind of into that training, that real world, that real life scenario where, especially for individuals, if there might be behaviors or other kind of things where they can be in that controlled environment, not just mean for their own comfort and maybe to get rid of some anxiety, but also for safety and everything else, and just to be able to, I don’t know, control the uncontrollable and really be able to help folks get to where they need to be; more independent. So I love what you guys are doing, and thank you so much for coming on and telling us all about it.

Thank you so much for the opportunity. I appreciate it.

Do you have a question about assistive technology? Do you have a suggestion for someone we should interview on Assistive Technology Update? If so, call our listener line at 317-721-7124. Send us an email at tech@eastersealscrossroads.org or shoot us a note on Twitter at INDATA Project. Our captions and transcripts for the show are sponsored by the Indiana Telephone Relay Access Corporation or INTRAC. You can find out more about INTRACT at relayindiana.com. A special thanks to Nicole Prieto for scheduling our amazing guests and making a mess of my schedule. Today’s show was produced, edited, hosted, and fraught over by yours truly. The opinions expressed by our guests are their own and may or may not reflect Those of the INDATA Project, Easter Seals Crossroads are supporting partners or this host.

This was your Assistive Technology Update. And I’m Josh Anderson with the INDATA Project at Easter Seals Crossroads in beautiful Indianapolis, Indiana. We look forward to seeing you next time, bye-bye.