AI glasses fill in gaps for patients with low vision

At Helen Keller Services in Brooklyn and Long Island, Ray-Ban Meta glasses are a prescribable low vision device. Subsequently, we have dispensed many of the smart glasses with the objective of enhancing independence and accessibility for our patients.

Notably, Meta glasses were not initially designed for individuals with low vision. They were curated for the average-sighted individual seeking hands-free social media content creation techniques. However, this device has become one of the most helpful tools we provide at our clinic for patients with low vision. The Meta glasses are not the only pair of smart glasses that could provide assistance - in this competitive market, glasses such as the Google Android XR have similar features.

Many text-to-voice reading technologies are available today, including several mobile apps, various sizes of closed-circuit televisions (CCTVs), and incredible technology (eg, the OrCam) that can bring written text to life for patients with low vision. The Meta glasses bring that experience to a whole new level. The Meta glasses have excellent OCR technology that can easily read many different types of written text, such as forms, restaurant menus, signs, and handwritten notes. The AI technology is not only able to read any provided text but also summarize it on cue as needed. The glasses are a much easier way of accessing written material than using a bulky CCTV with OCR or trying to line up the phone perfectly with the paper to capture the entire page. The ability to read any typed or handwritten text has been the most helpful for our patients with low vision. The convenience of the technology sitting on the face while accessing documents at work, reading mail at home, and analyzing personal finances has been revolutionary for patients with all levels of vision deficit.

The most advanced feature of the Meta glasses is, undoubtedly, their capacity to respond to inquiries about the user’s surroundings. The inquiries may be as broad as, “Hey, Meta. What is in front of me?” Through these broad inquiries, patients could determine the presence of a door, a person, or a sink. This technology assures patients with low vision of their ability to independently locate seating and navigate through open doors. Meta can also respond to specific questions, such as “Is the door before me open or closed?” or “Is there an unoccupied seat in front of me?” Patients shared intriguing anecdotes regarding the use of Meta glasses, including one individual who described a scenario involving a snack table. This individual wanted potato chips without requesting assistance, so he asked Meta, “Are there potato chips on this table?” The glasses then successfully identified the location of the barbecue potato chips on the table. To those with functional vision, these unique inquiries may appear insignificant, but for our patients with low vision seeking greater independence, this technology is truly life-changing.

Additionally, Meta can respond to inquiries about color, which is very helpful for patients who have never been able to discern color due to congenital conditions or for those who have lost a significant portion of their color vision due to progressive diseases. Simply asking, “What color is this folder?” or “What color is this shirt?” would be useful for people who wish to coordinate their clothing or place important documents in the correct folder at work or home.

Lastly, it is also possible to sync the glasses with the user’s contacts on their phone, enabling hands-free texting and phone calls. Patients with low vision who frequently have one hand engaged with their white cane and/or guiding dog may greatly benefit from this technology’s hands-free feature. Patients may find it challenging to pause in the middle of a busy street, locate their phone in their bag, and make the appropriate call, but with the Meta glasses, they could continue walking and use voice commands to call anybody they need. This would reduce incidents such as patients losing their phones or bumping into people after stopping in the middle of the road, as well as navigating detours to locate a safe place to stop and send a text message or make a call. In a similar context, the glasses can answer questions that other voice assistance technologies could answer, such as “How many cups are in 4 oz?” when cooking or baking at home. Patients with low vision already struggle with activities of daily living around the house, so accessing information that would help in activities such as cooking without relying on their limited vision or looking for a phone is a game changer.

To ensure privacy, the volume in the glasses can be adjusted. By saying, “Hey, Meta. Set volume to 50%,” a user can do this without having to open the smartphone app. Based on my own experience, it seems to provide privacy in a quiet room if the volume is set to 50% or less. In an outdoor environment with birds chirping and vehicles passing by, even a volume of 100% would ensure privacy, as the messages being read out loud would not be audible to those in the vicinity.

My experience with the Meta glasses has been one of immense hope for the patient. I’ve witnessed patients cry tears of joy as they consider all the possible ways it could help them achieve independence and expand their accessible world. Better yet, the integration of AI technology into smart glasses is only getting started. With software updates and eventual hardware updates, patients with low vision will be able to augment their experience of this world despite their limited vision. There is so much hope to be gained and shared, and I believe it is our responsibility as eye care professionals to stay current with technological advancements that could improve the lives of our patients.