HaptImage LLC has developed a novel haptic device that converts digital graphics into real-time physical sensations. Here, inventor Ting Zhang works with a HaptImage prototype to explore the blood smear pictured on screen. Credit: Purdue Research Foundation image/Oren Darling David Schwarte, an assistive technology specialist on the Innovative Learning Team at Purdue University, knows what it is like to learn in a classroom when challenged with a visual impairment.
Schwarte, who has a visual impairment, says it is nearly impossible to understand what a professor is talking about when teaching from a PowerPoint on a large screen. To understand what is happening, the student has to depend on either an audio description or a 3-D-printed mockup of the image on screen.
Now a new device designed by HaptImage LLC, a Purdue-affiliated startup, is assisting students of all ages with visual impairments to "see" and learn about what is on the screen.
"There are many areas in science, engineering and technology where the end-user needs to visualize a bunch of different graphics," Schwarte said when he used the HaptImage device. "This system has the advantage of helping people visualize the graphics more instantly. HaptImage's technology is important because students can come into a very technical field and still be able to succeed."
These students are not alone. According to the World Health Organization, there are an estimated 1.3 billion people living with some form of visual impairment across the globe. HaptImage has a better alternative: an image-accessing system with novel haptic capabilities.
Ting Zhang, a doctoral student in the School of Industrial Engineering, created the system with her co-advisors, Juan Wachs, a professor in the School of Industrial Engineering and Bradley Duerstock, a professor in the School of Industrial Engineering and the Weldon School of Biomedical Engineering.
Credit: Purdue University She was inspired by a colleague who voiced difficulty in seeing blood smear slides during lab. He often needed a partner and didn't feel independent.
Zhang co-founded HaptImage with Shruthi Suresh, a doctoral student in Weldon School of Biomedical Engineering, in order to commercialize the technology which helps individuals with blindness or visual impairments study images related to science, technology, engineering and mathematic topics in real time.
Already, the startup has received $47,500 in entrepreneurial investments through its involvement with the Purdue Foundry and participated in the Fall 2017 National Science Foundation I-Corps cohort.
"There is no instantaneous method of helping a student with blindness understand an image," Suresh said. "In STEM, so much information can come from an image. When you don't have access to that digital image, or a viable alternative, you feel discouraged to pursue a career in that STEM-related field."
The HaptImage device uses a novel algorithm to convert digital images into physical sensations that mimic the would-be surface of the depicted object. Users hold a pen-like joystick, and as they move it across the allotted space, the joystick creates vibrations and resistance corresponding to the digital object's shape and texture.
"We, as sighted people, take images for granted and don't realize how valuable accessible visual alternatives are for someone with impaired sight. I think having that access is valuable," Zhang said.
HaptImage co-founders Ting Zhang (left) and Shruthi Suresh (right) use their system to “touch” and interact with digital blood cells on screen. The HaptImage device translates digital images into real-time haptic feedback using a novel algorithm developed by Zhang. Credit: Purdue Research Foundation image/Oren Darling To help students with visual impairments, instructors commonly use tactile graphics or 3-D printouts as stand-ins for digital images, but this process can be time-consuming, expensive and limiting for the students. However, the HaptImage device streamlines this process, so that students can interact in real time and directly with an image.
"The device combines tactile feedback through resistance with friction and sounds to give a person the idea of what the image 'looks' like," Suresh said. "They can feel the height and shape of the image, and they feel where the shape changes. That comes across a lot easier for someone who is blind or visually impaired."
The technology gives students with visual impairments the ability to "touch" a digital object due to its sophisticated haptic responses and makes these images instantly accessible. The founders hope this software can encourage more people living with visual impairments to pursue STEM-related careers.
"The algorithm translates an image into different intensities of vibration, different pitches or amplitudes of sound and haptic feedback, which allows you to feel that shape," Zhang said. "It's adding a whole other dimension of understanding for these students."
Currently, the startup led by Duerstock, is preparing to launch its device through a tiered subscription in the fall. HaptImage is currently developing a portable prototype that will allow individual students to interact with digital images on their smart devices.
Explore further: Assistive tech for people with visual impairments to identify scientific images on a computer screen