The UW researchers initially tested PupilScreen with a 3-D printed box that controls the eye's exposure to light. They are now training their deep learning algorithms to produce similar results with the smartphone camera alone. Credit: Dennis Wise/University of Washington University of Washington researchers are developing the first smartphone app that is capable of objectively detecting concussion and other traumatic brain injuries in the field: on the sidelines of a sports game, on a battlefield or in the home of an elderly person prone to falls.
PupilScreen can detect changes in a pupil's response to light using a smartphone's video camera and deep learning tools—a type of artificial intelligence—that can quantify changes imperceptible to the human eye.
This pupillary light reflex has long been used to assess whether a patient has severe traumatic brain injury, and recent research finds it can be useful in detecting milder concussions—opening up an entirely new avenue for screening.
The team of UW computer scientists, electrical engineers and medical researchers has demonstrated that PupilScreen can be used to detect instances of significant traumatic brain injury. A broader clinical study this fall will put PupilScreen in the hands of coaches, emergency medical technicians, doctors and others to gather more data on which pupillary response characteristics are most helpful in determining ambiguous cases of concussion. The researchers hope to release a commercially available version of PupilScreen within two years.
"Having an objective measure that a coach or parent or anyone on the sidelines of a game could use to screen for concussion would truly be a game-changer," said Shwetak Patel, the Washington Research Foundation Endowed Professor of Computer Science & Engineering and of Electrical Engineering at the UW. "Right now the best screening protocols we have are still subjective, and a player who really wants to get back on the field can find ways to game the system."
As described in a paper to be presented Sept. 13 at Ubicomp 2017, PupilScreen can assess a patient's pupillary light reflex almost as well as a pupilometer, an expensive and rarely used machine found only in hospitals. It uses the smartphone's flash to stimulate the patient's eyes and the video camera to record a three-second video.
The video is processed using deep learning algorithms that can determine which pixels belong to the pupil in each video frame and measure the changes in pupil size across those frames. In a small pilot study that combined 48 results from patients with traumatic brain injury and from healthy people, clinicians were able to diagnose the brain injuries with almost perfect accuracy using the app's output alone.
In amateur sports today, even the best practices that coaches or parents use if an athlete is suspected of a concussion during a game—asking them where they are, to repeat a list of words, balancing, touching a finger to their nose—essentially consist of subjective assessment.
By contrast, PupilScreen aims to generate objective and clinically relevant data that anyone on the sidelines could use to determine whether a player should be further assessed for concussion or other brain injury.
The U.S. Centers for Disease Control and Prevention estimates about half of the 3.8 million concussions per year in the U.S. from recreational sports injuries alone still go undiagnosed, putting millions of young players and adults at risk for future head injury and permanent cognitive deficits.
Historically, there's been no surefire way to diagnose concussion—even in the emergency room, said co-author Dr. Lynn McGrath, a resident physician in UW Medicine's Department of Neurological Surgery. Doctors usually run tests to rule out worst cases like a brain bleed or skull fracture. After more serious head injuries are excluded, a diagnosis of concussion can be made.
Medical professionals have long used the pupillary light reflex—usually in the form of a penlight test where they shine a light into a patient's eyes—to assess severe forms of brain injury. But a growing body of medical research has recently found that more subtle changes in pupil response can be useful in detecting milder concussions.
"PupilScreen aims to fill that gap by giving us the first capability to measure an objective biomarker of concussion in the field," McGrath said. "After further testing, we think this device will empower everyone from Little League coaches to NFL doctors to emergency department physicians to rapidly detect and triage head injury."
PupilScreen combines a smartphone's flash and video camera with machine learning tools to track how the eye's pupil reacts to light. The app can currently distinguish between the pupillary light reflex of healthy people (shown above) and patients with severe traumatic brain injury. Additional studies this fall will help determine what characteristics are most useful in detecting milder concussions. Credit: Dennis Wise/University of Washington While the UW team initially tested PupilScreen with a 3-D printed box to control the eye's exposure to light, researchers are now training their machine learning neural network to produce similar results with the smartphone camera alone.
"The vision we're shooting for is having someone simply hold the phone up and use the flash. We want every parent, coach, caregiver or EMT who is concerned about a brain injury to be able to use it on the spot without needing extra hardware," said lead author Alex Mariakakis, a doctoral student in the Paul G. Allen School of Computer Science & Engineering.
One of the challenges in developing PupilScreen involved training the machine learning tools to distinguish between the eye's pupil and iris, which involved annotating roughly 4,000 images of eyes by hand. A computer has the advantage of being able to quantify subtle changes in the pupillary light reflex that the human eye cannot perceive.
"Instead of designing an algorithm to solve the specific problem of measuring pupil response, we moved this to a machine learning approach—collecting a lot of data and writing an algorithm that allowed the computer to learn for itself," said co-author Jacob Baudin, a UW medical student and doctoral student in physiology and biophysics.
The PupilScreen researchers are currently working to identify partners interested in conducting additional field studies of the app, which they expect to begin in October.
Explore further: Know the signs of concussion