In Touch
‘Wearables’ Shed Light on Brain, Behavior
Photo: Ernie Branson
From cell phones to wristwatches, technology is getting smarter and smaller. For one engineer, a project that began with a cumbersome wearable computer evolved into small sensors in glasses and hats that could interpret facial expressions. These innovations paved the way for even smaller sensors embedded in wristbands that measure emotional peaks. It’s life-changing technology for patients with autism, epilepsy and depression and has the potential to ameliorate the effects of other medical conditions.
“We’re now able to make outwardly observable something that previously was only inwardly happening,” said Dr. Rosalind Picard, founder and director of Affective Computing Research at the Massachusetts Institute of Technology Media Lab and faculty chair of MIT’s Wellbeing Initiative Mind+Hand+Heart. Picard—who spoke to a packed room at a recent NIMH Director’s Innovation Speaker Series lecture in the Neuroscience Center—co-founded two companies, Empatica, Inc. and Affectiva, Inc., which build the affect-sensing and wearable devices.
Picard creates technology that helps computers interpret human emotion, a growing area of research that she named “affective computing.” One of the signals used is electrodermal activity (EDA). Regions deep in the brain activate the sympathetic nervous system, which causes sudomotor innervation, which changes the electrodermal properties of the skin. These changes can be measured noninvasively by a sensor worn on the skin’s surface.
“I’ve got to find things I can measure that are objective, reliable and repeatable,” said Picard. Detecting furrowed brows, extreme stress, excitement and other emotional cues provides data that potentially can be applied to everything from health behavior changes to market research.
What’s more, the latest sensors can track EDA for hours, days, even weeks at a time, providing more than just snapshot data. “We now get all-day data,” said Picard. “You can see what’s normal for a person, if they walk around permanently with a larger EDA response on one side than on the other, or if they only become asymmetric when they have meetings with their boss.”
Interestingly, many patients show very high EDA during sleep, but the peaks usually end just before the person goes into REM sleep. One reason the EDA may be so high is because learning and memory formation occur during sleep, said Picard. Also, the amygdala and hippocampus—regions of the brain associated with emotion—are at work during sleep; stimulating these regions directly has been shown to elicit large EDA.
Photo: Credit Ernie Branson
Picard started out researching whether her sensors could help monitor stress levels in children with autism. Among her innovations are glasses and a handheld device with a circuit that mapped emotions to the brightness of an LED. She hoped the technology could give voice to people who had difficulty communicating.
In one young, autistic child, ankle sensors showed skin conductance peaks when the youngster was roused and during meltdowns. During one therapy session, the child’s EDA peaked while climbing on a swing. Picard said there is ambiguity in such a case: the EDA could peak from emotions as well as from the physical and cognitive (motor planning) effort of climbing onto the swing. However, the EDA decays rapidly once swinging happens; such a decay has also been observed with repetitive motion in autism that can be calming. Since unfamiliar places can cause extreme, needless anxiety in patients with autism, Picard wanted to build something that could measure a person’s physiology anywhere.
“This is the first time we built the sensors so they could leave the lab and measure autonomic activation in a particular sympathetic nervous system through the skin in a child who—if you brought her into the lab—might have a very different experience than if you measured her in her natural setting,” said Picard.
An unexpected breakthrough occurred when one of Picard’s MIT students borrowed a pair of wristband sensors over winter break to test on his young autistic brother, who couldn’t speak. When reviewing the data, Picard was stunned to see an enormous signal spike, much higher than she’d seen when testing the sensors on stressed students or even Boston drivers. She previously used only one wristband at a time, but the student had fortuitously put the bands on both of his brother’s wrists at the same time.
Curiously, the surge was on only one side of the child’s body; Picard wondered if the other wristband was broken. The signal then stabilized, leading to a clear sleep signature. When she questioned the student, who kept a careful diary, he said the timeframe was 20 minutes before his brother had a grand mal seizure.
Was it possible to have a huge nervous system surge that long before a seizure and on only one side? Soon, Picard got research approval to enroll patients in round-the-clock EEG, ECG and EDA monitoring. Of 16 grand mal seizures recorded during the study, all had significant skin conductance peaks with the event.
While the EDA peaks did not usually precede the seizures, they did show that it was now possible to achieve more accurate automated seizure detection.
Searching for an explanation as to why the event was measured only on one side, Picard found evidence in the literature that certain regions deep in the brain, particularly those in the temporal lobe, map to only one side of the body’s sudomotor nerve response. “There are a lot of brain-imaging studies that suggest the two sides are going to act differently,” she said.
Picard and her MIT colleagues continued their engineering and design work to refine their sensors. Newer versions are becoming more sensitive and accurate. Thanks to the efforts of spin-off company Empatica, they’ve since shrunk the technology down to a device called Embrace, a water-resistant watch that can run physiology algorithms and measure physical activity, sleep and temperature. Most importantly, it alerts others that the patient may need help.
“People are much more likely to survive the period of time after a seizure if someone is there,” said Picard, referring to a lesser known neurological disease, SUDEP—sudden unexpected death from epilepsy—which may be preventable through an alert system. SUDEP takes more lives in the U.S. every year than house fires or sudden infant death syndrome. After stroke, SUDEP is the second leading neurological cause of years of potential life lost, more than multiple sclerosis, Alzheimer’s, Parkinson’s, amyotrophic lateral sclerosis and meningitis/encephalitis. Yet most people have never heard of SUDEP.
With continued refinements, Picard intends for her research to help predict and prevent certain neurological conditions, from seizure to depression. It’s a long journey of learning to connect our neurological and behavioral wiring and of getting the research out of the lab and into production so more data can be collected and people can benefit.
“We’re just at the beginning,” said Picard. “It’s all going to help make it easier to forecast important health changes.”