Amy Sliva is a pioneer in the emerging field of security informatics, taking a Big Data approach to mining the labyrinth of terrorism’s contextual markers to predict when and where violence might erupt.

It’s a method suited to the times, says Sliva, because the information stream related to violence and terrorism has never been more abundant. From smartphone activity to broadcast media reports, we have at our fingertips the perfect storm of indicators when large-scale violence is brewing. The question is, how do we sort and make sense of all the clues we have at our disposal?

By using many of the same principles that experts in bioinformatics use to map and predict disease, Sliva and her colleagues are building artificial intelligence models to analyze and forecast potential threats. Such predictions could help officials make smarter security policy, prevent bloodshed, and save lives.

Among researchers focused on cleaning up the world’s black market of Internet insecurity, Robertson is a leader—in large part because he has learned to think like a cybercriminal.

This highly sophisticated set of hackers, members of a global Internet mafia, sit quietly behind backlit screens, coding their way into our sensitive data. From credit card numbers to computing power, nothing is off limits. So Robertson spends his days studying their malicious software: how it is constructed, how it works, and how it behaves. With that knowledge, he can create more robust security tools and safer systems.

For instance, Robertson uses machine-learning techniques to develop security programs that recognize the normal behavior of users and other programs. Then, when a threat presents itself and demonstrates anomalous behavior, the security program can automatically intervene to stop it. Robertson’s goal is not to catch the criminals, but rather to fatally cripple their ability to operate.

Healthy adults with fully developed vocal systems convey more information by producing speech and changing the melody of their voice.

But children and adults with severe speech-motor disorders tend to rely more heavily on melodic cues, such as volume and duration. Patel uses her understanding of speech melody to create computational tools that can dramatically improve a disordered speaker’s ability to interact with the world.

In one project, Patel overlays meolodic fluctuations from a disordered speaker’s voice with a sentence spoken by a healthy donor of the same demographic. By merging the two signals, she creates a novel synthetic voice that conveys the user’s personal identity.

For children learning how to read, Patel also develops digital tools with visual cues—such as a rising and falling line—that signal pitch changes. Research suggests that by understanding the melody of speech earlier, children may achieve greater reading comprehension.

Timothy Bickmore is using technology to help patients manage their own healthcare in a way no one else has. Meet Tanya, an avatar or “relational agent” in Bickmore’s phrase, who can serve as a nurse and personal health advocate.

By studying the behavior of real nurses and then turning his observations into complex computational algorithms, Bickmore is able to create avatars that show empathy and converse naturally with patients. They can access a patient’s medical records and provide information about drug treatments.

Avatars have unlimited time to walk patients through often confusing postclinic procedures, such as when to take their medication and how to dress a wound—abilities relevant to the critical issue of hospital readmissions.

In fact, a majority of patients involved in early clinical trials—particularly those with limited health and computer literacy—reported feeling more at ease interacting with avatars like Tanya than with live nurses.

Wearable devices: Matthew Goodwin is using sensors, such as the device shown here on his wrist, to accurately monitor anxiety and repetitive behaviors in children with autism.

Here’s a scenario that Matthew Goodwin is all too familiar with. A child with autism is sitting at a desk, seemingly checked out, staring into space. A teacher asks the child to get back to work, and the child stands up, flips over the desk and runs out of room.

“One second he’s fine and the next he’s having a tantrum,” says Goodwin, assistant professor of health sciences at Northeastern University in Boston.

But looks can be deceiving: Goodwin has shown that some children sitting and looking calm may actually be deeply anxious, with a pulse racing at 120 beats per minute. And the child’s apparent ‘spacing out’ may be an attempt to self-regulate his physiology.

“If I knew the child’s internal state, I wouldn’t place a demand on that kid. I might encourage him to relax or take a walk,” says Goodwin. “I would adjust my interaction style to calm him back down.”

The problem is that many children on the more severe end of the autism spectrum are nonverbal, and even those who can talk often have difficulty identifying and expressing how they feel. Goodwin is trying to develop alternative ways to measure these children’s internal states and in turn help teachers and parents modulate their interactions with them.

Goodwin is tackling this task with myriad monitors — from ceiling cameras and microphones to wearable sensors that track heart rate, temperature and sweat — and computer algorithms. Together, these may be able to determine when a child is stressed and what triggered the episode, and to evaluate the most effective strategy for making him feel better.

Similar tools could be used to assess treatments as well — automated monitoring may provide a way to more quantitatively measure changes in hyperactivity, for example, and even irritability and aggression, which are typically measured by short questionnaires.

Researchers who have worked with Goodwin uniformly comment on his unique ability to think about how to apply technology to autism care.

“Many people are experts in the autonomic nervous system, signal processing and hardware design, but he is able to bring these roles together and think about how we can develop methodologies that can ultimately impact care,” says James Rehg, professor of interactive computing at the Georgia Institute of Technology and a collaborator.

Early entry:

Goodwin began working with children who have autism early in his career, volunteering at a school for children with autism for 20 hours a week during a year of college spent at Oxford University in the U.K.

“He’s an experimental psychologist but also really tuned in to the kids — I’ve been in this field for 30 years, and there are not a lot of people like that,” says Terry Hamlin, chief of staff at the Center for Discovery in Harris, New York, a residence facility on a farm in the Catskills for people with autism and other disabilities. “He also makes wonderful connections and brings people together.”

The children at the Oxford center had challenges making eye contact and with joint attention, classical features of autism, Goodwin recalls. “But after showing up repeatedly and just spending time together, they would start to look at me and talk to me and show some empathy toward me,” he says. “Around then, I started reading the literature, which says these kids have no theory of mind, but that didn’t match the behavior I saw.”

Goodwin returned to the U.S. for college in 1995 and began interning at the Groden Center in Rhode Island, a day and residential program that serves profoundly impaired children with autism.

He noticed that stress and anxiety often aggravated the children’s behavioral problems. The clinical staff would try to calm the children down using a variety of methods, such as deep breathing or cognitive exercises. Goodwin says he wanted to understand what triggered the anxiety in the first place, and how effective the different methods were.

“That requires some measure of how stressed or non-stressed a person is,” says Goodwin. But if children can’t identify or communicate how they feel, how can a scientist adequately measure it? Or efficiently treat it? “Most stress research is based on surveys or direct behavioral observation, and herein lies the problem,” Goodwin says.

Flapping hands:

Fortunately for Goodwin, two technology trends were then beginning to be incorporated into the study of human health.

The first was wearable computing — sensors on the body or in clothing or accessories that can measure an individual’s biology or behavior. These are especially advantageous for studying children with autism, who often have sensory and movement issues that make traditional monitoring technologies unsuitable. They can be also used in real-world settings, such as the home or classroom, and can monitor a child for hours, days or weeks, rather than in a limited lab session.

Magic wristband: The Affectiva sensor tracks stress and other measures in children with autism, wirelessly transmitting the data to a computer.

The second trend was ubiquitous computing, in which sensors built into spaces, such as classrooms, record what’s going in the environment.

One of Goodwin’s first targets was hand flapping, a repetitive behavior seen in 70 percent of children with autism. “We don’t know why kids do this. We don’t know if it’s stimulatory or self-soothing,” says Goodwin. “It’s certainly socially stigmatizing.”

Hand flapping and other repetitive behaviors are a hot-button issue among families, educators and clinicians. Some education programs try to stop children from engaging in these behaviors, but that can make the child agitated or aggressive.

Goodwin is passionate about trying to understand these behaviors. “We ignore them, restrain them or medicate them,” says Goodwin. “But before we decide what to do about it, let’s try to decide why they do it.”

Preliminary research suggests that people with autism engage in repetitive behaviors for a variety of reasons — sometimes to calm down, sometimes to excite themselves. Hand flapping may even act as form of communication, showing happiness when they get something they want or frustration when they can’t get out of a situation they don’t like.

“If this is how they communicate, regulate stress and sensation, and feel their body, the last thing I want to do is stop them from doing it or medicate them,” says Goodwin.

Most studies measuring restricted and repetitive behaviors in autism use either parent report or direct observation, both of which can be unreliable. Two observers rating repetitive behaviors in real time agree only a third of the time, says Goodwin, largely because the behaviors can start and stop so quickly. Video recording is more accurate but is slow and expensive.

For his doctoral dissertation, Goodwin analyzed data from children with autism who wore three accelerometers — small devices that detect movement — one on each wrist and one around the waist. He created algorithms that, after a short training period, can automatically detect when a child is flapping or rocking, and found that the three devices together have an accuracy of 90 percent¹.

Complex sensors:

Goodwin’s latest work incorporates more complex sensors, which can track skin conductance — an indirect measure of the autonomic nervous system — as well as heart rate, movement and body temperature.

Goodwin strapped one such device around my wrist when I visited his lab, and we watched as it conveyed a stream of data to a laptop. A set of lines on the screen rose and fell throughout the conversation as my attention focused or wavered.

One of the biggest challenges of the project is to figure out how to interpret the sensor’s signals. Unlike, say, an electrocardiogram, which records electrical signals from the heart, there is no standard pattern for skin conductance.

Goodwin and his colleagues are analyzing data collected from ten children with autism wearing accelerometers and heart rate monitors in a classroom during a variety of tasks and emotional states.

Their goal is to determine whether repetitive behaviors are triggered by particular activities or internal states. The answer is unlikely to be simple: According to their preliminary findings, repetitive movements appear to be linked to physiology in some children but not in others.

Goodwin is also part of a five-year, $10 million project, funded by the National Science Foundation, to create automated tracking technology to help diagnose people with autism and track the outcome of therapies. The project involves bringing together a mix of sophisticated technologies, including cameras, sensors and machine learning — computational techniques that learn from data — to solve clinical problems.

He is also working with Hamlin on a pilot project at the Center for Discovery, set in a classroom outfitted with ten ceiling cameras and two microphones. The children and staff all wear wireless monitors that record their heart rate, temperature and movement.

The researchers are creating algorithms to automatically identify problem behaviors, such as wandering off or self-injury, based on data from the sensors. They can also look at the physiological data leading up to a behavioral outburst, as well as the physiological consequences of the behavior.

Hamlin says they have been recording for about a year and that the computers are now able to automatically recognize different behaviors.

The next challenge will be figuring out what to do with the enormous volume of data being collected. “We have to get it into the hands of clinicians to figure out which measures are predictive,” Goodwin says. He is also setting up instruments at his own testing center at Northeastern.

Goodwin has also helped launch a new graduate program at Northeastern that bridges technology and medicine.

“I was trained as a behavioral scientist and got interested in computer science late in life,” says Goodwin. (At 36, late is a relative term.) “The idea of the program is to train the next generation simultaneously, so they will be able to do what we can’t.”

References:

1: Goodwin M.S. et al. J. Autism Dev. Disord. 41, 770-782 (2011) PubMed

Article originally appeared at the Simons Foundation Autism Research Initiative

Shay McDo­nough, a senior infor­ma­tion sci­ence major, spent her first two co-​​op cycles at the phar­ma­ceu­tical giant Novartis working as a pro­grammer, ana­lyst, and project man­ager. She honed her skills and received valu­able real-​​world work expe­ri­ence at a large firm. For her third co-​​op, she went in a dif­ferent direction—working for a startup. It’s an expe­ri­ence that opened her eyes to an arena that has since become her passion.

“The beauty of working at a startup is that, even as a co-​​op or an intern, there is so much to do that you have no other choice but to get involved in every­thing,” McDo­nough said. Her co-​​op posi­tion was at Boston-​​based EverTrue, which builds mobile net­working plat­forms and was the result of a new col­lab­o­ra­tion between North­eastern and the startup accel­er­ator Mass­Chal­lenge. The part­ner­ship is aimed at pairing stu­dents with star­tups for their co-​​op posi­tions. McDo­nough thrived in her role—even staying on part-​​time after her co-op—and is seeking that kind of envi­ron­ment after graduation.

“I was asked to do so many things at EverTrue,” she said. “I know I have a lot to give.”

A leader in the field of personal health informatics, Intille is taking a unique approach to promoting healthy behavior—adapting a device nearly everyone owns, the smartphone, to assist with the motivating.

Intille and his team are building on the smartphone’s capabilities to help us track data related to the choices we make affecting our health. They are creating tools to help us understand our choices—about eating, exercise, even socializing—and developing mathematical models that enable computers to synthesize and respond to the information almost immediately.

One goal: an app that serves as a personal health coach, capable of sending instantaneous messages that reinforce positive health behaviors—an encouraging voice message from a spouse, for example.

The results of Intille’s trials will shape fields like exercise science, sleep science, and weight loss and nutrition—issues affecting everyone, from teenagers fighting obesity to aging baby boomers.

A couple weeks ago Angela Herring wrote a story about some work related to the Boston Marathon bombings that network scientists in David Lazer’s lab are working on. They’re asking Android phone users to donate a little time as well as the data from the calls and texts they made in the hours following the attacks. Researchers do have access to the anonymized call logs from cellular phone use, but without a little context about who those calls were made between and why, those data don’t say much. So they’re asking people to tell them in a brief survey in an app available at the Google Play store.

The goal is to get a better sense of how people use their social networks during emergencies. Another way the team is looking at this question is through Twitter. Yu-Ru Lin, an assistant research professor on the team, created a great interactive Google map that shows all of the Tweets using fear-related words that came out of Boston on April 15, 2013. While people were apparently a little on edge all day — 26.2 miles will do that to you — there’s a very obvious spike at 2:49pm, when the first bomb went off. In the visualization below, you see the whole city light up with red dots, representing those fearful Tweets:

Here’s a static representation of the tweets, showing that clear spike right when the bombs go off:

If you’re interested in participating in Lazer’s Android project, you can learn more about the project on his website, VolunteerScience, which is a new platform his team developed to investigate these kinds of questions more readily. Also, it’s worth noting that the team will donate $3 to One Fund Boston for every person that participates.

One in 88 children is diagnosed with an autism spectrum disorder. It is more common than childhood cancer, AIDS, diabetes, and spina bifida combined. This creates a public health problem: There will always be more people with ASD than experts to assess and teachers to assist them.

Yet much of today’s research doesn’t have a direct impact on the people who are living with ASD or their caregivers. It’s primarily focused on what causes the disorder, and we’re a long way from understanding that.

Most current research focuses on a convenience sample of high-functioning children with the mildest form of ASD; children who have normal IQs and good verbal ability. This sample is “convenient” because they can go to a lab with an unfamiliar person for some undefined period of time and perform tasks they’ve never done before—all of which requires a lot of self-regulation.

But at as many as half of children on the autism spectrum are too severely impacted to comply with current research protocols. These are the children we understand the least and the ones we need to help the most.

So we’re taking the lab to them. I work with computer scientists and electrical engineers to make that happen—experts who create sensors that can be woven into clothes, embedded into accessories, or inserted into devices that can be carried or worn. The devices continuously record physical activity patterns and autonomic nervous system sensing—that is, how a body is responding biologically.

To interpret the data, we also need context: where the person is and what he or she is doing. So we also “instrument spaces” with video cameras, microphones, and radio-frequency identification tags.

By bringing these wearable and environmental technologies together, we get powerful information in natural settings—at home, at school, and in the community—about what is happening to an individual with more challenging forms of ASD. This helps us understand them better and identify more impactful treatments.

Listen to Professor Goodwin’s podcast at WAMC.

Sometimes without warning, one of the autistic students in a classroom at the Center for Discovery will lose control. He will scream and cry. Throw things. Bang his head against the wall.

The six adolescent boys in this Monticello, N.Y., classroom, some of the hardest-to-handle students in New York State, cannot explain what is upsetting them. Unable to talk, they seem to live in their own world.

Matthew Goodwin, an assistant professor at Northeastern University, is trying to better understand their world by carefully tracking the boys’ movements and their environment. He has the boys wear sensors on their ankles and wrists that measure arousal levels, while cameras mounted on the walls record activities in the classroom, with the goal of finding what triggers episodes in the boys.

This is one of the early projects in a new program at Northeastern University to develop personal health informatics: devices and apps to improve health.

“The goal is really to be observing what happens from a patient’s point of view,” said Stephen Intille, one of the program’s founding faculty members. “Where can we insert technology to make their experience better?”

The five core faculty members, including Goodwin and Intille, believe that technology can help people take greater control of their health while improving the delivery of care. The only way that’s possible, they argue, is if technology is designed with users in mind and is proved to be effective with rigorous research.

Successful technology, Intille­ said, needs to be easy to use, easy to interpret, and embedded in the environment, like Goodwin’s sensors. Just telling people how far they have run or how many calories are in their dinner will not be enough to change behavior.

“Most people aren’t motivated by data,” Intille said.

Many of the current projects at the Northeastern Personal Health Informatics program are aimed at giving individuals — particularly people with limited resources — more opportunities to take charge of their health.

Assistant professor Andrea Grimes Parker is launching a social media program that will allow residents in Roxbury who participate in a once-a-week gym program to share their exercise tips. She believes those tips will be more useful to participants because they are “locally grounded,” coming from people in the same community.

She tried a similar program in Atlanta and found “that small feedback loop — that ‘what I’m sharing has value’ — was very encouraging and empowering to the users.”

For example, some participants wear sensors to measure their activity levels and earn small rewards for getting extra exercise, she said.

Intille and colleague Timothy Bickmore are collaborating on a project to embed patient-focused technology in hospital rooms that would allow patients to track their pain, for example, answer basic questions for them, and remind them of their doctors’ names and specialties.

Bickmore envisions a “bedside presence with a range of sensors that can tell what’s going on in the room and what’s going on with the patient.”

In another project, a Northeastern student is developing technology to identify when a patient with a neurological disorder such as ALS subtly starts losing tongue control. This can cause swallowing problems and lead to pneumonia and hospitalization, so catching it early can be beneficial to both patients and the health care system.

“There is a huge cost-savings potential,” and it has obvious benefits for the patient, said Rupal Patel. He is an associate professor in the Department of Speech-Language Pathology & Audiology and another founding member of the personal health informatics program.

The innovation came from a student who did not know anything about neurological problems, Patel said, but who saw a problem he could solve with his expertise in computer science.

That is another unusual hallmark of the Northeastern program: It matches a wide range of specialists, such as computer scientists, language pathologists, behavioral experts, and game designers, for example, with people who understand health challenges.

“All of these perspectives working collectively in this space — that’s how we’re going to have potential to have a substantive impact on public health,” said Lisa A. Marsch, director of the Center for Technology and Behavioral Health at Dartmouth College, who is not involved with the Northeastern program.

Goodwin said he could never have designed his project for autistic students without such collaborations.

Theresa Hamlin, associate executive director at the Center for Discovery, said that until now, nothing has worked to keep these students calm enough to be able to participate in typical classrooms, or even to live with their families.

“The entire system has tried everything there is,” she said. “You name it; they’ve tried it.”

Goodwin said that tracking the students and their environment will allow their teachers and caregivers to see the world as they do and better understand what triggers their tantrums.

“Something is driving [their] changes,” Goodwin said. “Demands in the classroom? Shifting stress? Seizures? Gastrointestinal problems? We don’t know.”

Part of what Goodwin and other faculty members will teach Northeastern students is how to design studies like this to make sure their technology is effective. It doesn’t matter how cool a technology is, he said, if it doesn’t provide health benefits to the user.

This article originally appeared in The Boston Globe