My lab, with the support of the NSF, is launching a crowd-sourced study of Hurricane Sandy, so as to better understand how people react in emergencies. If you were affected by Hurricane Sandy and use an Android phone, I hope you will be willing to help out. This will take 10-15 minutes of your time. And if you weren’t, then I hope you can pass this post on to someone that was affected by Sandy.

How do people respond in large-scale emergency situations, like earthquakes and hurricanes? Understanding this should inform more effective responses to save lives and reduce hardships. Getting hard behavioral data in the moment and aftermath is difficult–because people have better things to do than to participate in a study. There is quite a bit of valuable research based on interviews after the fact, but such research necessarily relies on reconstructed memories of behavior.

There is another path–which is to study the data passively collected about people by the sociotechnical systems relied upon during emergencies. An outstanding example of this is the paper by Bagrow et al that examined behavior as captured by mobile phones during a set of emergencies. The power of this approach is that it offers hard behavioral data at massive scale. The shortcoming, however, is that it cannot contextualize (beyond geography) the data. Who, exactly, are people calling? Their spouses? Friends? What are they communicating–the need for help, reassurances that they are ok?

Here we are launching a study that sits between these two approaches. Essentially, we are asking people to load an app on their Android phones (iPhone users: sorry, but for now we could only afford to develop for one platform), and the app will ask about their situations during Hurricane Sandy, and look at their calling and texting behaviors, asking them about their relationships with those individuals. We will therefore get a precise record of behaviors before/during/after Hurricane Sandy, and contextualize within personalize circumstances and particular relationships.

My motivation here is scientific and personal. I think there is the possibility to do great science here that is potentially consequential for people’s lives, that can inform interventions that will help people. And, having grown up on Long Island, and spent the early part of my career Red Bank, New Jersey –near the shore (“shaw”)– I could see a lot of suffering occur among my friends and family in the aftermath, where there was very little I could do. But this study is at least something good that I can make out of a terrible thing.

We have posted more information about the study on our newly launched crowd-sourced science website, Volunteer Science, or you could go directly to the Google Play store.

I’ve said it here before: I’m not much of a gamer. My 9-year-old nephew gets exasperated every time he sets me up in front of the Wii and ultimately just takes the controller away from me so he can deal with both characters at once. But that doesn’t mean I didn’t get excited when I heard about the next Pop Up Open Lab Experience and Reception: Play + Innovate. Northeastern students and faculty members will be gathering in the Digital Media Commons at Snell Library on Monday afternoon from 4 p.m. to 6 p.m. to present 11 different game-related research projects. The event will feature interactive demos from the interdisciplinary research teams.

This week, I caught up with a couple of the teams whose research will be on display, just to get a taste of what we have in store for us. Here’s one thing I learned: games are getting smarter. When I was a kid, Duck Hunt and the Oregon Trail were fairly predictable. You knew you’d have to ford a river at some point, and eventually you could learn the pattern by which the ducks entered the screen. Well…some of you could. I couldn’t. I would get deeply frustrated because I was so terrible at the whole endeavor.

Russell Pensyl, professor in the Department of Art and Design, professor Magy Seif El-Nasr, who has joint appointments in the College of Arts, Media and Design and the College of Computer Science, and PhD candidate Bardia Aghageigi are developing a system that the child-me would have appreciated quite a bit (and actually, the current version of me probably would too).

A camera on the computer or the mobile device takes regularly-timed pictures of the player’s face. The pictures are compared to a huge database of photos of facial expressions that are associated with particular emotions. If you’re smiling and giddy, the computer will know it. Then it’ll dive into the back end of the game and take a look at what’s happening there. If you’re winning by a landslide, the game will adapt itself to be a little more difficult, making the game more challenging, and thus (hopefully) more rewarding. If you’re frowning, as I almost always am when I have a controller in my hand, it might make itself a little easier, tone down the number of ducks flying across the screen, that sort of thing.

This is a form of adaptive content generation. Seif El-Nasr is interested in the technique to help promote engagement with games for health and learning. “So while you’re playing, things can change to make it more engaging or get it to be more effective at a learning or health component,” she explained.

There are other ways a game can change to handle other sorts of challenges. Automatic content generation means a game is never static, each time you start it up, you’re faced with a different kind of task or a new puzzle. Instead of responding to a player’s experience, this is completely random. You would never be able to figure out the pattern of target entry if Duck Hunt incorporated ACG.

Assistant professors Casper Harteveld and Gillian Smith are combining ACG with community gaming in an attempt to promote interest and learning in a game called Gram’s House, which aims to promote computer science interest among middle school girls.

Grandma loves her house and doesn’t want to leave for an assisted living facility, so players of Gram’s House try to equip her home with assistive technologies that can help her live there for as long as possible. This kind of story is thought to be more engaging for a young girl than, say, one that asks you to shoot all the bad guys. It uses puzzle games to teach computer science concepts like mapping or the binary number system. But it’s currently a totally static game. Players can’t interact with one another and the puzzles are always the same.

Like many researchers, Smith and Harteveld are curious whether adding a community aspect and ACG will improve the outcomes of the game. They developed GrACE, or Gram’s House Automatic Content gEnerator, named for the famous computer scientist Grace Hopper, to find out. Northeastern second-year student Gregory Loden developed a puzzle game that asks players to identify the shortest path between important points in the home. The idea is that a robot will travel along this course as it helps Gram with her daily tasks. But it’s really teaching players about the concept of “minimum spanning time.”

These are just two of the projects that’ll be strutting their stuff on Monday. Another looks at how virtual improv can promote social intelligence (this one will be on full display, actors and all!), while others explore how games can help us deal with issues like sustainability and security.

There will be other adventures that I don’t have time to get into here. You’ll just have to come to the Digital Media Commons on Monday to see for yourself.

In late February, something happened to the Italian government that had never happened before: a hung parliament. After 75 percent of the population turned out to vote, it took two days to tally the results. Now, almost three weeks later, the center right and center left parties remain in a steadfast gridlock. A third party–the so-called 5 Star Movement–received only marginally fewer votes. Which wouldn’t be all that exciting, except that the 5 Star Movement is just a single guy, and that guy is a comedian–Beppe Grillo–who refuses to speak on television or radio, lest his message be skewed. Instead, he blogs and he speaks in the piazzas around Italy.

The hung parliament and the comedian contender mark an odd time for Italy, which will have it’s next election in less than a year, according to people who know about Italian politics (of which I’m not one). Things are changing online and off, and the old ways of doing things are clearly not holding up. That’s even true of the polling methods that were used to predict the outcome of the election in the weeks leading up to it.

In a great article on the Daily Beast, which gives an awesomely readable and accessible account of the ever-so-complicated Italian political climate, Tim Parks recaps the final counts for the four main candidates, Luigi Bersani, Mario Monti, Silvio Berlusconi, and Grillo, respectively:

So 30 percent to grim support of the old workplace; 10 percent to the gentleman upholding the international monetary system; 30 percent to the rich, old guy inviting us all to have a good time; and 25 percent to the wild man who wants to kick ass.

How did the pollsters fair? Not great. They gave 40 percent to Bersani (the old workplace), 20 percent to Monti (the gentleman), less than 20 percent to Berlusconi (the old rich guy), and 15 percent to comedian Beppe.

Northeastern professor Alessandro Vespignani was also watching the election closely, and he and his team at the Laboratory for Modeling Biological and Socio-technical Systems made a few predictions of their own. They got all of them right except for one: Berlusconi.

How did they do it? And why were they off for the old rich guy? Well, it all goes back to that old ways thing I said earlier. Calling people on the phone seems to no longer be the easiest or most efficient way to probe society’s sentiments. In collaboration with researchers at the Institute for Scientific Interchange Foundation in Italy, Vespignani’s team tracked voter’s intentions by what they said on Twitter, instead. And actually, according to one of the project coordinators, Northeastern researcher Nicola Perra, “We did not make predictions. We just studied the raw signal, and there are biases, in geography, age, et cetera.”

On their website, TweetPolitik (it’s all in Italian), the team demonstrates results from three different types of analyses, which they performed in the weeks leading up to the election.

First, they made activity maps using geo-localization data. These show where in the country people tweeting about each of the candidates reside. “Each dot in the maps is a tweet. The transparent circles represent how intense the signal was in each census area,” said Perra. So, the map of Twitter activity about Beppe Grillo looks like this:

Activity map displaying tweets mentioning 5 Start Movement candidate, Beppe Grillo. 53% of the tweets came from the northern region of the country. Image via TweetPolitik.

While the one for Berlusconi looks like this:

Activity map showing tweets for Berlusconi. Nearly 50% of these tweets came from the North. Image via TweetPolitik.

Conversation maps show the highways of conversation on Twitter for each party. These data came from hashtags associated with the various parties and represent people in different areas tweeting about the same thing:

Conversation map showing Twitter discussions using hashtags associated with Bersani’s party, the center left. Most conversations took place between residents of large metropolitan areas, usually in the North or South of the country. Image via TweetPolitik.

Finally, (and this one is my favorite), the monitoring stream graphic shows how various discussion topics evolved over time in a ten day period surrounding the election.  On February 19th around 7 o’clock, Beppe was a popular subject:

A snapshot of twitter activity by Italian voters in the days leading up to the election on February 24. Image via TweetPolitik.

All of this is just reporting the signal that they saw in the data. The “predictions” (which weren’t really predictions, but just signals) came from the global share of activity seen for each party. This stuff isn’t presented on the site.

“In summa, some numbers were really close to the final outcome, others were not,” said Perra. “The reasons of the discrepancies are biases. We could have corrected the signals considering age distribution, Twitter penetration in different areas, but we decided not to make it about prediction, at this round.”

But this still leaves the question of why Berlusconi evaded their glance, even if it wasn’t a prediction. That has to do with social phenomena, said Vespignani. Berlusconi promised to repeal an important but hefty tax put in place by the last president, Monti (who actually took over for Berlusconi when he was obviously flailing during the height of the economic crisis). Everyone knew this was a bad idea for the country, but it was also incredibly tantalizing for each individual taxpayer. So, no one in their right mind would claim to support Berlusconi, in person or on Twitter. But behind the safe shield of a voting booth? That’s another story.

Ever since she was a small child, Samantha Grimaldo has had to carry her voice with her.

Grimaldo was born with a rare disorder, Perisylvian syndrome, which means that though she’s physically capable in many ways, she’s never been able to speak. Instead, she’s used a device to speak. She types in what she wants to say, and the device says those words out loud. Her mother, Ruane Grimaldo, says that when Samantha was very young, the voice she used came in a heavy gray box.

“She used to have to carry this device around that was at least 4 or 5 pounds,” Ruane says, “and she was only, like, 70 pounds herself. The poor thing had to carry this back and forth to school every day on the bus.” It was miserable having to lug her voice around that way — a clunky box sitting on the seat next to her.

Today, fortunately, Samantha’s voice takes up much less space. She types into a special program on an iPhone or iPad, and a synthesized voice in the program says the words aloud. The voice, one of several types on the market, is called “Heather.” That’s a nice enough name — easygoing and accessible — but Grimaldo doesn’t like to use the voice if she can help it.

Her mother has noticed that when the family goes out to restaurants, Samantha prefers to write out her menu choices. Apparently, as she explains to her mother, this is because Samantha has some reservations about the voice itself — the cold metal sound of it.

“Because [it's] weird,” Samantha says of the mechanical voice — speaking in the voice itself.

It’s not just that the voice is artificial and disjointed. It sounds, Samantha says, “older.” Samantha is only 17, and the sound of the voice — deep, methodical, mature — doesn’t exactly align with her sense of herself. Like any teenager, she feels self-conscious about it.

“I don’t want [people to] hear,” she says.

The Voice For The Voiceless

If you don’t have a voice, who speaks for you? Today there are more than 60 different options for people who need to use synthetic voices to communicate, but for the majority of people who use them, there is a single answer to that question: “Perfect Paul.”

Rupal Patel, a speech scientist at Northeastern University, estimates that between 50 and 60 percent of the people who use synthetic voices use the same one — the Perfect Paul voice. If you have ever heard Stephen Hawking speak, or listened to the weather radio, you have heard the voice of Perfect Paul.

Perfect Paul is used so widely because some studies have shown that his voice is easiest to understand in a variety of situations, including classrooms and public outdoor spaces. Still, some in the community of people who rely on synthetic voices have found the Perfect Paul version frustrating — not because it’s a bad voice, but because it’s limiting.

In fact, it was through confronting the clear limits of Perfect Paul that speech scientist Patel came to the conclusion that people like Samantha Grimaldo needed new options.

It happened around 10 years ago when Patel was at a conference for the makers and users of synthetic voices.

“I was watching a demonstration of a new technology, and someone came up and said something in their synthesized voice, and then someone else came up,” Patel says.

Both spoke in the same voice — Perfect Paul’s. Then a third person arrived, and another.

“It was the same voice saying different things,” says Patel. “And sometimes they were saying the same phrase, but off by a few seconds … so it felt like it was this echo going on. It was just a strange thing.”

Standing there, in the middle of all these radically different people with the exact same voice, Patel had an idea: Isn’t there something we can do to make these voices more individuated?

So, around seven years ago, Patel started working to change synthetic voices. When a person speaks, two things are happening. First, the source of speech comes from the voice box, which vibrates to produce sound. Then, the mouth shapes those sounds into speech.

In many people who have speech disorders, it’s mainly the second part of the system that doesn’t work. “In people with speech disorders, the source is pretty preserved,” Patel says. “I thought, ‘That’s where the melody is — that’s where someone’s identity is, in terms of their vocal identity.’ ”

So Patel decided to capture the melody of a voice. She primarily works with kids, and so she asked kids with speech disorders who can still make some sounds to come into her lab and do something really simple. “We just need them to say a sustained sound, like ahhhhh,” she says.

Patel can take that sound, run it through a computer and find out all kinds of things about how that person would sound if that person could speak words. “We can determine their pitch, the loudness, the breathiness of their voice, the changes in clarity,” she says.

She then takes a recording of the voice of what she calls a “healthy donor” — for example, the voice of a child who is roughly the same age as the child she’s trying to help — and gets them to say a large number of words. So she ends up with samples of the sounds they produce when they talk. She then combines that voice with the pitch, breathiness and other characteristics of the child with the voice disorder.

Patel played me examples of two different voices she’s created. If you listen, you can clearly hear different pitch and clarity in the different voices.

These voices Patel can make are unique for each individual. Which brings us back to Samantha Grimaldo.

‘You Need A Voice’

When Patel was getting started, Samantha was one of the first kids with a voice disorder who came to her lab to give a voice sample. At the time, Patel wasn’t at the stage where she was actually constructing voices. But she’s since figured it out, and recently, she created a new voice using Samantha’s ahhhhh sample.

Last week, she gave the personalized voice to Ruane and Samantha so they could hear it. The voice was constructed from a sample taken when Samantha was much younger. For a current version of Samantha’s voice, you’d need to take a new sample. Still, it was the first time that Samantha and her mother had heard anything close to Samantha’s voice.

Ruane had listened earlier in the day, when Samantha was still at school, and was clearly deeply moved by the experience. It made her realize in a fresh way, she says, how difficult it had been for her to never hear her daughter’s voice.

“When I heard it, I thought, ‘Yeah! This could be it!’ ” Ruane says through tears. To her ear, the voice had a sweetly familiar quality. “My son — my son Nicholas — I could hear some of his voice in it,” she says.

And so, when Samantha got home from school that afternoon, they sat down together to listen. Samantha’s young voice, it turns out, is clear and light.

Ruane told me that when Samantha heard the voice, her eyes lit up and a smile broke out on her face. Both thought that the voice sounded happy.

Personalized voices like these aren’t yet available to everyone. Patel has figured out how to do it, but not how to make it work on all of the different electronic devices that people use to play a synthetic voice. But Ruane Grimaldo hopes that voices like these will be available one day, very soon.

“You need a voice,” she says. “You need a voice.”

Article originally appeared on NPR.com

Interactive health technologies are a hot topic these days. Between Nike’s FuelBand and mobile phone apps like LoseIt!, the world has come to realize that interactive computing has a lot to offer  the layperson in the way of managing her own health.

These new platforms were just starting to emerge when professor Andrea Parker began her career as computer programmer who happened to be interested in social activism. “It was very exciting, but I noticed that it sort of was being done with this implicit assumption that the opportunity to achieve a healthy lifestyle was equal for everyone,” she said.

Researchers were taking the “if we build it, they will come” approach without accounting for barriers that might be getting in the way for some underserved populations.

For instance, it’s not always just the lack of an app that prevents people from living healthfully. “In some neighborhoods parents don’t want their kids to go play outside because it’s not safe,” said Parker. “In some neighborhoods they’re not eating a balanced diet because the access to healthy foods is lower.”

As she began carving out her niche in the field, Parker realized she could use her skills to design programs that put not only the power of health in peoples’ hands, but also the power for change.

As a post-doctoral researcher at Georgia Tech, Parker developed tools for children to critically engage with advertising or for community members to inspire each other to eat more healthfully. Having joined the faculty at Northeastern’s College of Computer and Information Sciences this winter, Parker is reeling with ideas on how to apply the things she’s learned in new and bigger ways.

She has already begun to collaborate with Carmen Sceppa and Jessica Hoffman, both professors in the Bouvé College of Health Sciences. Sceppa and Hoffman are co-investigators on Healthy Kids, Healthy Futures (HKHF), a program aimed at preventing childhood obesity by supporting health promoting environments in the home, school, and community .

One component of HKHF is called Open Gym, and it provides a safe place for kids and their parents to engage in physical activity once a week. The idea is to encourage children to be active, but also for parents to model healthy behaviors. “But how do we encourage physical activity in that whole week in between when they come to Open Gym?” Parker asked.  As you might imagine, she thinks a solution lies in interactive technologies.

One idea she has is to give kids activity monitors to wear throughout the week which will wirelessly transmit data on how active they’ve been. The more active they’ve been, the more points they get toward unlocking games back at Open Gym.

This particular situation is uniquely complicated by the fact that there are two populations who respond to pretty different motivations, said Parker. The things that get an eight year old excited aren’t always the same things that get their parents revved. But one of the approaches that Parker has taken in previous studies is to engage the community in the development process itself. Involving users in the design of technologies can not only ensure that those technologies will have the components they are seeking, but users will also be more likely to want to engage with the technologies. “It can help provide a sense of ownership over the system,” said Parker.

“Her involvement in the Healthy Kids, Healthy Futures team is exciting to us,” said Sceppa. “Her expertise in information and communication technologies will be instrumental in creating long-lasting physical activity promotion opportunities for our families and the neighborhoods HKHF serves.”

A gen­er­a­tion ago, stu­dents would say they “grad­u­ated from col­lege,” but now they “grad­uate col­lege.” These tiny fluc­tu­a­tions in the way we use lan­guage are ubiq­ui­tous because “chil­dren don’t learn the lan­guage their par­ents actu­ally speak,” according to David Smith, an assis­tant pro­fessor in the Col­lege of Com­puter and Infor­ma­tion Sci­ence.

The dis­crep­an­cies don’t sig­nif­i­cantly impede our ability to under­stand our chil­dren and grand­chil­dren, he said, “but accu­mu­la­tion of small changes over long periods of time is enough to make our Eng­lish sound a lot dif­ferent from Shake­speare, Chaucer, or Beowulf.”

Backed by a Google Fac­ulty Research Award, Smith is cur­rently studying how lan­guages have changed over the last sev­eral hun­dred years. But he’s doing it in a way only recently made pos­sible through tech­no­log­ical devel­op­ments in the dig­ital human­i­ties and nat­ural lan­guage pro­cessing. In the last few decades, libraries have been working to dig­i­tize lit­er­a­ture. Now that mil­lions of books are avail­able as search­able files, researchers are able to ask ques­tions that couldn’t be asked before.

Smith and his team will use cor­pora like the Penn Tree­bank, which includes the syn­tactic analyses of 30,000 sen­tences from The Wall Street Journal, to build sta­tis­tical models that auto­mat­i­cally detect the syntax of a sen­tence in a dig­i­tized book.

The main chal­lenge will be building models that work across a diverse range of texts over the last sev­eral hun­dred years, including news­pa­pers, blogs, and tele­phone con­ver­sa­tions. “The sta­tis­tical models pre­dict which words are con­nected to other words in a sen­tence,” Smith explained. “The problem is that over 500 years, pre­cisely because of the very phe­nom­enon we’re trying to model, words’ pat­terns of attach­ment change.”

Once the researchers have a com­pu­ta­tional pro­gram in place that doesn’t require human super­vi­sion, they will be able to visu­alize the evo­lu­tion of lan­guage. It will also have a far-​​reaching impact on cul­tural and his­tor­ical analyses, Smith said. “If we have a better model for lan­guage changes, we can recon­struct lan­guages that don’t exist any­more,” he said. Fur­ther, if we under­stand how lan­guages influ­ence each other through his­tory, we might get a better under­standing of how cul­tures connect.

Smith’s research is pri­marily focused on com­pu­ta­tional lin­guis­tics, “but texts can be evi­dence for lots of things in the human­i­ties,” he explained. “Not just lan­guage itself, but what people talk about with lan­guage.” His work, he said, can reveal what aspects of a cul­ture people find inter­esting or how texts are evi­dence for com­mu­ni­ca­tion, trans­porta­tion, and social net­works that are oth­er­wise not observable.

In a recently published security brief, Professor Engin Kirda of Northeastern University along with experts from RSA and Booz Allen Hamilton set out the components for a Big Data-oriented security management system.

Read more here

Stephen Hawking and a 9-​​year-​​old girl with a speech dis­order most likely use the same syn­thetic voice. It’s called Per­fect Paul and it’s easy to under­stand, espe­cially in acousti­cally chaotic envi­ron­ments like class­rooms full of chil­dren. While new, more natural-​​sounding voices are avail­able, Per­fect Paul remains the most oft-​​used syn­thetic voice in the com­mu­nity of dis­or­dered speakers.

But Per­fect Paul con­veys none of the per­son­ality inherent in vocal iden­tity, explains Rupal Patel, an asso­ciate pro­fessor of com­puter sci­ence and speech lan­guage pathology and audi­ology.

“What we’re trying to do is improve the quality,” she said, “but also increase the per­son­al­iza­tion of those voices, by not just making it a little kid’s voice, but making it that little kid’s voice.”

Backed by a grant from the National Sci­ence Foun­da­tion, Patel and her research team are devel­oping ways to create per­son­al­ized syn­thetic voices that resemble users’ vocal iden­ti­ties while remaining as under­stand­able as those of the healthy donors.

These voices have already elicited great responses from par­ents; one said, “If [my son] had been able to talk, this is what he would sound like.” How­ever, the early ver­sion of VocaliD used a difficult-​​to-​​scale  approach that is not easily repro­ducible. Patel said, “We’d like to be able to allow users to create new voices as they mature in the same way a nat­ural voice evolves.”

With the sup­port of another grant from the National Sci­ence Foun­da­tion, her team is cur­rently adding phys­i­o­log­ical infor­ma­tion on top of the acoustics.  “When you hear speech, it’s a com­bi­na­tion of your source and your filter,” Patel said. The source, she explained, derives from the voice box in the larynx whereas the filter is deter­mined by the shape and length of the vocal tract.

Vocal characteristics—such as pitch, breath­i­ness, and loudness—all emerge from the vocal folds in the larynx and give rise to vocal iden­tity. Mod­u­lating those fea­tures by changing the shape of our mouths and moving our tongues gives rise to dis­tinct vowel and con­so­nant sounds, which, Patel said, are typ­i­cally impaired in dis­or­dered speech.

Using data from a set of sen­sors placed on par­tic­i­pants’ tongues and mouths, the researchers will deter­mine the most effi­cient way to approx­i­mate the phys­ical aspects of the dis­or­dered speaker’s vocal tract. They can then add this infor­ma­tion into the voice-​​synthesis soft­ware to create voices that will grow and change as the users mature.

The aca­d­emic com­mu­nity has long accepted the source-​​filter theory of speech, but more work needs to be done in order to under­stand it, according to Patel, espe­cially as researchers develop more advanced speech tech­nolo­gies for secu­rity and other applications.

Patel’s work in par­tic­ular also aims to inform basic research ques­tions such as, “How much do both the source and filter con­tribute to the iden­tity of a speaker’s output?”

Patel’s soft­ware is com­pat­ible across assis­tive tech­nology plat­forms, including main­stream touch-​​pad devices, a fea­ture she hopes will increase its adop­tion within the com­mu­nity. Patel spec­u­lates that assis­tive com­mu­ni­ca­tion devices will even­tu­ally appeal to healthy people as a new way of learning, com­mu­ni­cating, and interacting.

“The iPad rev­o­lu­tion is helping to break down bar­riers and increasing the emphasis on user inter­face issues,” said Patel, who has been working to improve assis­tive com­mu­ni­ca­tion tech­nolo­gies for more than 16 years. “Lots of kids, both healthy and impaired, are using screens to interact now.”