Paper Requirement for Langxuan (James) Yin

August 11, 2010

In the 2 years and 11 months I have been at Northeastern University, I have co-authored 5 papers, including 2 journal articles, 2 conference papers and 1 workshop poster. Of these, I took a lead role on the most recent conference paper accepted by the Intelligent Virtual Agents 2010 conference and the poster paper presented at the CHI 2010 WISH workshop, and had significant contributions to another two papers, one of which was honored with the Best Paper Award at the IVA 2009 conference. Two of the papers I took a lead role on or had significant contributions to are listed below.
Conference Paper: Yin, L., Bickmore, T., Cortes, D. (2010) The Impact of Linguistic and Cultural Congruity on Persuasion by Conversational Agents Intelligent Virtual Agents conference 2010, to appear. pdf

Conference website: Intelligent Virtual Agents (IVA) 2010
Paper submitted as a full paper (acceptance rate 30%) but accepted as a short paper (66% overall acceptance rate for all papers, long and short).

Description:
We present an empirical study on the impact of linguistic and cultural tailoring of a conversational agent on its ability to change user attitudes. We designed two bilingual (English and Spanish) conversational agents to resemble members of two distinct cultures (Anglo-American and Latino) and conducted the study with participants from the two corresponding populations. Our results show that cultural tailoring and participants' personality traits have a significant interaction effect on the agent's persuasiveness and perceived trustworthiness.

Student contribution:
This paper has been accepted as a short paper at IVA2010. I came up with the initial idea of using cultural tailoring techniques on relational agents, took lead in the design and development of the empirical experiment in collaboration with a researcher who is an expert in the Latino culture at Harvard University. I also took charge of participant recruitment and supervised the study process. I was the "lead" (only) student author on this paper.

Advisor's statement about student contribution:
This paper reports a human subjects study that represents a significant amount of work - almost a year's effort in total. James performed almost all research, experimental design, development, and data analysis tasks himself and drafted the entire manuscript. There were some flaws identified by the reviewers, but overall this was a solid piece of work that should have been accepted in its original format (e.g., he used the most commonly used outcome measure for persuasion studies that two of the reviewers criticized). Given that James is now late meeting his paper requirement I recommended that he proceed with this short paper publication rather than withdrawing it and resubmitting to another venue.

Original full paper: pdf

Reviewer's comments:
Reviewer1: Interesting paper on a very important topic. The empirical study seems a bit exploratory and bears some methodological questions (number of participants, dependent variables to operationalize persuasion, preparation of verbal stimuli). Depending on the number of papers in the different categories, accept either as (strong) short paper or as (weak) full paper.
Reviewer2: One of the two independent variables "Need For Cognition" is not explained in this paper, therefore the results of the statistical analysis are not understandable. Since the main results are not understandable, I cannot recommend this paper. I also found several format problems. This paper is space-and-a-half instead of single-spaced, thus the amount of body text is much less than that of standard full papers. Another problem is that some references have a missing year number. Although the study itself may be worth presenting at the conference, this paper has to be rewritten before publication.
Reviewer3: - I think this is an interesting work for investigating influence of cultural and linguistic to human-agent interaction even though it's just at a preliminary stage. - The experimental setting, results and analysis seem technically sound.
Reviewer4: The paper set up a study to look at linguistic and cultural tailoring and how it impacts persuading a person to change their attitudes towards their health. Seems like a weak way to measure persuasion, ranking scores before and after the dialog. More like an increase in knowledge about the issues, rather than changing their minds, is this a short term effect or long term? Seemed like this was more a measure of the attitude towards the look and style of the agent delivering the message rather than the tailoring of the message
Conference Paper: Bickmore, T., Schulman, D., and Yin, L. (2009) Engagement vs. Deceit: Virtual Humans with Human Autobiographies Intelligent Virtual Agents conference 2009, Amsterdam. (Best Paper Award) pdf

Conference website: Intelligent Virtual Agents (IVA) 2009
Acceptance rate: 26%.

Description:
We discuss the ethical and practical issues involved in developing virtual humans that relate personal, fictitious, human autobiographical stories ("back stories") to their users. We describe a virtual human exercise counselor that interacts with users daily to promote exercise, and the integration of a dynamic social storytelling engine used to maintain user engagement with the agent and retention in the intervention. A longitudinal randomized controlled experiment tested user attitudes towards the agent when it presented the stories in first person (as its own history) compared to third person (as happening to humans that it knew). Participants in the first person condition reported enjoying their interactions with the agent significantly more and completed more conversations with the agent, compared to participants in the third person condition, while ratings of agent dishonesty were not significantly different between the groups.

Student contribution:
I developed a narrative generation engine which weaves together small pieces of story fragments with a story linking technique, which was employed in this empirical study to generate real-time long stories over the period of a month. Although the outcome measures of this study were focused on the comparison between user reactions to first- and third- person storytelling, the narrative generation engine was essential in making the user's conversations with the agent natural and nonrepetitive, and the idea underneath it was novel in the narrative generation field of research. I authored the portion of this paper regarding the technical details of the narrative generation engine and its related works.

Note: this paper was extended into the following journal article, although my contribution was the same as for the conference paper.
Journal Article: Bickmore, T, Schulman, D, Yin, L (to appear) Maintaining Engagement in Long-term Interventions with Relational Agents International Journal of Applied Artificial Intelligence special issue on Intelligent Virtual Agents, to appear. pdf

Advisor's statement about student contribution:
James developed one component of the system reported in this article (a story generation system), and wrote the portion of the paper describing it, including literature review and technical description. James led the development of this component over the duration of a semester, including collection of story fragments from a member of our research staff, and design and implementation of a system that weaves these together into unique conversational stories each time the system is run. The other student on the paper (Dan Schulman) developed the experimental framework used to evaluate James' storytelling component. In terms of sheer effort, Dan had the greater contribution, but James' contributions were more central to the topic of the paper.