IS4800 – Empirical Research Methods in Information Science

Grades will be based on the following:

• Quizzes (10%).
• Class participation (10%), including in-class presentations.
• Individual homework (20% divided equally among assignments).
• Midterm exam (10%).
• Final exam (10%).
• Individual Term Project (40%, divided 25% proposal, 25% IRB application, 50% final writeup).

Homework alternatives (deadlines remain the same):

• Individual Homework #2 – Research Plan Review & Library Research: Draft Proposal Section B (Background & Significance) related to your area of interest (2-3 pages).
• Individual Homework #3 – Ethics in Dealing with Human Subjects: Draft the Human Subjects and Data Safety and Monitoring Plan Sections related to your area of interest (2-3 pages). This will require you to make a lot of assumptions about the study you might do. Document these assumptions in a short paragraph.
• Individual Homework #4 – Ethnography: Draft a short study proposal (2-3 pages) describing a qualitative study related to your area of interest.
• Individual Homework #5 – Designing Research Models: Design 3 different research models for studies related to your area of interest. For each, include a box-and-arrow diagram and text describing the purpose of the study, the measures you plan to use, and what the results would be useful for. For any experimental studies, identify a possible confounding variable and how you would control it. 1-2 pages per design.
• Individual Homework #6 – Objective Measures & Descriptive Statistics in SPSS: just do the 4800 exercise.
• Individual Homework #7 – Usability Performance Test: just do the 4800 exercise, but better if you can relate it to your area of interest.
• Individual Homework #8 – Behavioral Measures : do the 4800 exercise in pairs.
• Individual Homework #9 – Designing a Composite Measure for a Questionnaire: do the 4800 exercise, but make the subject matter of the survey related to your area of interest.
• Individual Homework #10 – Chi^2 and Correlation Testing in SPSS: interpret the 4800 exercise as "find a meaningful excuse to use chi-square goodness of fit and pearson correlation stats on your survey data".
• Individual Homework #11 – Designing and Analyzing an Experiment: Your mongo project proposals and IRB applications are due at this time, so I'll take those in lieu of this homework.

Resources:

• Inside the NIH grant review process [video & supporting docs]
• NIH/NIAID - How to Write a Human Subjects Application
• NIH/NIAID - How to Write a Grant Application
• NIH/NIAID - Annotated Sample Grant Application
• NIH/NCI - Quick Guide for Grant Applications

Suggested papers for review:

• [Good example / Modeling] Paul M. Fitts (1954). The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, volume 47, number 6, June 1954, pp. 381-391. (Reprinted in Journal of Experimental Psychology: General, 121(3):262--269, 1992). [PDF]
• [Good example(?) / Modeling] Nielsen, J. and Landauer, T. K. 1993. A mathematical model of the finding of usability problems. In Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems (Amsterdam, The Netherlands, April 24 - 29, 1993). CHI '93. [PDF]
• [Good example(?), Chi-Sq & correlation] Measuring usability: are effectiveness, efficiency, and satisfaction really correlated? Proceedings of the SIGCHI conference on Human factors in computing systems [ACM]
• [Bad example(?), uses several unusual stats] Aesthetics and apparent usability: empirically assessing cultural and methodological issues [ACM]
• [Bad example] Cassell, J. & Thórisson, K. R. (1999). The Power of a Nod and a Glance: Envelope vs. Emotional Feedback in Animated Conversational Agents. International Journal of Applied Artificial Intelligence, 13(4-5): 519-538. [PDF]
• [Bad example] Tim Bickmore, Justine Cassell (in press) "Social Dialogue with Embodied Conversational Agents" In J. van Kuppevelt, L. Dybkjaer, and N. Bernsen (eds.), Natural, Intelligent and Effective Interaction with Multimodal Dialogue Systems. New York: Kluwer Academic. [PDF]
• [Good example, meta-analysis] Yee, N., Bailenson, J. N., Rickertsen, K. (2007). A meta-analysis of the impact of the inclusion and realism of human-like faces on user experiences in interfaces. Nominated for Best Paper Award in Proceedings of the Conference on Computer-Human Interaction (CHI). April 28 - May 3, California, USA [PDF]
• [Good example?, ANOVA, some advanced methods] Okita, S.Y., Bailenson, J., Schwartz, D. L. (2007). The mere belief of social interaction improves learning, In Proceedings of the Twenty-ninth Meeting of the Cognitive Science Society [PDF]
• [Good example, ANOVA] Rickenberg, R. and Reeves, B. 2000. The effects of animated characters on anxiety, task performance, and evaluations of user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (The Hague, The Netherlands, April 01 - 06, 2000). CHI '00. [ACM]

Suggested papers for Advanced Design discussions:

• Baron, R. and Kenny, D., The Moderator-Mediator Variable Distinction in Social Psychological Research: Conceptual, Strategic, and Statistical Considerations, Journal of Personality and Social Psychology 51, 6 (1986) 1173-1182 (Tim has)
• Rosenthal and DiMatteo, Meta-Analysis: Recent Developments in Quantitative Methods in Literature Reviews, Annual Review of Psychology 52 (2001) 59-82.
• Longitudinal Data Analysis:
  • Fitmaurice, Laird & Ware, "Applied Longitudinal Analysis" (Tim has book)
  • Zeger & Liang, "Longitudinal data anlaysis for discrete and continuos outcomes", Biometrics, 1986, 42:121-30.
  • Hardin & Hilbe, "Generalized estimating equations", Chapman & Hall, 2003.
  • Davis, "Program for regression analysis of repeated measures using generalized estimating equations", Comput Methods Programs Biomed, 1993, 40:15-31.
• Gottman & Roy, "Sequential analysis: a guide for behavioral researchers" (Tim has book)
• Kenny, Kashy & Cook, "Dyadic Data Analysis" (Tim has book)
• Causal Path Analysis aka Structural Equation Modeling
  • Bollen, 1989, "Structural equations with latent variables", Wiley
  • Bollen & Long, 1993, "Testing structural equation models", Sage
  • "Identification in Structural Equation Modeling" intro in Chapter 6 "Data Analysis in Social Psychology", "The Handbook of Social Psychology" (Tim has book)
• Multilevel data analysis
  • Intro in Chapter 6 "Data Analysis in Social Psychology", "The Handbook of Social Psychology" (Tim has book)
  • Hardin & Hilbe, "Generalized estimating equations", Chapman & Hall, 2003.
• Logistic regression(?)