Schedule for IS 4800

Please note that the schedule is subject to minor revision based upon the number of students in the class and their backgrounds and interests, to be assessed in the first few weeks of the course.

I# = individual assignment # (although some allow up to 2-person teams)
T# = Team assignment #

Optional reading:

Advanced study designs (B&A Ch 11)

Homework (Individual Assignments)

The following assignments are to be completed individually and emailed to the instructor (by noon on the due date unless otherwise specified). All homeworks should be emailed as a single PDF file, with relevant output (graphs, tabular output, statistics, but not commands) copied and pasted into the document, along with any other materials considered for grading (e.g., survey instrument). Please put "[Methods] Assignment [Assignment#] [Your name]" in the email subject line (e.g., "[Methods] Assignment I2 Stephen Intille").

I1: Individual Homework #1 (not graded - due next class)

Sign up for the Piazza group.

I2a: Individual Homework #2a - Ethnography (due 1/30)

First, read these two papers to get a better understanding of the role of ehnography in information science:

• Blowing in the Wind: Unanchored Patient Information Work during Cancer Care -CHI'12
• Unpacking Exam-Room Computing: Negotiating Computer-Use in Patient-Physician Interactions - CHI'12

Now, imagine that Northeastern has just hired you to make the first floor of the new ISEC building at Northeastern more useful and efficient. Pick a location where talking is acceptable and spend an hour people watching with a notebook and pencil. Put away all your electronic devices so you are not distracted. (If you are answering text messages or emails, then you are doing this wrong.) Spend some time watching the kinds of activities that people are engaged in, and think about how technology could help improve these activities. Pick one such activity to focus on and study. For your chosen activity watch several people perform the task and make detailed notes about the series of steps they go through, any objects ("artifacts") they use, whether they interact with other people and, if so, the step-by-step details of this interaction. Conduct two or more unstructured interviews with your subjects.

Some examples of activities (you can study one of these or, better yet, come up with your own):

• finding a meeting room
• ordering at the cafe
• navigating somewhere in the building
• having a meeting
• rendezvousing with friends

What to Email:  Your report should include a one-paragraph summary of why you picked your particular activity to focus on, followed by an overview of the activity, the kinds of people you observed engaging in it, and a description of any artifacts they used. Describe the individuals you interviewed (not by name) and what you learned from the interviews. Following this, provide a detailed description of the activity and any variations you observed. Grading will be based largely on the coherence and organization of your report, the level of detail in your observations, and your analysis and interpretation. Remember, that proper grammar, typo-free writing, and good overall writing is important as well. Here's a good example from a prior class.

I2b: Individual Homework #2b - Designing Research Models (due 1/30)                                                                       

Identify two measurable variables from your ethnographic study that might conceivably be associated. Design separate descriptive, correlational and experimental research designs for studying these variables. For each, include a diagram (such as the one in Figure 6 from the sample research plan, but with just one or two boxes, as appropriate) and text describing the purpose of the study, the measures you plan to use, and what the results would be useful for. Identify a possible "third variable" that might invalidate predictions made with results from the correlational study and how this will be controlled in the experimental study.

Optional Python exercise (depending upon platform selected in Class 1) for extra credit: Pick at least 5 of your favorite albums. Create an Excel spreadsheet with the following columns: Album, Tracks, Cost, PctCost (with the last column reflecting the cost of each album as the percent of the cost of all albums in this collection). Fill in the data. Save a copy to csv format and import to Python. Create a bar chart showing the number of tracks per album. Create a pie chart showing PctCost labeled by Album. Include these charts in your report.

I3: Individual Homework #3 - Usability Testing (due 2/6)
First, read through the following papers as examples of usability/performance testing of a user interface (in this case new sensing inputs for a PDA):

• Sensing techniques for mobile interaction - UIST'00
• A Usability Study of Physicians' Interaction with PDA and Laptop

Next, pick an obscure piece of software with a user interface (ideally one you may have created for a class). Define two simple tasks using the software (something you can describe in 1-3 sentences and take less than three minutes to do) and write them down on two pieces of paper. Select two or more interval or ratio measures from pages 194-195 of the Nielsen reading that you think may be relevant to the software, in addition to at least one nominal or ordinal measure (could be sociodemographic).

Ask three (or more) classmates or friends to help you with a user study. Make sure they have not used the software before. Obtain verbal consent (as discussed in class). Provide a brief description of the software (but not how to use it). Then, give each participant each task and watch them attempt to complete it. Do not provide any help. Collect your measures.

Submit a brief writeup of your test plan, descriptive statistics of your data, and any design recommendations resulting from your tests.

Here is a sample writeup of a descriptive analysis.

You may work individually or in teams of two.

I4: Individual Homework #4 - Designing a Composite Measure for a Questionnaire (due 2/20)

In this assignment you will design a new composite self-report measure to assess something about Northeastern students. One example would be a person's "Homework Procrastinaton," or degree to which a person procrastinates on homework. You should come up with your own idea about an interesting construct to measure. Assume it only has one factor, but use at least five scale items. Incorporate information from at least one literature reference. Assess the face and content validity of your measure and work through a bivariate analysis of your items.

Decide on one method for assessing validity (besides face & content) for your measure that you can also assess in your questionnaire. This should be an additional question (or an additional previously validated composite measure) on your survey and should provide a numeric measure.

Add any other items to your questionnaire that you feel are important. Implement your questionnaire on SurveyMonkey, Google Forms, or similar.
Post your questionnaire on Piazza with the title: “Questionaire: [your topic here].”

You are also obligated to reply to any questionnaires posted within 48 hrs!

Collect your data.

Compute the reliability (internal consistency) of your measure using Python. Compute descriptive statistics for your measure and any other items you may have included on the questionnaire. Assess the validity of your measure (you can do this qualitatively, e.g., using scatterplots).

Document and submit all of the above.

You may work individually or in teams of two.

I5: Individual Homework #5 - Designing and Analyzing an Experiment (due 2/27)

Work individually on this one. Answer this question referring only to the text and to the sample research plan.

Part 1 - Designing an Experiment

You are a manager at BigBucks, Inc., but you have noticed that your employees are becoming increasingly sedentary. You believe this is impacting productivity, and so you decide to conduct an experiment to test out two new fancy wearable fitness devices, the FlitBlit and the mappleBlotch, to see if they can motivate more physical activity after eight weeks of use with the company staff. Outcome measures will include steps per day and satisfaction. Satisfaction will be measured using the short-form WearableLoverSatisfaction 12-item index (Cronbach alpha=0.87, test-retest correlation of 0.91, correlation with the standard 100-item QuantifiedNerdSatisfaction index was 0.75).

From studies at other sites, you expect to see a difference in activity of approximately 3,500 (SD 1,100) steps per day between the products. Assume a 30% response rate to your recruitment ad, and a surprisingly high 94% retention rate for subjects.

Be sure to include the following in your plan:

• Hypotheses
• Research model (the boxes and arrows diagram) and description of variables/measures
• Human subjects issues, including eligibility criteria, recruitment procedures, and the number of potential subjects you need to reach with your recruitment ad.
• Detailed protocol, including recruitment, sampling and randomization methods
• Analysis plan

Your complete plan should be about 2-3 pages long, single spaced. Refer to the sample research plan for inspiration.

Part 2 - Analyzing Data from an Experiment Using Python

You've successfully completed your study, and have the following data. Write a 1-2 pages Results and Discussion section of study report describing your findings. (Hint: Be sure to review the normality of data, but for this exercise you can assume all distributions are normal.)

Team Projects

The following assignments are to be completed by teams. Team project reports should be emailed to the instructor by noon on the due date. In-class presentations should be prepared using PowerPoint and emailed to the instructor by noon on the due date. You should also have the presentation on your computer and be prepared to present in class using your laptop.

Overview
Each project has a two-week duration. Study proposals are due three class sessions before deadline, and should be about one page long covering: general topic; research question(s); measures, eligibility, and any human subjects ethics issues you are unsure about. Note: if you plan to use any questionnaires, you must include a complete copy of each in your proposal. You may not collect data until the instructor sends you an email explicitly approving your study protocol. Data collection, analysis, writeup and preparation for oral presentation are due on the due date (the schedule will be posted on Piazza). Your projects must be related in some way to IS, and follow the Northeastern IRB guidelines for student projects discussed in class (e.g., you can only do a project on adults, not children). Your final reports must be at least two pages long and no more than three pages (not counting raw data in an appendix) and contain both raw data, visualizations, analysis (statistics), and discussion (interpretation & implications). At the completion of each project you will be asked to evaluate yourself and your teammates.

Presentations
Oral presentations must be at most 10 minutes in length (hard upper bound - you will be cut off if you go over), leaving 5 minutes critique per team. Your talk should encompass the main idea/research question, hypotheses (if appropriate), study design, results, conclusions, and (very important) appropriate visualizations of your results.

Tips for a good presentation:

• Lots of visuals - ideally little or no narrative text. Your charts should be artifacts that you are presenting and talking about (it's OK to have notes).
• What research question were you addressing?
• What was your study design?
• What was your protocol?
• Who were your participants?
• What were your measures?
• What data did you collect? (Use visualizations like bar charts, pie charts, etc. as appropriate to summarize your data.)
• What were the characteristics of your distributions?
• What analyses did you conduct? Did you do any subgroup analyses?
• What are your conclusions?
• What are the implications?

Example presentations from past classes: [descriptive1] [descriptive2] [correlational1] [correlational2]

T1: Team Study #1 - Ethnographic/Descriptive

The first study should be ethnographic and/or descriptive. Priorities: Motivation, Methodology, Form (of writeup & presentation), Conclusions/Lessons Learned.

T2: Team Study #2 - Correlational / Quasi-Experimental

The second study should be correlational.

T3: Team Study #3 - Experimental

The third study should be experimental.