IS4800 – Empirical Research Methods in Information Science

 

 

[Syllabus] [Schedule] [Homework] [Projects] [Bibliography] [Resources] [Directory]

 

Homework (Individual Assignments)

 

The following assignments are to be completed individually and emailed to is4800@ccs.neu.edu by noon on the due date. All homeworks should be emailed to as a single Word or PDF file, with relevant R output (graphs, tabular output, statistics, but not commands) copied and pasted into the document, along with any other materials considered for grading (e.g., survey instrument).

 

I1

Individual Homework #1 (not graded - due next class)

 

  1. If you do not have a CCIS account and have not requested one, you should do so immediately. An account is required to access the online readings. See the Resources page for details.
  2. Send an email with your name and preferred email address to is4800@ccs.neu.edu.

  

 

I2-3

Individual Homework #2-3 – Background Research & Human Subjects (due in 1 week)

 

  1. Describe how knowledge acquired from conducting the study specified in the sample research plan meets (or does not meet) the criteria for “scientific explanations”.
  2. Describe the roles that background research play in the sample research plan, giving an example of each.
  3. Find at least three technical references that relate to the proposed study in different ways. For each, give the full citation, describe how you found it, and describe how it contributes to the research plan.
  4. Describe how the sample research plan addresses the three ethical principles of (a) respect for persons; (b) beneficience; and (c) justice, discussed in class.
  5. What is the overall purpose of the “Data and Safety Monitoring Plan” described in Section E.8 of the sample research plan?
  6. What is the overall point of Section “D.3.4.1 Study Subjects”?

 

 

I4

Individual Homework #4 – Ethnography (due in one week)

 

First, read these two papers to get a better understanding of the role of Ethnography in Information Science:

                              

Northeastern has just hired you to make your favorite dining hall more efficient and friendly. Pick a location and spend an hour people watching with a notebook and pencil. Spend some time watching the kinds of activities that people are engaged in, and think about how technology could help improve these activities. Pick one such activity to focus on and study. For your chosen activity watch several people perform the task and make detailed notes about the series of steps they go through, any objects (“artifacts”) they use, whether they interact with other people and, if so, the step-by-step details of this interaction. Conduct two or more unstructured interviews with your subjects.

 

Some examples of activities (you can study one of these or, better yet, come up with your own):

  • finding a table
  • choosing a meal
  • choosing a restaurant (if several)
  • rendezvousing with friends

 

What to Email  Your report should include a one-paragraph summary of why you picked your particular activity to focus on, followed by an overview of the activity, the kinds of people you observed engaging in it, and a description of any artifacts they used. Describe the individuals you interviewed (not by name) and what you learned from the interviews. Following this, provide a detailed description of the activity and any variations you observed. Grading will be based largely on the coherence and organization of your report, the level of detail in your observations, and your analysis and interpretation.

 

I5

Individual Homework #5 – Designing Research Models (due next class)  

 

Identify two measurable variables from your ethnographic study that might conceivably be associated. Design separate descriptive, correlational and experimental research designs for studying these variables. For each, include a diagram (such as the one in Figure 6. from the sample research plan, but with just one or two boxes, as appropriate) and text describing the purpose of the study, the measures you plan to use, and what the results would be useful for. Identify a possible “third variable” that might invalidate predictions made with results from the correlational study and how this will be controlled in the experimental study.

 

First R exercise: Pick at least 5 of your favorite albums. Create an Excel spreadsheet with the following columns: Album, Tracks, Cost, PctCost (with the last column reflecting the cost of each album as the percent of the cost of all albums in this collection). Fill in the data. Save a copy to csv format and import to R. Create a bar chart showing the number of tracks per album. Create a pie chart showing PctCost labeled by Album. Include these charts in your report.

 

 

I6

Individual Homework #6 – Objective Measures & Descriptive Statistics in R (due next class)

 

Enter the following physician order entry dataset into Excel, and import into R. Provide frequency tables, histograms, and descriptive statistics as appropriate. Tabulate counts of JobCategory by Gender. Create a scatter plot of EHREntryTime vs. YearsComputerExperience. Provide boxplots of Accuracy by gender.

 

Turn in: the results of your analyses with narrative text describing the results.

Extra credit: turn in the R program for computing all of the above directly from the imported data frame exactly as specified below.

 

ProviderID

Gender

JobCategory

YearsComputerExperience

EHREntryTime

Accuracy

1

female

attending

10.5

5.5

82.5

2

male

intern

0.8

15.5

76.9

3

female

attending

7.0

7.0

89.4

4

male

intern

1.6

9.5

47.9

5

female

resident

7.0

6.0

88.3

6

female

intern

2.0

12.0

57.1

7

male

intern

1.5

10.5

63.2

8

male

resident

5.0

4.0

73.5

9

female

resident

3.8

12.5

68.5

 

 

I7

Individual Homework #7 – Usability Performance Test (due in one week)

 

First, read through the following papers as examples of usability/performance testing of a user interface (in this case new sensing inputs for a PDA):

 

Next, pick an obscure piece of software with a user interface (ideally one you may have created for a class). Define two simple tasks using the software (something you can describe in 1-3 sentences and take less than 3 minutes to do) and write them down on two pieces of paper. Select two measures from pages 194-195 of the Nielsen reading that you think may be relevant to the software.

 

Ask three (or more) classmates or friends to help you with a user study. Make sure they have not used the software before. Obtain verbal consent (as discussed on 1/15). Provide a brief description of the software (but not how to use it). Then, give each participant each task and watch them attempt to complete it. Do not provide any help. Collect your measures.

 

Submit a brief writeup of your test plan, descriptive statistics of your data, and any design recommendations resulting from your tests.

 

 

I8

Individual Homework #8 – Behavioral Measures (due in one week)

 

Your manager thinks everyone in the department is slacking off. She asks you to do an objective study of how hard her employees are working at their computers.

 

Define observational measures to categorize a computer worker’s behavior as “intensely working”, “casually working”, or “not working”. Think about how to encode a worker’s behavior during 10 minutes of observation (hint: try 30-second or 1-minute intervals method). Write a brief coding manual so that someone else could follow your directions.

 

Pick a public area in which people are using computers and observe two people for 15 minutes each. If you cannot surreptitiously observe users, be sure to obtain verbal consent first (tell them you are studying how people behave when using computers, not that you are studying how hard they are working). Code their behavior.

 

Individual homework: turn in your coding manual, data, and any descriptive statistics that make sense.

 

Team of 2 students (preferred): both code observations for the same subjects at the same time. In addition to the above, compute and report Kappa. If your Kappa is below 0.7 discuss the nature of your discrepancies and how you would update your coding manual to be more reliable.

 

I9

Individual Homework #9 – Designing a Composite Measure for a Questionnaire (due in one week)

 

Design a new composite self-report measure to assess a person’s degree of addiction to Facebook. Assume it only has one factor, but use at least five scale items. Incorporate information from at least one literature reference. Assess the face and content validity of your measure and work through a bivariate anlaysis of your items.

 

Decide on one method for assessing validity (besides face & content) for your measure that you can also assess in your questionnaire (should be a numeric measure).

 

Add any other items to your questionnaire that you feel are important.

 

Email your questionnaire to the class (is4800-all@ccs.neu.edu). (You are also obligated to reply to any questionnaires mailed to you within 48 hrs.)

 

Extra credit: implement your questionnaire on surveymonkey.com.

 

Compute the reliability (internal consistency) of you measure using R Alpha. Compute descriptive statistics for your measure and any other items you may have included on the questionnaire. Assess the validity of your measure (you can do this qualitatively, e.g., using scatterplots).

 

Document and submit all of the above.

 

You may work individually or in teams of two. Due in one week.

 

I10

Individual Homework #10 – Chi^2 and Correlation Testing in R (due in one week)

 

Work individually on this one. For each problem follow the steps to hypothesis testing we followed in class (statement of hypotheses, specification of testing criteria, etc.). Turn in your documentation for each step. Use R for the calculations, but copy and paste the relevant part of the R output (tables, charts) into your Word document.

 

Part 1 – Chi-Square Goodness of Fit

Translate scores from your "Facebook addiction " I9 survey into an ordinal measure with three values spanning equal parts of your index’s range. For example, if your index ranges from 1 to 10, then 1.0 to 3.33 would be labeled “low”, 3.33 to 6.66 would be labeled “medium”, and 6.66 to 10.0 would be labeled “high”. Determine how well your sample matches an expected distribution in which individuals fall equally into the 3 categories.

 

Part 2 – Pearson Correlation

Determine how well your index of "Facebook addiction " co-varies with your validation measure (final part of I9). Provide scatter plot, calculation of r, and hypothesis test. What does your result tell you about the "Facebook addiction " composite measure? 

 

 

I11

Individual Homework #11 – Designing and Analyzing an Experiment (due 3/12)

 

Work individually on this one.

 

Part 1 – Designing an Experiment

Write a research plan for conducting an experiment comparing WizziWord vs. CoolText word processors using admins from BigBucks, Inc (both of these are new products). Outcome measures to include productivity (words per day output during the 8th week after the new word processors are introduced), and satisfaction, using the ILoveWordProcessors 6-item index (Cronbach alpha=0.82, test-retest correlation of 0.93, correlation with the standard 100-item WordProcessorsAreGreat index was 0.72). From studies at other sites you expect to see a difference in productivity of approximately 3,000 (SD 1,600) words per day between the products. Assume a 90% retention rate for subjects. Be sure to include the following in your plan:

  • Hypotheses
  • Research model (the boxes and arrows diagram) and description of variables
  • Human subjects issues, including eligibility criteria and recruitment procedures
  • Detailed protocol , including recruitment, sampling and randomization methods
  • Analysis plan
  • Power analysis

Refer to sample research plan for inspiration. Your complete plan should be about two pages long.

 

Part 2 – Analyzing Data from an Experiment Using R

You’ve successfully completed your study, and have the following data. Write a one paragraph summary describing your findings.

 

ID Condition Performance Satisfaction
1 C 5293 3.4
2 C 1602 6.5
3 C 6231 2.3
4 C 2350 1.4
5 W 4873 4.4
6 W 8320 6.2
7 W 7332 2.4
8 W 4239 5.2
9 C 3214 2.3
10 C 2431 4.9
11 W 6321 5.9
12 W 5321 6.1
13 C 4276 3.4
14 C 2302 3.9
15 W 4712 1.4
16 W 6904 2.1