Learning to Code for the Archival Project

The Archival Project has developed a specific coding scheme for volunteers to follow. As a supplement to that scheme, the following tutorial can be used to gain more facility with identifying article structure and understanding terminology.

Abstracts

An article's abstract summarizes the most important contents of the article. Because it summarizes the most important questions asked by the researchers, and usually their results, we use the abstract as a guide as to which information needs to be coded. In the Archival Project, we only code results mentioned in the abstract.

This doesn't mean, however, that you only need to read the abstract. What it means is that you should identify the topics mentioned there, and look for them in the rest of the article.

As you read an article, scan the abstract for results and general statements about relationships between variables.

Example 1

Abstract from Fischer, Greitemeyer, and Frey (2008).


In the abstract above, the highlighting indicates the relationships that the researchers investigated. As stated in the first sentence, Fischer et al "investigated the impact of self-regulation resources on confirmatory information processing." If that was all that the abstract said, we would code any test that had to do with the relationship between self-regulation resources (the IV) and confirmatory information processing (the DV).

This is still the case, however, these highlighted portions give us more direction. The green highlighting indicates that in at least four tests, the authors examined depletions of the IV and their impacts on the DV. Blue highlighting indicates that the researchers tested three different IVs (eg. ego threat) and their impact on the DV. Lastly, the yellow highlighting indicates that the original IV was tested against a new DV, "commitment to their [the participant's] standpoint," and how that DV impacted the original DV of confirmatory information processing. Based on our look at the abstract then, we will likely see nine tests in this article that need to be coded.


Practice Questions


Read the following abstract:

"Research has found that positive affect broadens attention. However, these studies have manipulated positive affect that is low in approach motivation. Positive affect that is high in approach motivation should reduce the breadth of attention, as organisms shut out irrelevant stimuli as they approach desired objects. Four studies examined the attentional consequences of approach-motivated positive-affect states. Results were consistent with predictions. Participants showed less global attentional focus after viewing high-approach-motivating positive stimuli than after viewing low-approach-motivating positive stimuli (Study 1) or neutral stimuli (Study 2). Study 3 found that greater trait approach motivation resulted in less global attentional focus after participants viewed approach-motivating positive stimuli. Study 4 manipulated affect and approach motivation independently. Greater approach-motivated positive affect caused lower global focus. High-approach-motivated positive affect reduces global attentional focus, whereas low-approachmotivated positive affect increases global attentional focus. Incorporating the intensity of approach motivation into models of positive affect broadens understanding of the consequences of positive affect."

Gable and Harmon-Jones (2008)

Which sentence describes an effect to be coded?

"Participants showed less global attentional focus after viewing high-approach-motivating positive stimuli than after viewing low-approach-motivating positive stimuli (Study 1) or neutral stimuli (Study 2)."

"Positive affect that is high in approach motivation should reduce the breadth of attention, as organisms shut out irrelevant stimuli as they approach desired objects."


Read the following abstract:

"People often diverge from members of other social groups: They select cultural tastes (e.g., possessions, attitudes, or behaviors) that distinguish them from outsiders and abandon tastes when outsiders adopt them. But while divergence is pervasive, most research on the propagation of culture is based on conformity. Consequently, it is less useful in explaining why people might abandon tastes when others adopt them. The 7 studies described in this article showed that people diverge to avoid signaling undesired identities. A field study, for example, found that undergraduates stopped wearing a particular wristband when members of the “geeky” academically focused dormitory next door started wearing them. Consistent with an identity-signaling perspective, the studies further showed that people often diverge from dissimilar outgroups to avoid the costs of misidentification. Implications for social influence, identity signaling, and the popularity and diffusion of culture are discussed."

Berger and Heath (2008)

Which sentence describes an effect to be coded?

"But while divergence is pervasive, most research on the propagation of culture is based on conformity."


"Consistent with an identity-signaling perspective, the studies further showed that people often diverge from dissimilar outgroups to avoid the costs of misidentification."

Hypotheses

Hypotheses are the researchers' expectations of their experiments' results. Hypotheses can be directional or non-directional.

A directional hypothesis means that the researchers have a specific idea of how the dependent variables will relate to one another. In other words, the researchers make a prediction that one group's average dependent variable will be different from another group's average dependent variable. Sometimes this can mean one is expected to be larger, or faster, or more memorable.

A non-directional hypothesis means that the researchers do not make specific predictions of how the dependent variables of different groups will relate to one another. In other words, they say that they anticipate a difference, but not that a particular group will be larger, faster, or more memborable.

Examples

Directional:
"We expected that participants who were shown the visual stimulus would have greater recall than those who were not."
"We predicted that more errors would be made in the group without time for practice."
Non-directional:
"Do breakfast eaters differ in weight on average from non-breakfast eaters?"
"We will explore the relationship between depth perception and distance estimates."

Practice Questions


What kind of hypothesis is found in Santos, 2008?

"If this is the case, and if adults with ASD have reduced top-down influence, then categorization should have a smaller influence on perceptual processing in adults with ASD than in control subjects."


Directional hypothesis


Non-directional hypothesis


What kind of hypothesis is found in Goshke, 2008?

"We predicted that PM cues would be overlooked (i.e., not acted upon) more often on high- than on low-conflict trials because on high-conflict trials the current goal (i.e., to perform the ongoing task) is automatically shielded from distracting information."


Directional hypothesis


Non-directional hypothesis

Methodology

Methodology coding requires you to examine the processes researchers undertook to gather their data.

Archival measures: Some or all of the data used in the analysis was already collected, prior to the study, and was simply obtained by the researcher. For example:

A researcher compares population densities listed on government websites to traffic conditions

Brain Imaging: The analysis involves at least one variable derived from fMRI or other spatial brain imaging techniques. For example:

A researcher uses an fMRI to see brain activity in participants while they complete word puzzles

Judgment measures: Some or all of the data included in the analysis is an evaluation of the participant, made by other people. For example:

A researcher tests how perceptions of how personable participants are correlate with the participants' usual tip amount on a check

Non-imaging physiological measures: Some or all of the data included in the analysis is a methodology that takes information directly form the participant's body. For example:

Glucose levels derived from blood samples are compared to performance on a self-control task

Self report: Participants respond to a question and their answer is coded and used as data. For example:

Participants respond to a survey about their free time activities

Indirect verbal or response time: Data comes from the participants' responses to a question, but is not the actual response; instead, the data is some sort of assessment of their answers. For example:

A participant categorizes colors as good or bad and their response times are used to assess implicit bias

Behavioral/Choice measures: Some or all of the data is an analysis of the participant's actions and responses when presented with a decision. For example:

Participants are asked to take a snack, either a banana or chocolate bar, and researchers compare these decisions to the weight of the participants

Practice Questions

Based on the following quote, which methodologies were used in Burt, 2008?

"After 45 to 60 min, participants adjourned to individual rooms to complete sociometric rankings (on a scale of 1-4, with 1 meaning 'I liked this person best')...DNA was obtained via Oragene...saliva collection kits."

The researchers compared how well liked a person was to information found in their DNA.

  Archival Measures
  Brain Imaging Measures
  Judgment Measures
  Non-imaging Physiological Measures
  Self Report Measures
  Indirect Verbal or Response Time Measures
  Behavioral/Choice Measures

Based on the following quote, which methodologies were used in Demany, 2008?

"On each trial, the listener was presented with two successive ‘‘chords’’ (sums of pure tones) and had to indicate if they were identical or not. The listener, sitting in a sound attenuated booth, gave his or her responses by clicking a computer mouse on two virtual buttons."

The researchers compared how well liked a person was to information found in their DNA.

  Archival Measures
  Brain Imaging Measures
  Judgment Measures
  Non-imaging Physiological Measures
  Self Report Measures
  Indirect Verbal or Response Time Measures
  Behavioral/Choice Measures

Analytic Design Code

Analytic design code refers to the way in which the researchers analyzed their data–what kinds of tests were run to find effect.

Correlational/Multivariate internal analysis of manipulation check: A manipulation is assessed by examining a variable (one that is different from the study's IV) and its relationship with the study's DV. For example:

In a study that manipulates levels of anger and how they correspond to the force a person hits a punching bag with, the check looks at an anger measurement scale and the force with which a punching bag is hit in order. They do this to be sure that angrier people do actually hit the bag harder.

Correlational/Multivariate analysis without manipulation: The researchers look for a relationship among variables that they did not affect themselves, meaning there are no assigned groups. This differs from a quasi experiment in that here, there is no manipulation. For example:

Researchers look for a relationship between the number of siblings a participant has and how assertive they are.

Experimental analysis of manipulation effect: The researchers test the effects of manipulating a variable on randomly assigned groups. For example:

In a study where one group is asked to read a distracting text and another group isn't, researchers compare each groups' average number of remembered words from a vocabulary list presented before the distracting text.

Experimental analysis of repeated-measures effect: The researchers test the same group of participants for all manipulations in an experiment. A common use case is longitudinal studies. For example:

In a study of cancer patients suffering from depression, researchers measure all of the participants' serotonin levels and reported happiness throughout cancer treatment, looking to find if participants become less depressed as they progress in their treatment.

Combined experimental and repeated-measures effect: The study has multiple IVs, where one is between subjects and one is within subjects. In other words, there are at least two groups of participants, and each group is tested multiple times. For example:

Participants see a series of photos. Some participants see photos of daytime and others see nighttime photos. Within each of these groups, different landscapes, like a desert, city, or rainforest, are presented. The researchers investigate to see which images (night/day, natural/constructed) are considered to be the most soothing.

Quasi-experimental analysis of manipulation effect: Participants in the study belong to different groups and there is a manipulation, but the researcher is unable to randomize assignment of the IV. For example:

Researchers investigate competition by priming participants with a sports rival and compare the effect of the prime between men and women.

Based on the following information from Burt, 2008, which analytic design code is appropriate?

  • Participants give DNA samples
  • Participants rank their cohorts' personalities
  • Researchers assess DNA for serotenergic activity and analyze its relationship with personality rankings

Correlational/Multivariate Internal Analysis of Manipulation Check


Correlational/Multivariate Analysis Without Manipulation


Experimental Analysis of Manipulation Effect


Experimental Analysis of Repeated-measures Effect


Combined Experimental and Repeated-measures Effect


Quasi-experimental Analysis of Manipulation Effect

Based on the following information from Demany, 2008, which analytic design code is appropriate?

  • Participants all listen to pairs of tones where one tone is followed by another tone that has a 50% chance of differing from the first
  • Length of time between the first and second tone varied by trial
  • Participants made same/different judgments on the tones and the researchers compared their accuracy to the length of time between the two tones.

Correlational/Multivariate Internal Analysis of Manipulation Check


Correlational/Multivariate Analysis Without Manipulation


Experimental Analysis of Manipulation Effect


Experimental Analysis of Repeated-measures Effect


Combined Experimental and Repeated-measures Effect


Quasi-experimental Analysis of Manipulation Effect