Chapter 9 Complex Designs
I. Overview
II. Guidelines for Identifying an Experimental Design
· Researchers use complex designs to study the effects of two or more independent variables in one experiment.
· In complex designs, each independent variable can be studied with an independent groups design or with a repeated measures design.
III. Describing Effects in a Complex Design
· The simplest complex design is a 2 × 2 design—two independent variables, each with two levels.
· The number of different conditions in a complex design can be determined by multiplying the number of levels for each independent variable (e.g., 2 × 2 = 4).
· More powerful and efficient complex designs can be created by including more levels of an independent variable or by including more independent variables in the design.
A. An Example of a 2 × 2 Design
B. Main Effects and Interaction Effects
· The overall effect of each independent variable in a complex design is called a main effect and represents the differences among the average performance for each level of an independent variable collapsed across the levels of the other independent variable.
· An interaction effect between independent variables occurs when the effect of one independent variable differs depending on the levels of the second independent variable.
C. Describing Interaction Effects
· Evidence for interaction effects can be identified using descriptive statistics presented in graphs (nonparallel lines) or tables (subtraction method).
· The presence of an interaction effect is confirmed using inferential statistics.
D. Complex Designs with Three Independent Variables
IV. Analysis of Complex Designs
· In a complex design with two independent variables, inferential statistics are used to test three effects: the main effects for each independent variable and the interaction effect between the two independent variables.
· Descriptive statistics are needed to interpret the results of inferential statistics.
· How researchers interpret the results of a complex design differs depending on whether a statistically significant interaction effect is present or absent in the data.
A. Analysis Plan with an Interaction Effect
· If the analysis of a complex design reveals a statistically significant interaction effect, the source of the interaction effect is identified using simple main effects analyses and comparisons of two means.
· A simple main effect is the effect of one independent variable at one level of a second independent variable.
B. Analysis Plan with No Interaction Effect
· If the analysis of a complex design indicates the interaction effect between independent variables is not statistically significant, the next step is to determine whether the main effects of the variables are statistically significant.
· The source of a statistically significant main effect can be specified more precisely by performing comparisons of two means or using confidence intervals to compare means two at a time.
V. Interpreting Interaction Effects
A. Interaction Effects and Theory Testing
· Theories frequently predict that two or more independent variables interact to influence behavior; therefore, complex designs are needed to test theories.
· Tests of theories can sometimes produce contradictory findings. Interaction effects can be useful in resolving these contradictions.
B. Interaction Effects and External Validity
· When no interaction effect occurs in a complex design, the effects of each independent variable can be generalized across the levels of the other independent variable; thus, external validity of the independent variables increases.
· The presence of an interaction effect identifies boundaries for the external validity of a finding by specifying the conditions in which an effect of an independent variable occurs.
C. Interaction Effects and Ceiling and Floor Effects
· When participants’ performance reaches a maximum (ceiling) or a minimum (floor) in one or more conditions of an experiment, results for an interaction effect are uninterpretable.
D. Interaction Effects and the Natural Groups Design
· Researchers use complex designs to make causal inferences about natural groups variables when they test a theory for why natural groups differ.
· Three steps for making a causal inference involving a natural groups variable are to state a theory for why group differences exist, manipulate an independent variable that should demonstrate the theorized process, and test whether an interaction effect occurs between the manipulated independent variable and natural groups variable.
VI. Summary
REVIEW QUESTIONS AND ANSWERS
These review questions appear in the textbook (without answers) at the end of Chapter 9, and can be used for a homework assignment or exam preparation. Answers to these questions appear in italic.
1. Identify the number of independent variables, the number of levels for each independent variable, and the total number of conditions for each of the following examples of complex design experiments: (a) 2 × 3 (b) 3 × 3 (c) 2 × 2 × 3 (d) 4 × 3
(a) There are two independent variables, the first with two levels and the second with three levels. With factorial combination there would be six conditions.
(b) There are two independent variables, each with three conditions, for a total of nine conditions.
(c) There are three independent variables, the first two with two levels and the third with three levels, for a total of 12 conditions.
(d) There are two independent variables, the first with four levels and the second with three levels, for a total of 12 conditions. (pp. 273–275)
2. Identify the conditions in a complex design when the following independent variables are factorially combined: (1) type of task with 3 levels (visual, auditory, tactile) and (2) group of children tested with 2 levels (“normal,” “developmentally delayed”).
Factorial combination of these two independent variables would yield six conditions:
normal with the visual task, normal with the auditory task, normal with the tactile task, developmentally delayed with the visual task, developmentally delayed with the auditory task, and developmentally delayed with the tactile task. (p. 271)
3. Use the Kassin et al. results in Table 9.3 for interrogators’ efforts to obtain a confession to show there are two possible ways to describe the interaction effect.
The interaction effect in the Kassin et al. experiment could be described in either of two ways: the effect of interrogator expectation was different depending on the suspect status, or the effect of suspect status was different depending on the interrogator expectation. There are not two interaction effects in this complex design experiment; the same interaction effect in this experiment can be described in two ways. (pp. 278–279)
4. Describe how you would use the subtraction method to decide whether an interaction effect was present in a table showing the results of a 2 × 2 complex design.
If the differences between the means in each row (or column) are different, there is an indication of an interaction effect. (p. 282)
5. Describe the pattern in a line graph that indicates the presence of an interaction effect in a complex design.
Nonparallel lines in a line graph indicate the presence of an interaction effect in a complex design. Parallel lines in a line graph indicate that there is no interaction effect in a complex design. (pp. 281–282)
Chapter 10 Single-Case Designs and Small-n Research
I. Overview
II. The Case Study Method
A. Characteristics
· Case studies, intensive descriptions and analyses of individuals, lack the degree of control found in small-n experimental designs.
· Case studies are a source of hypotheses and ideas about normal and abnormal behavior.
B. Advantages of the Case Study Method
· Case studies provide new ideas and hypotheses, opportunities to develop new clinical techniques, and a chance to study rare phenomena.
· Scientific theories can be challenged when the behavior of a single case contradicts theoretical principles or claims, and theories can receive tentative support using evidence from case studies.
· Idiographic research (the study of individuals to identify what is unique) complements nomothetic research (the study of groups to identify what is typical).
C. Disadvantages of the Case Study Method
· Researchers are unable to make valid causal inferences using the case study method because extraneous variables are not controlled and several “treatments” may be applied simultaneously in case studies.
· Observer bias and biases in data collection can lead to incorrect interpretations of case study outcomes.
· Whether results from a case study may be generalized depends on the variability within the population from which the case was selected; some characteristics (e.g., personality) vary more across individuals than others (e.g., visual acuity).
D. Thinking Critically about Testimonials Based on a Case Study
· Being mindful of the limitations of the case study method can be helpful when evaluating individuals’ testimonials about the effectiveness of a particular treatment.
III. Single-Subject (Small-n) Experimental Designs
· In applied behavioral analysis, the methods developed within the experimental analysis of behavior are applied to socially relevant problems.
A. Characteristics of Single-Subject Experiments
· Researchers manipulate an independent variable in single-subject experiments; therefore, these designs allow more rigorous control than case studies.
· In single-subject experiments, baseline observations are first recorded to describe what an individual’s behavior is like (and predicted to be like in the future) without treatment.
· Baseline behavior and behavior following the intervention (treatment) are compared using visual inspection of recorded observations.
B. Specific Experimental Designs
· In the ABAB design, baseline (A) and treatment (B) stages are alternated to determine the effect of treatment on behavior.
· Researchers conclude that treatment causes behavior change when behavior changes systematically with the introduction and withdrawal of treatment.
· Interpreting the causal effect of the treatment is difficult in the ABAB design if behavior does not reverse to baseline levels when treatment is withdrawn.
· Ethical considerations may prevent psychologists from using the ABAB design.
· In multiple-baseline designs, a treatment effect is shown when behaviors in more than one baseline change only following the introduction of a treatment.
· Multiple baselines may be observed across individuals, behaviors, or situations.
· Interpreting the causal effect of treatment is difficult in multiple-baseline designs when changes are seen in a baseline before an experimental intervention; this can occur when treatment effects generalize.
C. Problems and Limitations Common to all Single-Subject Designs
· Interpreting the effect of a treatment can be difficult if the baseline stage shows excessive variability or increasing or decreasing trends in behavior.
· The problem of low external validity with single-subject experiments can be reduced by testing small groups of individuals.
