Chapter 5 Survey Research
CHAPTER OUTLINE AND OBJECTIVES
I. OVERVIEW
II. USES OF SURVEYS
· Survey research is used to assess people’s thoughts, opinions, and feelings.
· Surveys can be specific and limited in scope or more global in their goals.
· The best way to determine whether results of a survey are biased is to examine the survey procedures and analyses.
III. CHARACTERISTICS OF SURVEYS
· Survey research involves selecting a sample (or samples) and using a predetermined set of questions.
IV. SAMPLING IN SURVEY RESEARCH
· Careful selection of a survey sample allows researchers to generalize findings from the sample to the population.
A. Basic Terms of Sampling
· The identification and selection of elements that will make up the sample is at the heart of all sampling techniques; the sample is chosen from the sampling frame, or list of all members of the population of interest.
· Researchers are not interested simply in the responses of those surveyed; instead, they seek to describe the larger population from which the sample was drawn.
· The ability to generalize from a sample to the population depends critically on the representativeness of the sample.
· A biased sample is one in which the characteristics of the sample are systematically different from the characteristics of the population.
· Selection bias occurs when the procedures used to select a sample result in the overrepresentation or underrepresentation of some segment(s) of the population.
B. Approaches to Sampling
· Two approaches to selecting a survey sample are nonprobability sampling and probability sampling.
· Nonprobability sampling (such as convenience sampling) does not guarantee that every element in the population has an equal chance of being included in the sample.
· Probability sampling is the method of choice for obtaining a representative sample.
· In simple random sampling, each element of the population has an equal chance of being included in the sample; in stratified random sampling, the population is divided into subpopulations (strata), and random samples are drawn from the strata.
V. SURVEY METHODS
· Four methods for obtaining survey data are mail surveys, personal interviews, telephone interviews, and Internet surveys.
A. Mail Surveys
· Although mail surveys are quick and convenient, there may be a problem with the responses rate when individuals fail to complete and return the survey.
· Due to problems with response rate, the final sample for a mail survey may not represent the population.
B. Personal Interviews
· Although costly, personal interviews allow researchers to gain more control over how the survey is administered.
· Interviewer bias occurs when survey responses are recorded inaccurately or when interviewers guide individuals’ responses.
C. Telephone Interviews
· Despite some disadvantages, telephone interviews are used frequently for brief surveys.
D. Internet Surveys
· The Internet offers several advantages for survey research because it is an efficient, low-cost method for obtaining survey responses from large, potentially diverse and underrepresented samples.
· Disadvantages associated with Internet survey research include the potential for response rate bias and selection bias, and lack of control over the research environment.
VI. SURVEY-RESEARCH DESIGNS
- The three types of survey design are the cross-sectional design, the successive independent samples design, and the longitudinal design.
A. Cross-Sectional Design
· In the cross-sectional design, one or more samples are drawn from the population(s) at one time.
· Cross-sectional designs allow researchers to describe the characteristics of a population or the differences between two or more populations, and correlational findings from cross-sectional designs allow researchers to make predictions.
B. Successive Independent Samples Design
· In the successive independent samples design, different samples of respondents from the population complete the survey over a time period.
· The successive independent samples design allows researchers to study changes in a population over time.
· The successive independent samples design does not allow researchers to infer how individual respondents have changed over time.
· A problem with the successive independent samples design occurs when the samples drawn from the population are not comparable—that is, not equally representative of the population.
C. Longitudinal Design
· In the longitudinal design, the same respondents are surveyed over time in order to examine changes in individual respondents.
· Because of the correlational nature of survey data, it is difficult to identify the causes of individuals’ changes over time.
· As people drop out of the study over time (attrition), the final sample may no longer be comparable to the original sample or represent the population.
VII. QUESTIONNAIRES
A. Questionnaires as Instruments
· Most survey research relies on the use of questionnaires to measure variables.
· Demographic variables describe the characteristics of people who are surveyed.
· The accuracy and precision of questionnaires requires expertise and care in their construction.
· Self-report scales are used to assess people’s preferences or attitudes.
B. Reliability and Validity of Self-Report Measures
· Reliability refers to the consistency of measurement and is frequently assessed using the test-retest reliability method.
· Reliability is increased by including many similar items on a measure, by testing a diverse sample of individuals, and by using uniform testing procedures.
· Validity refers to the truthfulness of a measure: Does it measure what it intends to measure?
· Construct validity represents the extent to which a measure assesses the theoretical construct it is designed to assess; construct validity is determined by assessing convergent validity and discriminant validity.
C. Constructing a Questionnaire
· Constructing a questionnaire involves deciding what information should be sought and the type of questionnaire, writing a draft of the questionnaire, pretesting the questionnaire, and concluding with specifying the procedures for its use.
· The wording of questionnaires should be clear and specific using simple, direct, and familiar vocabulary.
· The order in which questions are asked on a questionnaire needs to be considered seriously because the order can affect respondents’ answers.
VIII.THINKING CRITICALLY ABOUT SURVEY RESEARCH
A. Correspondence Between Reported and Actual Behavior
· Survey research involves reactive measurement because individuals are aware that their responses are being recorded.
· Social desirability refers to pressures that respondents sometimes feel to respond as they “should” believe rather than how they actually believe.
· Researchers can assess the accuracy of survey responses by comparing these results with archival data or behavioral observations.
B. Correlation and Causality
· When two variables are related (correlated), we can make predictions for the variables; however, we cannot make inferences about the causes of the relationship.
· When a relationship between two variables can be explained by a third variable, the relationship is said to be “spurious.”
· Correlational evidence, in combination with a multimethod approach, can help researchers identify potential causes of behavior.
IX. SUMMARY
REVIEW QUESTIONS AND ANSWERS
These review questions appear in the textbook (without answers) at the end of Chapter 5, and can be used for a homework assignment or exam preparation. Answers to these questions appear in italic.
1. Briefly explain how correlational research differs from experimental research, and describe the major goal of survey research.
Correlational research assesses the relationships among naturally occurring variables and, unlike experimental research, typically does not involve the manipulation of independent variables. The goals of survey research are to describe people’s opinions, attitudes, and preferences, and to make predictions about people’s behavior. (p. 136)
2. Describe the information you would examine to determine whether survey results are biased because the sponsoring agency of the survey has a vested interest in how the results turn out.
Knowing the sponsor of a survey research project is not sufficient to judge whether the study is biased. It is more important to know whether a biased sample was used, or whether the wording of questions was slanted, or whether the data have been selectively analyzed or reported. (pp. 137–138)
3. What two characteristics do surveys have in common regardless of the purpose for which the survey has been done?
Two characteristics that surveys have in common: (1) they involve sampling, and (2) there is a set of predetermined questions used for all respondents. (p. 138)
4. Explain why there is likely to be a serious threat to the interpretability of the results of a survey when a convenience sample is used.
The greatest threat to the interpretability of the results of a survey when a convenience sample is used is a high risk of obtaining a biased sample that will not be representative of the population. (p. 142)
5. Explain the relationship between the homogeneity of the population from which a sample is to be drawn and the size of a sample needed to ensure representativeness.
The more homogeneous the population, the smaller the sample size needed to ensure representativeness. (p. 143)