Chapter 8: External Validity Research Methods Notes

Chapter 8: External Validity of Research

External validity: Are the findings of a study specific to the conditions under which the study was conducted or do they represent general principles of behavior that apply under a wide-ranging set of conditions?

Population – sampling frame (the people from who you’re actually going to select your sample) – sample (the actual people selected)

• Ex. Population: SSI recipients living in the Bronx, all ages, all disabilities
• Sampling Frame: inclusion: turning 18 in 2007; exclusion: vision & hearing impaired non-ambulatory, people not residing in Bronx
• Sample: n = 100 people randomly selected from sampling frame

Aspects of External Validity

• Generalizing across: whether the results of a study pertain equally to more than one setting, population, or subpopulation, such as both women and men

• Generalizability

• Generalizing to: whether the results of a study pertain to a particular setting or population, such as hospitalized psychiatric patients

• Ecological validity

• If a principle does not operate as expected under a particular set of circumstances, those circumstances are incorporated into the theory as boundary, or limiting, conditions 

Components of External Validity

• Structural: method by which a study is carried out, including factors such as the setting in which the study is conducted, the research procedures used, and the nature of the participant sample

• Setting Factors

• Physical setting; having a mirror or not in room affected responses
• Reactivity: people’s tendency to change their behavior when they know they are under observation

• Experimental realism: manipulating the IV in such an engaging manner that subjects become so psychologically involved in the situation that they give realistic responses (sexual arousal studies)

• Researcher attributes: gender, ethnicities
• Coparticipant attributes: influenced by other participants
• Ecological validity: behavior is different in lab vs natural settings

• Participant Sample Factors

• Convenience sampling: participants chosen on basis of availability rather than on the basis of representativeness of the population as a whole/ specific subpopulation
• Restricted sampling: only men, only White, etc.
• Ecological validity: representativeness of research sample; most are college students in America – low ecological validity
• Volunteer participants: have specific characteristics; differ from non-volunteers
• Person-by-situation interactions: different types of people respond differently to the same IV

• Research Procedure Factors

• Artificiality: labs are artificial and don’t generalize to real world phenomena
• Operational definitions: persons response can vary based on operational definition; constructs have many different definitions
• Levels of the IV: must cover entire range of IVs (responsibility definition problems in study about armed robbery trial)
• Ecological validity: artificial research procedures such as experimental simulations limit the application of research to natural settings

• Cultural Factors: individualistic vs collectivist cultures
• Time Factors

• Time sampling: more car accidents happen in winter so studies aimed at preventing accidents have to take seasonal variations in driving conditions into consideration; also must take into account time required for an IV to have an effect
• Changes over time: beliefs and behaviors change over time (physical attractiveness, sexual mores, gender roles)

• Functional: degree to which the psychological processes that operate in a study are similar to the psychological processes at work in a particular natural setting
• Conceptual: degree to which the problems studied in research correspond to the problems considered important in a natural setting
• Structural: generalizability / functional & conceptual: ecological validity
• Verisimilitude: similarity between research settings and natural settings and thus to the ecological validity of research

Relationships among components of external validity

• All 3 components are not entirely independent of each other but are necessary to accurately model a natural situation

• Chronic manipulations: found in natural settings and result from an accumulation of small effects over time, have been experienced for some time, and are expected to continue into the future (anxiety, self-esteem, stress)
• Acute manipulations: result from a single event, occur relatively suddenly, and end with the research situation

Assessing Generalizability