Abstract

This study examined whether the prevalence of self-reported health risk behaviors among high school students varied by survey setting (school versus home) and mode of administration (paper and pencil versus computer). Students in grades 9 and 11 were assigned randomly to one of four conditions—school paper-and-pencil instrument (PAPI), school computer-assisted self-interview (CASI), home PAPI, and home CASI. During the spring of 2004, 4,506 students completed identically worded questionnaires based on the Youth Risk Behavior Survey questionnaire. Logistic regression analyses controlling for sex, grade, and race/ethnicity revealed that setting was associated significantly with the reporting of 30 of the 55 risk behaviors examined, and mode was associated significantly with the reporting of 7 of the 55 behaviors. For all behaviors with a significant setting main effect, the odds of reporting the behavior were greater among students who completed questionnaires at school than among students who completed questionnaires at home. For all behaviors with a significant mode main effect, PAPI mode students had lower odds of reporting the behavior than CASI mode students. Because social measurement research assumes that higher prevalence estimates are more valid than lower estimates, methodological factors shown to increase estimates, such as setting and mode, should be considered when planning surveys.

Several national surveys measure health risk behaviors among adolescent populations (Grunbaum et al. 2004; Johnston et al. 2004; Substance Abuse and Mental Health Services Administration [SAMHSA] 2004). Although these surveys sometimes yield similar prevalence estimates of tobacco, alcohol, and other drug use, often the results differ considerably. Such differences raise important questions among policymakers, researchers, and other users of the data: How can these differences be explained? Which survey provides more valid results? Can survey methods be refined to produce more consistent results?

To answer these questions, the Office of the Assistant Secretary for Planning and Evaluation within the Department of Health and Human Services commissioned expert papers (Cowan 2001; Fendrich and Johnson 2001; Fowler and Stringfellow 2001; Harrison 2001; Sudman 2001) that compared the methodologies of three surveys: the Youth Risk Behavior Survey (YRBS) (Grunbaum et al. 2004); Monitoring the Future (Johnston et al. 2004); and the National Survey on Drug Use and Health, formerly known as the National Household Survey on Drug Abuse (SAMHSA 2004). These papers and other articles that have examined differences among these surveys (Coggeshall and Kingery 2001; Gfroerer, Wright, and Kopstein 1997) presented similar conclusions. All authors noted that although methodological factors such as survey setting, question wording, and data-processing techniques could account for the differences in results, methodological studies using experimental designs are needed to understand fully how these factors affect the reported prevalence of health risk behaviors.

One such study, conducted in 2002, was designed to examine the effect of question wording and appeals for honesty on the prevalence of self-reported risk behaviors (Brener et al. 2004a). In that study, high school students were assigned randomly to complete one of six versions of a paper-and-pencil questionnaire. Each questionnaire version represented a different combination of honesty appeal and type of question wording. The study found that when population, setting, questionnaire context, mode of administration, and data-editing protocols were held constant, varying honesty appeals did not have an effect on the prevalence of health risk behaviors. The study also found that differences in question wording could create statistically significant differences in some prevalence estimates, but no one type of question wording consistently produced higher or lower estimates.

Studies examining how survey setting affects prevalence estimates have found a higher prevalence of risk behaviors among students who completed questionnaires in school than among students who completed questionnaires at home (Gfroerer, Wright, and Kopstein 1997; Kann et al. 2002; Rootman and Smart 1985). These results, however, were confounded by differences in sample designs, sample sizes, and response rates. Another study randomly assigned students in a single sample to complete questionnaires in school or at home; the results did not show significant differences in prevalence estimates (Needle et al. 1983).

Computer-assisted self-interviewing (CASI) has been used with increasing frequency in surveys of health risk behaviors. Two studies conducted among adolescents in their homes found that the reporting of sensitive behaviors was more likely when using CASI than when using paper-and-pencil instruments (PAPI) (Turner et al. 1998; Wright, Aquilino, and Supple 1998). In contrast, two studies comparing CASI and PAPI modes of administration in a school setting found no significant differences in risk behavior prevalence between the two modes (Beebe et al. 1998; Hallfors et al. 2000), although higher prevalence was found among students using the PAPI mode than among students using the CASI mode when students’ computers were situated close together (Beebe et al. 1998).

These studies on setting and mode demonstrate that, when holding the mode of administration constant, prevalence of risk behaviors is equal or higher when questionnaires are administered in schools compared with when they are administered in students’ homes. The effect of mode, however, appears to vary by setting. To date, no study has varied systematically both setting and mode of administration to understand the effects of each. The purpose of this study is to examine the effects of setting and mode using random assignment of respondents to four conditions (school PAPI, school CASI, home PAPI, and home CASI), while holding other methodological factors constant. Based on the literature, it is proposed that prevalence estimates of health risk behaviors will be higher among students who complete questionnaires at school than at home and estimates will be lower among students using PAPI than those using CASI, although this effect might be modified by a setting-by-mode interaction. Specifically, it is proposed that prevalence estimates of health risk behaviors among students who complete questionnaires in school will not differ significantly by mode, whereas strong mode effects will be found among students who complete questionnaires at home.

This study also explores factors that might explain differences in the reporting of risk behaviors by setting and mode. For example, substance use prevalence has been shown to vary depending on the degree of respondent privacy (Aquilino, Wright, and Supple 2000; Gfroerer 1985; Schutz, Chilcoat, and Anthony 1994), perceived anonymity (Supple, Aquilino, and Wright 1999), and trust (Wright, Aquilino, and Supple 1998). These variables, as well as comfort and experience with computers, are assessed in this study to understand how they might modify any setting and mode effects.

Methods

SAMPLE

Participants were drawn from 64 schools in 8 states. Education agencies that had successfully completed a YRBS were selected purposively based on their willingness to participate and with the goal of creating a total sample of geographically, racially, and ethnically diverse students. Contacts at the education agencies that agreed to participate each identified between 6 and 11 schools containing grades 9 and 11 for possible participation. Each participating school then identified two ninth-grade and two eleventh-grade classes from which to recruit students for the study. All students in selected classes were eligible to participate. The four selected classes within each school were assigned randomly to one of four conditions (school PAPI, school CASI, home PAPI, or home CASI). This study received clearance from the U.S. Office of Management and Budget and was approved by institutional review boards at the Centers for Disease Control and Prevention and Macro International (ORC Macro), the study contractor.

Of the 5,920 students enrolled in the selected classes, 83 percent returned parental permission forms granting them permission to participate in the study, 4 percent returned forms denying them permission to participate, and 13 percent did not return permission forms despite multiple reminders to students and their parents. Of the 4,935 students who had permission to participate in the study, 4,517 completed questionnaires, resulting in an overall response rate of 76 percent of all eligible students. This response rate is the American Association for Public Opinion Research (AAPOR) response rate 1 (AAPOR 2004). In the home setting, among students whose parents had granted permission, 3 percent refused to participate, and an additional 5 percent did not participate because data collectors were unable to schedule an appointment during the field period (February–May 2004) despite repeated attempts. In the school setting, 12 percent of students whose parents had granted permission for participation did not complete questionnaires, generally because they were absent on the day of survey administration.

QUESTIONNAIRE

Students in all four conditions completed an 80-item questionnaire that was labeled and introduced as a “student health survey.” Seventy of the items were identical to items on the standard 2003 YRBS questionnaire and assessed demographic characteristics as well as behaviors related to unintentional injuries and violence, tobacco use, alcohol and other drug use, sexual behaviors, dietary behaviors, and physical activity. Two additional items assessed how many days the student had missed school for any reason and without permission during the 30 days preceding the survey. One item measured the student’s preference for survey mode: “If you had a choice of taking this survey using a computer or taking this survey using paper and pencil, which would you choose?”

The remaining seven items were adapted from other studies (National Institute on Drug Abuse 1991; Tseng et al. 1998; Wright, Aquilino, and Supple 1998) and assessed factors that might help explain why the reporting of risk behavior might vary by setting or mode. Three items assessed factors relevant to setting and mode: perceived anonymity, perceived privacy, and trust. The other four items assessed factors related to mode: frequency of computer use, use of computers for surveys or tests, comfort with computers, and computer privacy (see table 3). Exact question wording for all questions can be found in the appendix in the online version of the journal.

DATA COLLECTION PROCEDURES

In all four conditions, surveys were administered by trained data collectors. The data collectors followed standardized protocols for all conditions, including reading aloud scripts that explained the survey procedures.

School. In classes assigned to the PAPI mode, survey administration followed standard procedures used for the national YRBS (Brener et al. 2004b). Questionnaires were administered in regular classrooms during a single class period during the school day. Students recorded their responses in a computer-scannable questionnaire booklet. In classes assigned to the CASI mode, data collectors brought in and set up laptop computers in the classroom for students’ use. In about one-fourth of the CASI mode classes, survey administration was conducted in alternate locations, such as multipurpose rooms or computer labs, because classroom space either was inadequate or not available at least 30 minutes before data collection was scheduled to take place.

In the CASI mode, the data collectors provided students with instruction on how to use the computer to complete the survey, and students completed three practice questions before beginning the actual survey. These practice questions allowed students to familiarize themselves with how to select answers, skip questions they chose not to answer, and return to previous questions.

Home. For the home setting, data collectors scheduled student appointments outside of school hours to administer the questionnaires. On arrival at each home, the data collector selected a semiprivate, quiet room to administer the survey, such as the kitchen. Although the student’s home was the preferred setting for data collection, about 300 (17 percent) students assigned to the home setting completed the questionnaire away from the home. These alternate sites were quiet public places such as libraries. Whether the survey was administered in the home or away from the home, the data collector remained in the room with the student, but not in view of the survey, in case the student had questions.

In the home PAPI condition, students recorded their responses on the same computer-scannable booklets used in the school PAPI condition. In the home CASI condition, data collectors provided the same instructions for using the computer that school CASI mode students received, and students completed the same practice questions before beginning the actual survey.

DATA ANALYSIS

All PAPI questionnaire booklets were scanned using standard YRBS procedures, and all data were edited for inconsistent and out-of-range responses according to standard YRBS procedures (Brener et al. 2004b). Of the 4,517 completed questionnaires, 3 were deleted because the respondents selected identical responses for 15 or more questions in a row, and an additional 8 were deleted because they did not have 20 valid responses remaining after editing. The final data set therefore contained 4,506 usable questionnaires.

Five of the 60 risk behaviors assessed were excluded from analyses because the numbers of students who reported engaging in the behaviors were too small for stable logistic regression models (Peduzzi et al. 1996). For the remaining 55 behaviors, responses were dichotomized so each question measured whether students engaged in a risk behavior. For 49 of these behaviors, this coding followed standard YRBS convention, with engagement in the behavior being the response of interest. For six behaviors, however, coding of the referent departed from standard YRBS procedures, such that engaging in the behavior was the referent and failure to engage in the behavior was the response of interest. This recoding simplified interpretation by allowing all behaviors to be coded as risks.

All analyses used SUDAAN to account for the clustering of students within classrooms. Chi-square analyses examined differences in demographic and ancillary variables by condition, and results were considered significant at the p < .05 level. Logistic regression was used to examine simultaneously the main effects of setting and mode on the reporting of risk behaviors, controlling for sex, race/ethnicity, and age. Age was coded as a five-level categorical variable (see tables 1 and 2). These categories corresponded to the response options on the questionnaire, except that “12 years old or younger” and “13 years old” were collapsed with “14 years old” to become “≤14 years old,” because only seven students indicated they were younger than 14.

Table 1.

Demographic Characteristics of Students in Grades 9 and 11 Nationwide and in the Study Sample, by Survey Condition

   School Setting Home Setting  
   
 

 
 
Characteristic National Distribution (%) Study Sample (%) PAPI (%) CASI (%) PAPI (%) CASI (%) χ2 (p-value) 
Sex       3.39 (0.34) 
    Female 49.5 56.5 56.5 59.5 54.6 55.4  
    Male 50.5 43.5 43.5 40.6 45.4 44.6  
Race       17.07 (0.053) 
    White, non-Hispanic 63.8 44.5 44.1 42.9 45.7 45.3  
    Black, non-Hispanic 15.9 36.9 35.7 34.9 37.8 39.3  
    Hispanic 16.0 12.1 12.5 13.3 11.5 11.0  
    Other 4.4 6.6 7.8 9.0 4.9 4.4  
Age (years)       16.06 (0.20) 
    ≤14 36.3 21.8 24.8 22.1 21.1 18.8  
    15 14.6 24.9 23.2 23.9 27.6 24.8  
    16 32.7 25.4 22.8 27.8 24.8 26.4  
    17 12.7 24.4 24.6 24.1 22.6 26.7  
    ≥18 3.6 3.5 4.6 2.1 4.0 3.3  
   School Setting Home Setting  
   
 

 
 
Characteristic National Distribution (%) Study Sample (%) PAPI (%) CASI (%) PAPI (%) CASI (%) χ2 (p-value) 
Sex       3.39 (0.34) 
    Female 49.5 56.5 56.5 59.5 54.6 55.4  
    Male 50.5 43.5 43.5 40.6 45.4 44.6  
Race       17.07 (0.053) 
    White, non-Hispanic 63.8 44.5 44.1 42.9 45.7 45.3  
    Black, non-Hispanic 15.9 36.9 35.7 34.9 37.8 39.3  
    Hispanic 16.0 12.1 12.5 13.3 11.5 11.0  
    Other 4.4 6.6 7.8 9.0 4.9 4.4  
Age (years)       16.06 (0.20) 
    ≤14 36.3 21.8 24.8 22.1 21.1 18.8  
    15 14.6 24.9 23.2 23.9 27.6 24.8  
    16 32.7 25.4 22.8 27.8 24.8 26.4  
    17 12.7 24.4 24.6 24.1 22.6 26.7  
    ≥18 3.6 3.5 4.6 2.1 4.0 3.3  
Table 2.

Percentage of Students Reporting Engaging in Health Risk Behaviors by Condition and Adjusted Odds Ratios (AOR) and 95% Confidence Intervals (CI)

 Condition (%) Setting (Home Is Referent) Mode (CASI Is Referent)  
 
 

 

 
 
Variable School PAPI School CASI Home PAPI Home CASI AOR 95% CI AOR 95% CI Setting × Mode Interaction (Wald F
Rarely or never wore bicycle helmetsa 85.7 89.4 84.0 85.7 1.34 (0.97, 1.85) 0.81 (0.59, 1.11) 0.70 
Rarely or never wore seat belts 11.7 12.2 7.3 8.4 1.69* (1.33, 2.15) 0.90 (0.70, 1.14) 0.16 
Rode with a driver who had been drinking alcoholb 25.5 28.6 23.7 22.2 1.25* (1.05, 1.49) 0.97 (0.82, 1.15) 2.37 
Drove after drinking alcoholb 4.8 10.4 3.8 7.1 1.50* (1.10, 2.05) 0.48* (0.35, 0.65) 0.51 
Carried a weaponb 13.9 13.0 8.6 10.2 1.60* (1.30, 1.98) 0.94 (0.77, 1.15) 1.24 
Carried a gunb 3.1 3.9 1.2 1.9 2.40* (1.56, 3.70) 0.69 (0.47, 1.01) 0.20 
Did not go to school because of safety concernsb 3.7 7.2 5.1 5.6 0.98 (0.73, 1.32) 0.64* (0.47, 0.86) 4.72* 
In a physical fightc 34.1 34.6 28.1 30.5 1.29* (1.09, 1.53) 0.92 (0.78, 1.09) 0.36 
Injured in a physical fightc 4.0 4.1 3.0 3.9 1.19 (0.86, 1.66) 0.82 (0.59, 1.14) 0.82 
Dating violencec 8.4 10.6 5.3 9.7 1.32* (1.05, 1.66) 0.66* (0.52, 0.84) 2.63 
Ever forced to have sexual intercourse 5.3 8.5 3.8 6.3 1.36* (1.05, 1.76) 0.61* (0.47, 0.80) 0.03 
Seriously considered attempting suicidec 17.7 19.1 12.8 14.6 1.36* (1.14, 1.61) 0.90 (0.76, 1.06) 0.29 
Made a suicide planc 13.1 14.2 8.5 10.3 1.45* (1.17, 1.79) 0.86 (0.70, 1.06) 0.60 
Attempted suicidec 9.1 10.4 5.8 8.0 1.41* (1.08, 1.83) 0.78 (0.60, 1.00) 0.99 
Lifetime cigarette use 52.7 53.9 48.9 52.2 1.15 (0.97, 1.37) 0.93 (0.79, 1.11) 0.17 
Smoked a whole cigarette before age 13 years 13.7 16.4 11.4 14.2 1.23 (0.99, 1.52) 0.78* (0.63, 0.97) 0.09 
Current cigarette useb 13.7 17.7 16.3 14.7 1.02 (0.82, 1.28) 0.92 (0.73, 1.15) 4.08* 
Purchased cigarettes at a store or gas stationb 3.0 4.9 3.9 3.3 1.13 (0.78, 1.63) 0.83 (0.57, 1.22) 5.73* 
Tried to quit smokingd 56.2 64.2 54.7 59.2 1.16 (0.82, 1.65) 0.78 (0.55, 1.13) 0.05 
Current smokeless tobacco useb 4.1 4.4 2.2 3.0 1.83* (1.18, 2.83) 0.82 (0.53, 1.27) 0.08 
Current cigar useb 8.9 9.7 7.9 8.7 1.18 (0.92, 1.52) 0.88 (0.68, 1.12) 0.01 
Lifetime alcohol use 72.5 76.5 62.7 68.4 1.55* (1.31, 1.83) 0.81* (0.68, 0.96) 0.02 
Drank alcohol before age 13 27.2 28.4 20.2 22.7 1.44* (1.23, 1.68) 0.90 (0.76, 1.05) 0.58 
Current alcohol useb 36.0 42.9 29.5 31.0 1.57* (1.32, 1.86) 0.84* (0.71, 0.99) 2.34 
Episodic heavy drinkingb 16.7 20.7 15.0 14.4 1.39* (1.14, 1.69) 0.88 (0.72, 1.08) 3.29 
Lifetime marijuana use 35.7 37.7 32.2 34.0 1.24* (1.04, 1.48) 0.94 (0.78, 1.11) 0.07 
Tried marijuana before age 13 7.4 8.6 5.5 6.6 1.38* (1.07, 1.79) 0.82 (0.64, 1.05) 0.12 
Current marijuana useb 17.2 19.1 14.8 15.2 1.29* (1.04, 1.61) 0.93 (0.75, 1.15) 0.39 
Lifetime cocaine use 3.1 4.5 2.5 2.1 1.63* (1.03, 2.57) 0.87 (0.56, 1.34) 1.37 
Lifetime inhalant use 10.3 12.2 7.8 6.5 1.61* (1.23, 2.11) 0.97 (0.76, 1.23) 2.20 
Lifetime methamphetamine use 3.2 3.8 1.8 1.7 1.97* (1.23, 3.16) 0.95 (0.59, 1.51) 0.27 
Lifetime ecstasy use 3.2 4.0 2.6 2.7 1.38 (0.95, 2.00) 0.90 (0.63, 1.30) 0.26 
Lifetime illegal steroid use 2.3 3.0 2.6 1.3 1.33 (0.89, 1.99) 1.13 (0.77, 1.65) 5.84* 
Ever had sexual intercourse 47.7 48.3 43.0 45.2 1.27* (1.07, 1.52) 0.97 (0.81, 1.16) 0.01 
Had first sexual intercourse before age 13 9.4 7.6 6.1 6.4 1.54* (1.17, 2.03) 1.07 (0.81, 1.41) 0.91 
Had ≥4 sex partners during lifetime 13.3 13.4 10.9 11.3 1.36* (1.10, 1.69) 0.96 (0.77, 1.19) 0.10 
Currently sexually activee 32.5 34.6 29.7 31.0 1.21* (1.02, 1.45) 0.94 (0.79, 1.13) 0.34 
Alcohol or drug use before last sexual intercoursef 14.2 18.0 12.5 14.9 1.28 (0.89, 1.82) 0.77 (0.54, 1.09) 0.05 
Condom use during last sexual intercoursef 72.1 70.3 73.3 73.1 0.85 (0.64, 1.12) 0.99 (0.75, 1.32) 0.02 
Birth control pill use before last sexual intercoursef 9.4 9.7 11.9 17.2 0.70 (0.48, 1.01) 0.81 (0.57, 1.17) 1.46 
Has been pregnant or gotten someone pregnant 4.5 5.4 3.7 3.6 1.39 (1.00, 1.93) 0.93 (0.67, 1.29) 0.94 
Described themselves as overweight 31.7 30.2 30.4 32.5 0.96 (0.84, 1.09) 1.00 (0.88, 1.14) 2.08 
Were trying to lose weight 48.6 45.8 46.9 44.8 1.02 (0.90, 1.17) 1.14 (1.00, 1.30) 0.22 
Fasted to control weightb 12.3 15.4 11.0 10.3 1.37* (1.13, 1.68) 0.91 (0.75, 1.10) 2.33 
Took diet pills to control weightb 5.6 7.5 3.8 4.8 1.56* (1.12, 2.18) 0.79 (0.58, 1.09) 0.06 
Vomited or used laxatives to control weightb 3.8 5.5 2.7 3.8 1.41* (1.01, 1.96) 0.69 (0.50, 0.96) 0.01 
Participated in insufficient vigorous physical activityg 38.9 40.2 31.8 38.5 1.19* (1.02, 1.38) 0.86 (0.74, 1.01) 2.20 
Participated in insufficient moderate physical activityh 74.0 73.6 76.6 78.1 0.82 (0.70, 0.96) 0.98 (0.83, 1.15) 0.26 
Did strengthening exercises fewer than 3 days/week 50.1 51.9 47.8 52.4 1.01 (0.87, 1.17) 0.91 (0.78, 1.06) 0.37 
Watched more than 2 hours/day of TV 47.3 49.1 47.7 48.5 1.04 (0.90, 1.20) 0.96 (0.83, 1.11) 0.91 
Not enrolled in PE class 54.5 47.3 43.7 45.4 1.33 (0.94, 1.88) 1.18 (0.83, 1.68) 0.32 
Not active in PE classi 20.0 19.4 14.5 17.4 1.24 (0.89, 1.72) 0.99 (0.70, 1.38) 0.89 
Did not play on a sports team 46.7 44.8 41.3 42.3 1.15 (0.99, 1.34) 1.04 (0.90, 1.22) 0.39 
At risk for becoming overweight 17.7 14.0 15.2 16.5 0.99 (0.83, 1.18) 1.10 (0.93, 1.31) 5.31* 
Overweight 15.1 14.4 16.5 17.3 0.86 (0.72, 1.02) 0.98 (0.82, 1.17) 0.35 
 Condition (%) Setting (Home Is Referent) Mode (CASI Is Referent)  
 
 

 

 
 
Variable School PAPI School CASI Home PAPI Home CASI AOR 95% CI AOR 95% CI Setting × Mode Interaction (Wald F
Rarely or never wore bicycle helmetsa 85.7 89.4 84.0 85.7 1.34 (0.97, 1.85) 0.81 (0.59, 1.11) 0.70 
Rarely or never wore seat belts 11.7 12.2 7.3 8.4 1.69* (1.33, 2.15) 0.90 (0.70, 1.14) 0.16 
Rode with a driver who had been drinking alcoholb 25.5 28.6 23.7 22.2 1.25* (1.05, 1.49) 0.97 (0.82, 1.15) 2.37 
Drove after drinking alcoholb 4.8 10.4 3.8 7.1 1.50* (1.10, 2.05) 0.48* (0.35, 0.65) 0.51 
Carried a weaponb 13.9 13.0 8.6 10.2 1.60* (1.30, 1.98) 0.94 (0.77, 1.15) 1.24 
Carried a gunb 3.1 3.9 1.2 1.9 2.40* (1.56, 3.70) 0.69 (0.47, 1.01) 0.20 
Did not go to school because of safety concernsb 3.7 7.2 5.1 5.6 0.98 (0.73, 1.32) 0.64* (0.47, 0.86) 4.72* 
In a physical fightc 34.1 34.6 28.1 30.5 1.29* (1.09, 1.53) 0.92 (0.78, 1.09) 0.36 
Injured in a physical fightc 4.0 4.1 3.0 3.9 1.19 (0.86, 1.66) 0.82 (0.59, 1.14) 0.82 
Dating violencec 8.4 10.6 5.3 9.7 1.32* (1.05, 1.66) 0.66* (0.52, 0.84) 2.63 
Ever forced to have sexual intercourse 5.3 8.5 3.8 6.3 1.36* (1.05, 1.76) 0.61* (0.47, 0.80) 0.03 
Seriously considered attempting suicidec 17.7 19.1 12.8 14.6 1.36* (1.14, 1.61) 0.90 (0.76, 1.06) 0.29 
Made a suicide planc 13.1 14.2 8.5 10.3 1.45* (1.17, 1.79) 0.86 (0.70, 1.06) 0.60 
Attempted suicidec 9.1 10.4 5.8 8.0 1.41* (1.08, 1.83) 0.78 (0.60, 1.00) 0.99 
Lifetime cigarette use 52.7 53.9 48.9 52.2 1.15 (0.97, 1.37) 0.93 (0.79, 1.11) 0.17 
Smoked a whole cigarette before age 13 years 13.7 16.4 11.4 14.2 1.23 (0.99, 1.52) 0.78* (0.63, 0.97) 0.09 
Current cigarette useb 13.7 17.7 16.3 14.7 1.02 (0.82, 1.28) 0.92 (0.73, 1.15) 4.08* 
Purchased cigarettes at a store or gas stationb 3.0 4.9 3.9 3.3 1.13 (0.78, 1.63) 0.83 (0.57, 1.22) 5.73* 
Tried to quit smokingd 56.2 64.2 54.7 59.2 1.16 (0.82, 1.65) 0.78 (0.55, 1.13) 0.05 
Current smokeless tobacco useb 4.1 4.4 2.2 3.0 1.83* (1.18, 2.83) 0.82 (0.53, 1.27) 0.08 
Current cigar useb 8.9 9.7 7.9 8.7 1.18 (0.92, 1.52) 0.88 (0.68, 1.12) 0.01 
Lifetime alcohol use 72.5 76.5 62.7 68.4 1.55* (1.31, 1.83) 0.81* (0.68, 0.96) 0.02 
Drank alcohol before age 13 27.2 28.4 20.2 22.7 1.44* (1.23, 1.68) 0.90 (0.76, 1.05) 0.58 
Current alcohol useb 36.0 42.9 29.5 31.0 1.57* (1.32, 1.86) 0.84* (0.71, 0.99) 2.34 
Episodic heavy drinkingb 16.7 20.7 15.0 14.4 1.39* (1.14, 1.69) 0.88 (0.72, 1.08) 3.29 
Lifetime marijuana use 35.7 37.7 32.2 34.0 1.24* (1.04, 1.48) 0.94 (0.78, 1.11) 0.07 
Tried marijuana before age 13 7.4 8.6 5.5 6.6 1.38* (1.07, 1.79) 0.82 (0.64, 1.05) 0.12 
Current marijuana useb 17.2 19.1 14.8 15.2 1.29* (1.04, 1.61) 0.93 (0.75, 1.15) 0.39 
Lifetime cocaine use 3.1 4.5 2.5 2.1 1.63* (1.03, 2.57) 0.87 (0.56, 1.34) 1.37 
Lifetime inhalant use 10.3 12.2 7.8 6.5 1.61* (1.23, 2.11) 0.97 (0.76, 1.23) 2.20 
Lifetime methamphetamine use 3.2 3.8 1.8 1.7 1.97* (1.23, 3.16) 0.95 (0.59, 1.51) 0.27 
Lifetime ecstasy use 3.2 4.0 2.6 2.7 1.38 (0.95, 2.00) 0.90 (0.63, 1.30) 0.26 
Lifetime illegal steroid use 2.3 3.0 2.6 1.3 1.33 (0.89, 1.99) 1.13 (0.77, 1.65) 5.84* 
Ever had sexual intercourse 47.7 48.3 43.0 45.2 1.27* (1.07, 1.52) 0.97 (0.81, 1.16) 0.01 
Had first sexual intercourse before age 13 9.4 7.6 6.1 6.4 1.54* (1.17, 2.03) 1.07 (0.81, 1.41) 0.91 
Had ≥4 sex partners during lifetime 13.3 13.4 10.9 11.3 1.36* (1.10, 1.69) 0.96 (0.77, 1.19) 0.10 
Currently sexually activee 32.5 34.6 29.7 31.0 1.21* (1.02, 1.45) 0.94 (0.79, 1.13) 0.34 
Alcohol or drug use before last sexual intercoursef 14.2 18.0 12.5 14.9 1.28 (0.89, 1.82) 0.77 (0.54, 1.09) 0.05 
Condom use during last sexual intercoursef 72.1 70.3 73.3 73.1 0.85 (0.64, 1.12) 0.99 (0.75, 1.32) 0.02 
Birth control pill use before last sexual intercoursef 9.4 9.7 11.9 17.2 0.70 (0.48, 1.01) 0.81 (0.57, 1.17) 1.46 
Has been pregnant or gotten someone pregnant 4.5 5.4 3.7 3.6 1.39 (1.00, 1.93) 0.93 (0.67, 1.29) 0.94 
Described themselves as overweight 31.7 30.2 30.4 32.5 0.96 (0.84, 1.09) 1.00 (0.88, 1.14) 2.08 
Were trying to lose weight 48.6 45.8 46.9 44.8 1.02 (0.90, 1.17) 1.14 (1.00, 1.30) 0.22 
Fasted to control weightb 12.3 15.4 11.0 10.3 1.37* (1.13, 1.68) 0.91 (0.75, 1.10) 2.33 
Took diet pills to control weightb 5.6 7.5 3.8 4.8 1.56* (1.12, 2.18) 0.79 (0.58, 1.09) 0.06 
Vomited or used laxatives to control weightb 3.8 5.5 2.7 3.8 1.41* (1.01, 1.96) 0.69 (0.50, 0.96) 0.01 
Participated in insufficient vigorous physical activityg 38.9 40.2 31.8 38.5 1.19* (1.02, 1.38) 0.86 (0.74, 1.01) 2.20 
Participated in insufficient moderate physical activityh 74.0 73.6 76.6 78.1 0.82 (0.70, 0.96) 0.98 (0.83, 1.15) 0.26 
Did strengthening exercises fewer than 3 days/week 50.1 51.9 47.8 52.4 1.01 (0.87, 1.17) 0.91 (0.78, 1.06) 0.37 
Watched more than 2 hours/day of TV 47.3 49.1 47.7 48.5 1.04 (0.90, 1.20) 0.96 (0.83, 1.11) 0.91 
Not enrolled in PE class 54.5 47.3 43.7 45.4 1.33 (0.94, 1.88) 1.18 (0.83, 1.68) 0.32 
Not active in PE classi 20.0 19.4 14.5 17.4 1.24 (0.89, 1.72) 0.99 (0.70, 1.38) 0.89 
Did not play on a sports team 46.7 44.8 41.3 42.3 1.15 (0.99, 1.34) 1.04 (0.90, 1.22) 0.39 
At risk for becoming overweight 17.7 14.0 15.2 16.5 0.99 (0.83, 1.18) 1.10 (0.93, 1.31) 5.31* 
Overweight 15.1 14.4 16.5 17.3 0.86 (0.72, 1.02) 0.98 (0.82, 1.17) 0.35 

note.—Models adjusted for sex, race/ethnicity, and age. Race/ethnicity categories included non-Hispanic white, non-Hispanic black, Hispanic, and other. Age was entered as a five-level categorical variable with the following categories: ≤14 years, 15 years, 16 years, 17 years, and ≥18 years.

a

Among students who had ridden a bicycle during the 12 months preceding the survey.

b

During the 30 days preceding the survey.

c

During the 12 months preceding the survey.

d

Among students who had smoked during the 30 days preceding the survey, those who tried to quit smoking during the 12 months preceding the survey.

e

Sexual intercourse during the three months preceding the survey.

f

Among currently sexually active students.

g

Did not exercise or participate in physical activities that made students sweat and breathe hard for ≥20 minutes on ≥3 of the 7 days preceding the survey.

h

Did not participate in physical activities that did not make students sweat and breathe hard for ≥30 minutes on ≥5 of the 7 days preceding the survey.

i

Among the students enrolled in PE class.

*

p < .05.

For each risk behavior, an additional analysis tested the effect of the interaction of setting and mode by adding to the model the cross-product of setting and mode. When the interaction reached significance at the p < .05 level, separate models were run stratified by setting. Similarly, for each risk behavior, a separate set of analyses tested the effects of the interaction of setting with student sex, age, and race/ethnicity. These analyses were conducted by adding simultaneously to each model the cross-product of setting and each demographic variable. For each interaction that reached significance at the p < .05 level, separate models were run stratified by the demographic variable.

Summary analyses were conducted by creating a count variable for each of seven risk behavior categories: injury-related behaviors, tobacco use, alcohol use, drug use, sexual behaviors, weight control behaviors, and physical activity. Each composite only included behaviors for which it was possible for all students to have a “yes” response. For example, condom use at last sexual intercourse was not included in the sexual behavior composite because this behavior is reported only among students who are currently sexually active. Students who had missing data on one or more behaviors included in a given composite were excluded from that composite. Each composite was then used as a dependent variable in a series of Poisson regression analyses using PROC LOGLINK in SUDAAN. As in the logistic regression analyses, each model examined the simultaneous effects of setting and mode while controlling for sex, age, and race/ethnicity, and an additional set of analyses was conducted to examine the effect of the setting-by-mode interaction.

Results

Of the 4,506 usable questionnaires in the sample, 1,153 were administered in the school PAPI condition, 1,144 in the school CASI condition, 1,157 in the home PAPI condition, and 1,052 in the home CASI condition. Using AAPOR response rate 1 (AAPOR 2004), response rates within each condition were similar: 78.2 percent for school PAPI, 74.0 percent for school CASI, 76.7 percent for home PAPI, and 75.7 percent for home CASI.

The demographic characteristics of the study sample differed from the national distribution of ninth- and eleventh-grade students (U.S. Bureau of the Census 2002). In the study sample, female and black non-Hispanic students were overrepresented, and white non-Hispanic students were underrepresented (table 1). Students in the study sample also tended to be older than those in the national distribution, which is likely a function of data collection timing: the national data were collected in the fall, whereas the study was conducted in the spring.

Student demographic characteristics across the four survey conditions did not differ significantly by sex or age (table 1). Differences by race/ethnicity approached significance (p = .053); further analyses revealed this was attributed to differences by condition in the “other” race/ethnicity category. No difference by condition was found in the percentage of students who reported missing school for any reason on one or more of the 30 days preceding the survey (χ2 = 1.93, p = .59) and missing school without permission on one or more of the 30 days preceding the survey (χ2 = 0.23, p = .97).

An analysis of item nonresponse by condition revealed that, although the level of nonresponse was low overall, the proportion of missing items varied by condition. Questionnaires completed in the CASI mode had an average of 0.6 percent of items missing in both the home and school settings; those completed in the PAPI mode had slightly higher averages (school = 2.1 percent, home = 1.6 percent).

Table 2 provides the prevalence of each self-reported risk behavior by study condition and the adjusted odds ratio (AOR) for the association of setting and mode with each risk behavior. After adjusting for sex, race/ethnicity, age, and mode, setting was associated significantly with the reporting of 30 of the 55 risk behaviors. For every risk behavior with a significant setting main effect, the odds of reporting the risk behavior were greater among students in the school setting than in the home setting. Mode was associated significantly with the reporting of 7 of the 55 risk behaviors controlling for sex, race/ethnicity, age, and setting. In every model with a significant mode main effect, students in the PAPI mode had lower odds of reporting the risk behavior than students in the CASI mode.

The setting-by-mode interaction was statistically significant in 5 of the 55 models tested (table 2, last column). Models stratified by setting showed that for three of the five behaviors (not going to school because of safety concerns, current cigarette use, and purchased cigarettes in a store), the odds of reporting the behavior were lower in the PAPI than in the CASI mode in the school setting, whereas no significant differences in odds by mode in the home setting were identified (data not shown). For one of the five variables (being at risk for becoming overweight), the odds were higher in the PAPI than in the CASI mode in the school setting, whereas no significant difference in odds by mode in the home setting were found. The odds of lifetime illegal steroid use were greater among students in the PAPI mode than in the CASI mode at home (AOR = 2.05; 95 percent confidence interval [CI] = 1.06, 3.95), but no significant difference in odds by mode at school was found (AOR = 0.74; 95% CI = 0.45, 1.20).

Analyses examining whether setting effects varied by respondent’s sex, age, and race/ethnicity yielded few significant results. These interactions could not be assessed for two risk behaviors (tried marijuana before age 13 and current inhalant use) because errors caused by zero values in frequency table cells occurred during modeling. Of the remaining 53 behaviors, 7 showed a significant setting-by-sex interaction, with Wald Fs ranging from 3.92 (p = .05) for lifetime cigarette use to 8.01 (p = .01) for having driven after drinking alcohol. Stratified analyses revealed that for all of these behaviors, which also included having drunk alcohol before age 13, lifetime marijuana use, current marijuana use, ever had sexual intercourse, and currently sexually active, male students had greater odds of reporting these behaviors in the school setting than in the home setting, whereas female students showed no significant setting effect. Specifically, AORs for male students ranged from 1.33 (95% CI = 1.05, 1.67) for lifetime cigarette use to 2.25 (95% CI = 1.50, 3.36) for having driven after drinking alcohol.

Two of the 53 behaviors showed a significant setting-by-age interaction: lifetime alcohol use (Wald F = 2.71, p = .03) and current alcohol use (Wald F = 4.40, p = .002). For the other behaviors, Wald Fs all were greater than p = .05. Stratified analyses for lifetime and current alcohol use revealed that, for both behaviors, students aged 16 years and younger had greater odds of reporting these behaviors in the school setting than in the home setting, whereas students aged 17 years and older showed no significant setting effect. For example, for lifetime alcohol use, AORs for students aged ≤14 years, 15 years, and 16 years were 2.00 (95% CI = 1.44, 2.78), 1.77 (95% CI = 1.31, 2.40), and 1.52 (95% CI = 1.08, 2.13), respectively, while AORs for students aged 17 years and ≥18 years were 1.05 (95% CI = 0.78, 1.43) and 1.25 (95% CI = 0.59, 2.61), respectively.

Two of the 53 behaviors showed a significant race-by-setting interaction (Wald F = 2.69, p = .05 for never or rarely wore bicycle helmets; Wald F = 2.77, p = .04 for ever had sexual intercourse). Stratified analyses revealed that black students had greater odds (AOR = 2.60, 95% CI = 1.55, 4.36) than students in other race/ethnic groups to report not using bicycle helmets in the school setting than in the home setting, while no significant setting effect was seen for the other race/ethnic groups. A similar pattern of results was found for ever having had sexual intercourse, although Hispanic students also showed significant setting effects for that variable (data not shown).

Summary analyses using behavior category composites revealed that, controlling for sex, age, race/ethnicity, and mode, setting was associated significantly with five of the seven composites (injury-related behaviors, alcohol use, drug use, sexual behavior, and weight control behaviors). Adjusted incidence density ratios (IDRs) for setting ranged from 1.08 (95% CI = 1.01, 1.15) for the weight control composite to 1.32 (95% CI = 1.19, 1.47) for the injury composite. In addition, three of these composites (injury-related behaviors, alcohol use, and drug use) also showed significant mode effects when controlling for sex, age, race/ethnicity, and mode, with IDRs for mode ranging from 0.84 (95% CI = 0.76, 0.94) for the injury composite to 0.91 (95% CI = 0.85, 0.98) for the alcohol use composite. All composite results that reached significance followed the same pattern as the significant results for the individual behaviors. That is, the mean number of reported behaviors in each composite was greater among students who completed questionnaires at school than among students who completed questionnaires at home, and the mean number of reported behaviors was lower for PAPI mode students than for CASI mode students. None of the setting-by-mode interactions for the composites reached significance.

ANCILLARY VARIABLES

About half of all students (51.3 percent) reported that they preferred to complete the survey on the computer, while 13.2 percent reported preferring paper and pencil and 35.5 percent said they had no preference. The preferred mode varied significantly by condition (χ2 = 509.03, p < .001) (table 3). Students in the CASI mode were more likely than students in the PAPI mode to indicate a preference for the computer, whereas students in the PAPI mode were more likely than students in the CASI mode to either prefer PAPI or have no preference. When asked if they thought answers on the survey could be linked with their name (perceived anonymity), 8.7 percent of all students reported “yes,” and 22.2 percent reported “not sure.” This distribution did not vary significantly by condition (χ2 = 4.24, p = .64) (table 3). The percentage of students who reported someone could see their answers during the survey (perceived privacy) differed significantly by condition (χ2 = 171.11, p < .001). Students in the school setting were more likely to report someone could see their answers than students in the home setting. The percentage of students who reported someone could see their answers also was greater in the CASI mode than in the PAPI mode at school, but not at home. The percentage of students who disagreed with the statement “most people can be trusted” (trust) varied significantly by condition (χ2 =13.44, p = .004), with the greater frequency at school compared with at home.

Table 3.

Frequency of Ancillary Variable Responses by Setting and Mode

 Condition 
 
 
Response Overall (%) School PAPI (%) School CASI (%) Home PAPI (%) Home CASI (%) χ2 (p-value) 
Preferred mode for taking surveya      509.03 (<0.001) 
    Computer 51.3 35.7 71.4 26.0 73.1  
    Paper and pencil 13.2 20.6 4.5 25.4 1.9  
    It would not matter to me 35.5 43.6 24.2 48.6 25.0  
No perceived anonymityb      4.24 (0.64) 
    Yes 8.7 9.1 9.4 8.1 7.9  
    No 69.1 67.4 68.9 71.1 69.1  
    Not sure 22.2 23.5 21.7 20.7 23.0  
No perceived privacyc      171.11 (<0.001) 
    Yes 14.1 15.5 22.7 10.7 7.0  
    No 68.7 63.9 47.5 79.5 84.9  
    Not sure 17.2 20.6 29.7 9.8 8.1  
Disagree that most people can be trustedd 38.6 43.3 40.1 34.0 36.7 13.44 (0.004) 
Frequency of computer use at school, home, or worke      5.36 (0.50) 
    Every day or nearly every day 61.9 60.6 61.1 62.9 63.1  
    A few times a week 24.8 24.4 24.9 25.7 24.1  
    A few times a month or less 13.3 15.0 14.0 11.3 12.8  
Ever used a computer to take a survey or testf 70.5 72.1 67.4 75.4 66.8 15.50 (0.002) 
Agree computer surveys make me nervousg 9.0 12.3 7.7 10.1 5.7 26.46 (<0.001) 
Agree computer surveys prohibit privacyh 19.4 22.0 19.5 19.1 16.7 7.44 (0.06) 
 Condition 
 
 
Response Overall (%) School PAPI (%) School CASI (%) Home PAPI (%) Home CASI (%) χ2 (p-value) 
Preferred mode for taking surveya      509.03 (<0.001) 
    Computer 51.3 35.7 71.4 26.0 73.1  
    Paper and pencil 13.2 20.6 4.5 25.4 1.9  
    It would not matter to me 35.5 43.6 24.2 48.6 25.0  
No perceived anonymityb      4.24 (0.64) 
    Yes 8.7 9.1 9.4 8.1 7.9  
    No 69.1 67.4 68.9 71.1 69.1  
    Not sure 22.2 23.5 21.7 20.7 23.0  
No perceived privacyc      171.11 (<0.001) 
    Yes 14.1 15.5 22.7 10.7 7.0  
    No 68.7 63.9 47.5 79.5 84.9  
    Not sure 17.2 20.6 29.7 9.8 8.1  
Disagree that most people can be trustedd 38.6 43.3 40.1 34.0 36.7 13.44 (0.004) 
Frequency of computer use at school, home, or worke      5.36 (0.50) 
    Every day or nearly every day 61.9 60.6 61.1 62.9 63.1  
    A few times a week 24.8 24.4 24.9 25.7 24.1  
    A few times a month or less 13.3 15.0 14.0 11.3 12.8  
Ever used a computer to take a survey or testf 70.5 72.1 67.4 75.4 66.8 15.50 (0.002) 
Agree computer surveys make me nervousg 9.0 12.3 7.7 10.1 5.7 26.46 (<0.001) 
Agree computer surveys prohibit privacyh 19.4 22.0 19.5 19.1 16.7 7.44 (0.06) 
a

If you had a choice of taking this survey using a computer or taking this survey using paper and pencil, which would you choose?

b

Do you believe that the answers you gave in this survey will be linked with your name?

c

While taking this survey, could anyone besides you see your answers?

d

How much do you agree or disagree with the following statement? Most people can be trusted.

e

How often do you use a computer at school, home, or work? Include activities such as being on the Internet, computer games, and e-mail.

f

(Before today), have you ever used a computer to take a survey or test?

g

How much do you agree or disagree with the following statement? Using a computer to take this survey would make (made) me feel nervous.

h

How much do you agree or disagree with the following statement? Using a computer to take this survey would keep (keeps) this survey from being private.

Overall, 61.9 percent of students used a computer daily or nearly daily, whereas 13.3 percent of students used a computer a few times a month or less. This distribution did not vary significantly by condition (χ2 = 5.36; p = .50). The percentage of students who had ever used a computer to take a survey or test varied significantly by condition (χ2 = 15.50, p = .002), with the percentage being greater in the PAPI mode (72.1 percent at school and 75.4 percent at home) than in the CASI mode (67.4 percent at school and 66.8 percent at home). More students in the PAPI than in the CASI mode agreed that computer surveys make them nervous (χ2 = 26.46, p < .001). The percentage of students who agreed that computer surveys prohibited privacy did not differ significantly by condition, although the analysis did approach significance (χ2 = 7.44, p = .06). The greatest frequency was observed in the school PAPI condition (22.0 percent), and the lowest frequency was observed in the home CASI condition (16.7 percent).

For the 30 risk behaviors with a significant setting effect, we tested whether the association could be explained by adding to the model the three ancillary variables that might explain why the reporting of risk behaviors varied by setting (perceived anonymity, perceived privacy, and trust). The main effect of setting became nonsignificant when these three variables were added to models for 5 of the 30 behaviors examined (drove after drinking alcohol, ever forced to have sexual intercourse, attempted suicide, lifetime cocaine use, and vigorous physical activity), suggesting the significant association of setting with these five risk behaviors may be explained by the ancillary variables.

For the seven risk behaviors with a significant mode effect, we tested whether the association could be explained by adding to the model the seven ancillary variables that might explain why the reporting of risk behaviors varied by mode (perceived anonymity, perceived privacy, trust, frequency of computer use, previous use of computers for surveys or tests, comfort with computers, and computer privacy). The ancillary variables did not explain the association for any of the behaviors examined.

We conducted parallel analyses for the five composite variables that showed a significant setting effect and the three composite variables that showed a significant mode effect. In no case did adding the ancillary variables to the model explain the association of the composite with setting or mode.

Discussion

More than half of the behaviors examined in this study, and five of seven behavior composites, showed a significant setting effect. In every case, students who completed questionnaires in school were more likely to report risk behaviors than were students who completed questionnaires at home. This finding is consistent with other studies comparing the prevalence of risk behaviors in school and home settings (Gfroerer, Wright, and Kopstein 1997; Kann et al. 2002; Rootman and Smart 1985). Mode effects were weaker than setting effects, with only 13 percent of behaviors examined, and only three of seven behavior composites, showing a significant mode effect. In every case, students who completed questionnaires on the computer were more likely to report the behaviors than were students who completed paper-and-pencil instruments. Contrary to expectations, only a few behaviors, and none of the composites, showed a significant setting-by-mode interaction. None of the significant interactions followed the pattern suggested by the literature, that students using CASI in the home setting would be more likely to report risk behaviors than those using PAPI at home (Turner et al. 1998; Wright, Aquilino, and Supple 1998), while no such effect would be seen in the school setting (Beebe et al. 1998; Hallfors et al. 2000).

Regarding the types of behaviors most strongly affected by setting and mode, the results of this study are similar to those of previous studies. Kann et al. (2002) found setting effects were strongest for illegal or socially stigmatized behaviors, such as drug use and sexual intercourse before age 13, whereas no significant setting effects were found for less sensitive behaviors, such as physical activity. Similarly, setting effects in the current study were less likely to be seen for behaviors related to tobacco use and physical activity than for behaviors related to violence, suicide, alcohol use, drug use, sexual behaviors, and unhealthy weight control. In addition, behaviors that showed mode effects in the current study all were related to injury, alcohol, and drug use, rather than physical activity, tobacco use, and weight control. These results are consistent with studies that demonstrate that mode effects are stronger for more sensitive behaviors (Turner et al. 1998; Wright, Aquilino, and Supple 1998).

Despite what has been suggested in the literature, this study generally found that perceived anonymity, perceived privacy, trust, and comfort and experience with computers did little to explain setting and mode effects. This null result might be explained by the finding that these ancillary variables did not always vary by setting and mode in the assumed direction. For example, Sudman (2001) noted that differences in survey results might best be explained by greater perceived anonymity in a school-based PAPI survey than in a CASI survey in which the data collector knows the respondent’s identity, as is true in the home setting. Supple, Aquilino, and Wright (1999) found adolescents who completed questionnaires on a computer perceived more response anonymity than adolescents who completed paper-and-pencil questionnaires. The current study, however, found perceived anonymity did not vary by setting or mode. Similarly, although Harrison (2001) suggested that lower reported prevalence of risk behaviors could stem from less perceived privacy in the home than at school, this study found perceived privacy to be lower among students who completed surveys at school than at home. Even ancillary variables that varied by setting or mode generally did not modify the effect of setting or mode on the reporting of risk behaviors. Students’ level of trust was higher in the school setting than the home setting, but that variable did not modify the association between setting and the reporting of risk behaviors. This is contrary to previous research, which found mistrust modified mode effects among young adults (Wright, Aquilino, and Supple 1998). In addition, although students in the PAPI mode were more likely than students in the CASI mode to agree that computer surveys made them nervous, this variable did not modify the association of mode with the reporting of risk behaviors.

One major limitation of this study is that the comparison between the school and home settings is confounded with group versus individual administration. All students in the school setting completed questionnaires in a group, whereas all students in the home setting completed questionnaires individually. This study cannot determine, therefore, whether students in the home setting are less likely to report risk behaviors because being at home increases the chance that a parent might see their responses or because they are completing the questionnaire individually and the data collector knows their name. Since perceived anonymity did not vary by setting, the former explanation may have more validity. This study did not assess whether a parent was present during survey administration, so it is not possible to determine the effect of parental presence on the reporting of risk behavior, but parental presence during survey administration has been shown to be associated with lower reported risk behaviors (Aquilino, Wright, and Supple 2000; Gfroerer 1985; Schutz, Chilcoat, and Anthony 1994).

This study also cannot determine whether students are more likely to report risk behaviors in a school setting because they are in the presence of peers or because of the perceived anonymity that any group provides. For example, asking students about tobacco use in a classroom setting where they are surrounded by peers might remind respondents of instances in which smoking occurred, assuming they smoke with their peers (Vilsaint et al. 2003). Others also have suggested that, to gain increased status among peers, students might overreport drug use when in the presence of peers (Percy et al. 2005). Thus, the influence of the school setting on reported risk behaviors likely goes beyond the perceived anonymity of being in a group. Further research is needed to disentangle the effect of setting with the effect of group versus individual administration.

This study also is limited by the use of a convenience sample. The demographic characteristics of the study sample differed from those of the national distribution of ninth- and eleventh-grade students (U.S. Bureau of the Census 2002), although analyses controlled for these characteristics. In addition, because this study sampled students in grades 9 and 11 only, the ages of the students not only were restricted to a narrow range but also were not evenly distributed. These factors limit the study’s ability to detect age differences in the results. Future research using a broader and more evenly distributed age range could examine this issue.

Another limitation is the variability of survey logistics within each condition. Not all students assigned to the home condition completed questionnaires in their homes, and not all students assigned to the school CASI condition completed questionnaires in a classroom. In the school CASI condition, the location could have affected the distance between students’ computers, which has been shown to affect the reporting of risk behaviors (Beebe et al. 1998). Unfortunately, we were not able to collect usable data on these logistics, so we cannot determine their effects.

Social measurement research assumes respondents underreport sensitive behaviors, such as health risk behaviors, when data are collected via self-report. Therefore, higher prevalence estimates are considered more valid (Gans and Brindis 1995; Moskowitz 2004; Turner et al. 1998). Biochemical measures of smoking prevalence provide some empirical evidence of this, at least for tobacco use (Hedges and Jarvis 1998). Consequently, when methodological factors such as setting and mode are shown to increase prevalence estimates of health risk behaviors, these factors should be considered when planning surveys. This study has shown that prevalence estimates of many health risk behaviors were higher in a school setting than in a home setting. In addition, this study has shown that setting has a greater effect on risk behavior reporting than does mode; this finding is consistent with a recent analysis of self-reported tobacco use (Moskowitz 2004). Use of CASI rather than PAPI in school settings, therefore, might not be justified given the complicated logistics and increased cost.

The authors gratefully acknowledge the valuable suggestions of the expert panel convened for this study: Lara Akinbami, Brett Brown, Kathryn Chandler, Sonia Chessen, James Colliver, Michael Errecart (deceased), Joe Gfroerer, Gary Giovino, Art Hughes, Ronaldo Iachan, Meredith Kelsey, Bronwyn Nichols, Patrick O’Malley, Terry Pechacek, Jim Scanlon, Kenneth Schoendorf, Judy Thorne, and Charles Turner.

References

American Association for Public Opinion Research (AAPOR).
Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys
 , 
2004
3d ed
Lenexa, KS
AAPOR
Aquilino
William S.
Wright
Debra L.
Supple
Andrew J.
Response Effects Due to Bystander Presence in CASI and Paper-and-Pencil Surveys of Drug Use and Alcohol Use
Substance Use and Misuse
 , 
2000
, vol. 
35
 (pg. 
845
-
67
)
Beebe
Timothy J.
Harrison
Patricia A.
McCrae
James A.
Jr
Anderson
Ronald E.
Fulkerson
Jayne A.
An Evaluation of Computer-Assisted Self-Interviews in a School Setting
Public Opinion Quarterly
 , 
1998
, vol. 
62
 (pg. 
623
-
32
)
Brener
Nancy D.
Anne Grunbaum
Jo
Kann
Laura
McManus
Tim
Ross
Jim
Assessing Health Risk Behaviors among Adolescents: The Effect of Question Wording and Appeals for Honesty
Journal of Adolescent Health
 , 
2004
, vol. 
35
 (pg. 
91
-
100
)
Brener
Nancy D.
Kann
Laura
Kinchen
Steven A.
Anne Grunbaum
Jo
Whalen
Laura
Eaton
Danice
Hawkins
Joseph
Ross
James G.
Methodology of the Youth Risk Behavior Surveillance System
Morbidity and Mortality Weekly Report
 , 
2004
, vol. 
53
 
RR-12
(pg. 
1
-
13
)
Coggeshall
Mark B.
Kingery
Paul M.
Cross-Survey Analysis of School Violence and Disorder
Psychology in the Schools
 , 
2001
, vol. 
38
 (pg. 
107
-
16
)
Cowan
Charles D.
Coverage, Sample Design, and Weighting in Three Federal Surveys
Journal of Drug Issues
 , 
2001
, vol. 
31
 (pg. 
599
-
614
)
Fendrich
Michael
Johnson
Timothy P.
Examining Prevalence Differences in Three National Surveys of Youth: Impact of Consent Procedures, Mode, and Editing Rules
Journal of Drug Issues
 , 
2001
, vol. 
31
 (pg. 
615
-
42
)
Fowler
Floyd Jackson
Jr
Stringfellow
Vickie L.
Learning from Experience: Estimating Teen Use of Alcohol, Cigarettes, and Marijuana from Three Survey Protocols
Journal of Drug Issues
 , 
2001
, vol. 
31
 (pg. 
643
-
64
)
Gans
Janet E.
Brindis
Claire D.
Choice of Research Setting in Understanding Adolescent Health Problems
Journal of Adolescent Health
 , 
1995
, vol. 
17
 (pg. 
306
-
13
)
Gfroerer
Joseph.
Rouse
Beatrice A.
Kozel
Nicholas J.
Richards
Louise G.
Influence of Privacy on Self-Reported Drug Use by Youths
Self-Report Methods of Estimating Drug Use: Meeting Current Challenges to Validity
 , 
1985
Rockville, MD
National Institute on Drug Abuse
(pg. 
22
-
30
DHHS Publication no. (ADM) 85–1402.
Gfroerer
Joseph
Wright
Doug
Kopstein
Andrea
Prevalence of Youth Substance Use: The Impact of Methodological Differences between Two National Surveys
Drug and Alcohol Dependence
 , 
1997
, vol. 
47
 (pg. 
19
-
30
)
Grunbaum
Jo Anne
Kann
Laura
Kinchen
Steve
Ross
James
Hawkins
Joseph
Lowry
Richard
Harris
William A.
McManus
Tim
Chyen
David
Collins
Janet
Youth Risk Behavior Surveillance—United States, 2003
Morbidity and Mortality Weekly Report
 , 
2004
May
21
, vol. 
53
 
SS-2
(pg. 
1
-
96
CDC Surveillance Summaries
Hallfors
Denise
Khatapoush
Shereen
Kadushin
Charles
Watson
Kim
Saxe
Leonard
A Comparison of Paper vs. Computer-Assisted Self Interview for School Alcohol, Tobacco, and Other Drug Surveys
Evaluation and Program Planning
 , 
2000
, vol. 
23
 (pg. 
149
-
55
)
Harrison
Lana D.
Understanding the Difference in Youth Drug Prevalence Rates Produced by the MTF, NHSDA, and YRBS Studies
Journal of Drug Issues
 , 
2001
, vol. 
31
 (pg. 
665
-
94
)
Hedges
Barry
Jarvis
Martin
Prescott-Clarke
Patricia
Primatesta
Paola
Cigarette Smoking
Health Survey for England: The Health of Young People
 , 
1995
London
The Stationery Office
(pg. 
191
-
221
)
Johnston
L. D.
O’Malley
P. M.
Bachman
J. G.
Schulenberg
J. E.
Monitoring the Future: National Results on Adolescent Drug Use: Overview of Key Findings, 2003
 , 
2004
Bethesda, MD
National Institute on Drug Abuse
 
NIH Publication no. 04–5506.
Kann
Laura
Brener
Nancy D.
Warren
Charles W.
Collins
Janet L.
Giovino
Gary A.
An Assessment of the Effect of Data Collection Setting on the Prevalence of Health Risk Behaviors among Adolescents
Journal of Adolescent Health
 , 
2002
, vol. 
31
 (pg. 
327
-
35
)
Moskowitz
Joel M.
Assessment of Cigarette Smoking and Smoking Susceptibility among Youth: Telephone Computer-Assisted Self-Interviews Versus Computer-Assisted Telephone Interviews
Public Opinion Quarterly
 , 
2004
, vol. 
68
 (pg. 
565
-
87
)
National Institute on Drug Abuse
National Household Survey on Drug Abuse: Main Findings 1990
 , 
1991
Rockville, MD
U.S. Department of Health and Human Services, Public Health Service
 
DHHS Publication no. (ADM) 91–1788.
Needle
Richard
McCubbin
Hamilton
Lorence
Jon
Hochhauser
Mark
Reliability and Validity of Adolescent Self-Reported Drug Use in a Family-Based Study: A Methodological Report
International Journal of the Addictions
 , 
1983
, vol. 
18
 (pg. 
901
-
12
)
Peduzzi
Peter
Concato
John
Kemper
Elizabeth
Holford
Theodore R.
Feinstein
Alvan R.
A Simulation Study of the Number of Events per Variable in Logistic Regression Analysis
Journal of Clinical Epidemiology
 , 
1996
, vol. 
49
 (pg. 
1373
-
79
)
Percy
Andrew
McAlister
Siobhan
Higgins
Kathryn
McCrystal
Patrick
Thornoton
Maeve
Response Consistency in Young Adolescents’ Drug Use Self-Reports: A Recanting Rate Analysis
Addiction
 , 
2005
, vol. 
100
 (pg. 
189
-
96
)
Rootman
Irving
Smart
Reginald G.
A Comparison of Alcohol, Tobacco and Drug Use as Determined from Household and School Surveys
Drug and Alcohol Dependence
 , 
1985
, vol. 
16
 (pg. 
89
-
94
)
Schutz
Christian G.
Chilcoat
Howard D.
Anthony
James C.
Breach of Privacy in Surveys on Adolescent Drug Use: A Methodological Inquiry
International Journal of Methods in Psychiatric Research
 , 
1994
, vol. 
4
 (pg. 
183
-
88
)
Substance Abuse and Mental Health Services Administration (SAMHSA)
Overview of Findings from the 2003 National Survey on Drug Use and Health
 , 
2004
Rockville, MD
SAMHSA
 
Office of Applied Studies, NSDUH series H-24, DHHS Publication no. SMA 043963.
Sudman
Seymour.
Examining Substance Abuse Data Collection Methodologies
Journal of Drug Issues
 , 
2001
, vol. 
31
 (pg. 
695
-
716
)
Supple
Andrew J.
Aquilino
William S.
Wright
Debra L.
Collecting Sensitive Self-Report Data with Laptop Computers: Impact on the Response Tendencies of Adolescents in a Home Interview
Journal of Research on Adolescence
 , 
1999
, vol. 
9
 (pg. 
467
-
88
)
Tseng
Hsu-Min
Tiplady
Brian
Macleod
Hamish A.
Wright
Peter
Computer Anxiety: A Comparison of Pen-Based Personal Digital Assistants, Conventional Computer and Paper Assessment of Mood and Performance
British Journal of Psychology
 , 
1998
, vol. 
89
 (pg. 
599
-
610
)
Turner
C. F.
Ku
L.
Rogers
S. M.
Lindberg
L. D.
Pleck
J. H.
Sonenstein
F. L.
Adolescent Sexual Behavior, Drug Use, and Violence: Increased Reporting with Computer Survey Technology
Science
 , 
1998
, vol. 
280
 (pg. 
867
-
73
)
U.S. Bureau of the Census.
Single Grade of Enrollment and High School Graduation Status for People 3 Years Old and Over, by Age (Single Years for 3 to 24 Years), Sex, Race, and Hispanic Origin: October 2002
2002
 
Vilsaint
M.
Haviland
M. L.
Allen
J.
Hersey
J. C.
Davis
K. C.
Mowery
P. D.
Do Peer Context Cues Explain Mode Effects in Youth Surveys?
2003
Paper presented at the annual meeting of the Society for Research on Nicotine and Tobacco, New Orleans, LA.
Wright
Debra L.
Aquilino
William S.
Supple
Andrew J.
A Comparison of Computer-Assisted and Paper-and-Pencil Self-Administered Questionnaires in a Survey on Smoking, Alcohol, and Drug Use
Public Opinion Quarterly
 , 
1998
, vol. 
62
 (pg. 
331
-
53
)