Method

Respondents

A sample of email addresses representing users and potential users of Predictive Services products and services was compiled using key contact and snowball approaches. Sources of addresses included the NPSG, a list of incident information officers, the National Wildland Fire Management Directory, contacts at various Federal agencies, and online directories. We compiled a national list of 2,999 Federal contacts. This initial list was composed of Federal sector fire management personnel within the selected agencies (see respondent description below) with a focus on assuring that fire management officers, fire behavior analysts, incident meteorologists, GACC managers, regional coordinators, public affairs and information officers, dispatchers, incident management team members, fire use personnel, and aviation personnel were included. A census listing was not available through any of the agencies, so a compilation from email lists containing fire management types, a training record, and other preestablished lists and directories was used. Because the Predictive Services group wanted as many respondents as possible, we used all members in our list whose email addresses could be obtained. The sample was intended to be as comprehensive as possible given the lack of a census listing.

Beyond those included in the sample, an additional number of individuals responded as volunteers. Two circumstances prompted volunteering. The first occurred when initial contacts forwarded the survey link to others after completing it themselves. In some cases, initial contacts felt they were not the best person to complete the survey and forwarded it to another contact within their agency.

The respondents included 1,078 individuals (including 63 volunteers or 5.8 percent of the sample). The majority (69.1 percent) were male, employed with the U.S. Forest Service (53.3 percent), NOAA and the National Weather Service (14.3 percent), Bureau of Land Management (12.6 percent), the National Park Service (10.0 percent), U.S. Fish and Wildlife Service (4.7 percent), and Bureau of Indian Affairs (3.5 percent). The remainder were employed within a Federal interagency group (.6 percent) and various other Federal agencies (.9 percent). Respondents had typically been in their current job for 3 years (median response). We had a final response rate of 36.5 percent with less than 1 percent of the sample refusing to participate (12 individuals). A random sample of nonrespondents was contacted by telephone and asked to complete a brief phone survey covering reasons for nonresponse, use of various GACC websites, and familiarity with products and services. The main reasons for nonresponse were lack of familiarity with the program and lack of time during the study period. Nonrespondents were similar to respondents in geographic location and agency of employment.

A comparison of the sample respondents and volunteers revealed that the volunteers were twice as likely to be employed within the Bureau of Indian Affairs (7.9 versus 3.3 percent of each sample) and Bureau of Land Management (22.2 versus 12 percent), and were less likely to be from the U.S. Forest Service (31.7 versus 54.7 percent). The average length of employment was significantly different, with volunteers reporting fewer years (2.9 years for volunteers and 6.2 years for the original sample, t=3.326, p=.001). Gender distribution was similar for the two groups. The volunteers and original sample members are combined for the purposes of this paper because further analyses showed that there were few differences between these two groups.

The Survey

Topics addressed in the survey included sociodemographics (e.g., employing agency, years in current position, and gender), who the Predictive Services audience should be, preferred information formats, preferred products and services, acceptability of risk and tolerance for errors, implications of risk in making decisions, trust and confidence in the products, reliance on Predictive Services products, reliance on other information, and facilitators and barriers to using Predictive Services information. The survey included closed-ended (including semantic differentials, check lists, and other formats) and open-ended questions. Some survey items were modeled after recent studies conducted by another Federal agency to allow for comparison, whereas others were developed specifically for this study’s purposes. A draft instrument was submitted through peer review and review of the Predictive Services group commissioning the study. The instrument was pretested with a random sample of respondents, and adjustments were made to items that seemed unclear or were described as confusing by pretest participants. The survey was posted on the Web service Question Pro (www.questionpro.com). A Web-based survey was desirable because of the significant cost reduction achieved by eliminating printing and mailing costs, greater availability of email addresses than mailing addresses for the sample members, and increased familiarity of Web-based instruments among Federal personnel. Failed addresses were typically bounced back within minutes rather than days, allowing for attempted correction and remailing, or, when appropriate, elimination from the sample.

Procedure

Respondents were emailed an invitation and brief letter describing the study, along with a link to the survey site. Three reminders were sent over the course of the data collection period, with a total of 42 days allowed for response. The first reminder was sent 10 days after the initial mailing, the second was sent 14 days after the first reminder, and the last was sent 10 days after the second mailing, 1 week prior to the close of the survey site. Each of the reminders contained a brief message and the link to the survey site. Reminders were sent to sample members who had not completed the survey and those who had not been removed from the sample due to email failures.