Results

Identified Audiences

Respondents were asked whom Predictive Services should include as the primary audience for their products. The primary audiences selected by the majority were local and district fire managers (75.8 percent), regional and State fire managers (75.3 percent), and national fire managers (65.5 percent). Non-fire land managers were listed as a primary audience by about one-third (33.5 percent), and the public was listed by about one-fourth of the respondents (27 percent; note that percents do not sum to 100 because respondents could provide multiple answers on this item).

Information Used and Utility of Information

Preferred Formats

Respondents were asked to indicate how useful each of 11 styles and formats of presenting information was to them. The average ratings of all items except one fell above 3 (the neutral point on the scale). From greatest to least, the most useful formats were regional or national maps (χ=3.91, sd=1.03, n=879), satellite maps (χ=3.76, sd=1.15, n=870), brief executive summaries of data (χ=3.75, sd=1.08, n=858), brief annotations that accompany data (χ=3.56, sd=1.02, n=850), radar maps (χ=3.53, sd=1.19, n=857), data in table form (χ=3.53, sd=1.05, n=863), bar charts or figures that summarize data (χ=3.37, sd=1.09, n=856), data in text form (χ=3.33, sd=1.03, n=849), Web-based ArcIMS maps with user-defined layers and scales (χ=3.31, sd=1.23, n=832), and data in spreadsheet form (χ=3.21, sd=1.10, n=853). Least useful to respondents was non-Web-based Geo database files (χ=2.62, sd=1.12, n=793).

Preferred Products and Services

Thirty-eight products and services were listed in the survey. Some of these products are available elsewhere as well and are provided as a courtesy to the website users. For each item, respondents were to indicate if they had not used the product, and, if they had, to rate the usefulness of that product. Several of the products stood out because at least 70 percent of the respondents had used the products and rated them as useful or very useful. They included daily fire weather forecasts, red flag warnings (this term is used by fire weather forecasters to alert users to an ongoing or imminent critical fire weather pattern, www.nwcg.gov/pms/pubs/glossary/index.htm), Incident Management Situation reports, drought information, and Interagency Situation reports. A number of products were used by a majority (at least 50 percent), although ratings of usefulness varied. A few items offered through Predictive Services had been used by a minority of respondents and were not rated as very useful by those who had used them. Among the products in this category were regional monsoon updates, upper air soundings, Predictive Services forms, and state-of-the-fuels reports.

Suggestions for Improved Formats and Products

Respondents gave several suggestions for improvement in response to open-ended questions. In terms of format, there were several suggestions for improving website performance, including making sure that GACC and Predictive Services employees could direct people to the right location, streamlining information searches by allowing users to bookmark relevant information, having a professional Web designer improve the sites’ navigability, and removing information that is no longer accurate. In terms of expanding products and services, there is a desire for more location-specific products and more two-way conversations between Predictive Services and people who work on the local level. This communication would support local decisionmaking and possibly increase the relevance and quality of information provided by Predictive Services. People working in off-season or prescribed burning capacities, or both, suggested more year-round coverage. Additional topics for website content were offered including information on smoke management, fuel moisture, safety, real-time information, and current fire behavior.

Acceptability of Risk and Issues of Accuracy

Respondents were asked to choose the statement that best fit their preference regarding error in predicting risk. The majority (67.3 percent) chose “Statements of danger or risk be issued with a greater margin of error allowing for an early response, knowing that this may lead to unnecessary alarms and response (better safe than sorry)” over “Statements of danger or risk should only be given with certainty, knowing that this may allow a few dangerous events to emerge that were not anticipated (don’t cry wolf)” (chosen by 23.9 percent of respondents). In other words, the majority preferred erring on the side of caution when reporting on fire danger and high fire potential.

Open-ended responses pointed to concerns surrounding information accuracy. Among the topics of concern were the need for clear statements of the limitations of the data and known degrees of accuracy (for example, some would like to see confidence intervals reported along with data). There was also an interest in sources and assumptions used in creating the products offered.

Implications of Risk in Decisionmaking

The perceived impacts of inaccurate information were examined. To address this concept, two items were used. The first was “Inaccurate Predictive Services information would decrease my ability to predict fire behavior” rated on a scale from 1 to 5 (1=strongly disagree, 5=strongly agree). The majority leaned towards slight agreement with this statement (χ=3.36, sd=1.16, n=712) with 12 percent indicating strong agreement and another 20 percent indicating agreement. Another 18.2 percent neither agreed nor disagreed with this statement.

The second item was “Inaccurate Predictive Services information used in my decisionmaking may adversely impact firefighter or public safety” again rated on the 1 to 5 scale. The majority leaned towards slight agreement on this statement as well (χ=3.48, sd=1.18, n=744), with 20.3 percent indicating agreement and 16.0 percent indicating strong agreement; 17.8 percent neither agreed nor disagreed with this statement.

Written comments pointed to concerns surrounding accuracy in data gathered to make predictions and communication issues. These comments revealed a disconnection between Predictive Services and local field units. Comments indicated that Predictive Services might benefit from a better awareness of local weather and fire problems. Communication-related comments addressed concerns over the need for consistency in content, streamlining of information, and concentration on materials directly relevant to fire-use decisions.

Trust, Confidence, and Reliance

Trust and confidence in the information provided by Predictive Services were assessed in a general item “How much trust and confidence do you have in the information provided by Predictive Services?” rated on a scale from 1 to 5 (1=none at all, 5=a great deal). Very few respondents selected 1 (none at all, 8.8 percent) or 2 (5.3 percent) on this item. About one-fourth (25.7 percent) indicated some trust and confidence, whereas almost half selected either 4 (35.4 percent) or 5 (12.8 percent), indicating a majority of respondents had trust and confidence in the information provided.

In addition, when asked about three specific trust-related issues that might be barriers to using Predictive Services, very few indicated that trust was an issue. Only 3.5 percent indicated that they did not trust the products and services, 1.4 percent indicated a lack of trust in advice about using the products, and less than one percent indicated a lack of trust in information produced by multiple agencies. These specific items suggest that most had trust in the information provided.

Comments specific to trust and confidence included the desire among respondents to have a working relationship with the people who provide the information, as exemplified by this quote: “The local weather service offices continue to provide one-on-one support for weather products. The level of trust in a forecast product is directly related to the personal conversations I have had with the forecasters.”

In spite of some trust, to a great deal of trust expressed by the majority of respondents, the majority do not rely on Predictive Services in decisionmaking. About 10 percent (9.6) relied on Predictive Services a great deal (a rating of 5 on a 1 to 5 scale, 1=none at all, 5=a great deal). About one-fifth (21.2 percent) provided a rating of 4 and about one-third indicated little to no reliance on Predictive Services information (12.5 percent gave a rating of 2; 21.5 percent a rating of 1).

When asked how true the statement “I rely on other sources more heavily than the products and services provided by Predictive Services,” the majority indicated that this statement was somewhat to very true (51.1 percent), whereas 16.8 percent indicated the statement was not at all true. The likelihood of taking action based on Predictive Services information received or gathered from a website suggested respondents were somewhat likely to take action (χ=2.96, sd=1.23, n=979, on a 1 to 5 scale, 1=not at all likely, 5=very likely).

Facilitators and Barriers to Utilization

Two facilitators to utilization were queried based on accessibility and utility. The first of these was “I can access and apply Predictive Services information as part of my job duties.” Almost half (46.3 percent) agreed or strongly agreed with this statement. Approximately another fifth were neutral (18.9 percent), and almost one-third did not supply a response (27.7 percent). The second item related to utilization facilitators was “Predictive Services information helps me perform my job with greater precision,” with which 13.7 percent agreed, whereas almost one-third (31.4 percent) disagreed or strongly disagreed. About one-third (32.5 percent) did not respond to this item.

Barriers to utilization were explored through a general item related to uniqueness of the information “I think there is overlap in the type of information that I can obtain from Predictive Services and other sources.” More than half (56.5 percent) indicated that this statement was somewhat to very true. Respondents noted overlap between the products and services offered by Predictive Services and other sources, particularly the National Weather Service. Some suggested a closer coordination between the two providers in order to reduce or eliminate redundancies.

Specific barriers to utilization not related to trust of the information (already presented above) were examined. The most frequent barrier selected was “I never thought about it,” (indicated by 26.9 percent). Other barriers selected by at least one-tenth of the respondents included “My current management practices don’t require the types of information provided by Predictive Services,” (14.7 percent); “I don’t know how to use these products,” (14.1 percent); and “I need information that is site specific” (13.5 percent). Some respondents also mentioned a lack of resources as a barrier (lack of time mentioned by 9.3 percent; lack of technology by 4.0 percent; and lack of money at 1.4 percent).

Open-ended responses offer additional insights into barriers in using Predictive Services including levels of awareness and access. Some respondents were either unaware of the products and services or indicated a limited knowledge of the array of available information and its potential uses. Respondents made several suggestions that would address this situation, including advertising to targeted markets, annual notices of new and existing products and services, and developing Web-based orientation or training, or both, to familiarize potential users with the products. Respondents also suggested presenting information in lay terms, including a glossary of acronyms to further enhance understanding, and creating a Web feature that allows users to earmark their most relevant Web links. Respondents suggested that improved graphics might ease information utilization.

Predicting Reliance and Use of Predictive Services Information

The ability to predict reliance on Predictive Services information, and the likelihood of taking action were examined through simultaneous multiple regression. Approximately 50 percent of the variance (R^{2 adj.} =.50, F _{4, 927} =234.16, p<.001) in “How much do you rely on the information provided by Predictive Services to assist in decision-making” was predicted by trust and confidence in the information provided, gender, years in current position, and educational level (Table: Regression results for predicting reliance).

Male respondents were significantly more likely to rely on Predictive Services information (t=6.36, df=483.68, p<.001, males χ=3.00, females χ=2.42). Reliance had an inverse relationship with years in position in job (r=-.086, p=.006, n=1,003); Federal employees with longer tenure were less likely to rely on Predictive Services information. Those expressing greater trust and confidence in Predictive Services were far more likely to rely on the information in decisionmaking (r=.704, p<.001, n=944). There was not a significant linear relationship between reliance and education.

Approximately 48 percent of the variance (R^{2 adj.} = .481, F _{4, 922} =215.71, p<.001) in “How likely are you to take action based on Predictive Services information that you gather or receive from a website” was predicted by trust and confidence in the information provided, gender, years in current position, and educational level (Table: Regression results for taking action).

Male respondents were significantly more likely to take action based on Predictive Services information (t=4.57, df=429.25, p<.001, males χ=3.08, females χ=2.66). Taking action had an inverse relationship with years in position (r=-.120, p<.001, n=979). Those expressing greater trust and confidence in Predictive Services were far more likely to take action based on the information (r=.688, p<.001, n=939). There was not a significant linear relationship between reliance and education.