IPM Planning and Evaluation

DOCUMENTING IMPACTS

It is possible, and frequently advantageous, for IPM specialists to assess the impact of Extension IPM programs. Many specialists may already do so as a routine component of their ongoing educational and training efforts. According to logical model conventions and the guidance currently provided by federal agencies that support IPM programming, impacts are changes in the short-term knowledge and intermediate-term behaviors and practices of target audience, as well as long-term changes in economic, environmental, or human health conditions.

Assessing changes in short-term knowledge may require that baseline data exist, or be collected, on the client's pre-program knowledge of IPM principles and practices. Because of their years of experience dealing directly with their clientele, county- or state-based IPM experts are often a good source for anecdotal data on target audience baseline IPM knowledge. In addition, specific tactics such as pre- and post-training evaluations using written program evaluations, "Turning Point" or other polling software techniques, can result in quality data on baseline knowledge. More formal methods, such as mailed, telephone, or face-to-face surveys are also useful tools to assess changes in clientele knowledge, skills, and attitudes regarding specific IPM practices.

Assessing intermediate-term behavioral changes also requires baseline data on the extent of client's pre-program adoption of IPM. Once again, county- or state-based IPM experts are often a good source for anecdotal data about client behavior and the extent to which they have adopted IPM practices. State, county, or other records (e.g., NASS Surveys or NRCS data) may also provide reliable information on the adoption of specific practices. As with short-term knowledge changes, quality baseline data for assessing behavioral changes can be obtained using pre- and post-training evaluations, written program evaluations, or other polling techniques. And formal surveys (mailed, telephone, or face-to-face) can be a useful tools to assess the extent of adoption of specific IPM practices. Adoption surveys that obtain information from large, statistically-valid samples and that are based on well-developed descriptions of an IPM system (e.g. IPM Guidelines, IPM Elements, IPM Protocols, IPM Standards, etc.), are most useful in this regard. See the section on "Other Resources" for some examples of previously developed surveys, guidelines and other useful information.

It can be more challenging to assess long-term changes in economic, environmental, or human health conditions that result from IPM educational programs. University-based scientists may have little experience with social science techniques, or they may have limited access to colleagues with such expertise. Further, a typical grant-funded Extension or research project often has a limited term. Budgets are most often focused on addressing specific hypotheses rather than on documenting whether potential end-users actually adopted techniques that were investigated or whether those techniques eventually had long-term impacts.

Although no two experimental sites, growing seasons, schools, playing fields, etc. are identical, it is possible to use existing research to support claims of long-term impact. Assume, for example, that a robust scientific literature shows a particular insecticide has a negative impact on human health, non-target organisms or the environment. If a research or demonstration project shows that implementing specific IPM techniques eliminates the use of that insecticide in an important agricultural crop and can also document that these techniques have been adopted on a large scale, this may be sufficient evidence to make a reasonable claim of a long-term impact.

Sample Measures for Short-, Intermediate-, and Long-Term Impacts

Example Short-Term Impacts and Possible Measures

Note that for each short-term impact, the measures are the same—you always need baseline data so that you can measure intermediate- and long-term impacts.

Short-Term Impacts Possible Measures

Target audiences increase awareness of pesticide and nutrient impacts on habitats

Collect baseline data to help measure intermediate- and long-term impacts

Use self assessments, pre-test/post-test and follow-up measurement tools to assess changes in knowledge, attitudes, satisfaction, aspirations

Target audiences increase knowledge of lower-risk materials

Target audiences increase knowledge of IPM as a means to prevent off-site movement

Target audiences increase knowledge of environmental benefits of using precision application equipment and technology

Target audiences increase knowledge of conservation programs including trade-offs (negative and positive) associated with pesticide use in managing invasive species

Target audiences increase awareness of positive IPM impacts

[Back To Top]


Example Intermediate-Term Impacts and Possible Measures

Note that some measures will give information about more than one intermediate-term impact.

Intermediate-Term Impacts Possible Measures

Reduce use of higher-risk pesticides or increase use of lower-risk materials

  • Measure reduced use of higher-risk pesticides or increased use of lower-risk materials
  • Document adoption of lower-risk timing of pest management activities
  • Measure number of TMDLs implemented to address impairments
  • Measure changes in residue detections and amounts of pesticides in water resources

Increase use of IPM tactics to protect public lands

  • Measure increased adoption and implementation of IPM
  • Document adoption of lower risk timing of pest management activities

Ensure dispersal equipment is properly calibrated and all drift minimization practices are used for all applications

  • Document calibration and drift minimization measures

Increase adoption of lower risk timing of pest management related activities

  • Measure increased adoption and implementation of IPM
  • Document adoption of lower risk timing of pest management activities

Increase adoption of precision application equipment and technology to minimize off-site impacts

  • Measure increased purchasing/use of precision application technology and equipment
  • Measure changes in residue detections and amounts of pesticides in water resources

Increase adoption of lower risk application technology

  • Document adoption of lower-risk application technology

Increase adoption of conservation measures to protect natural resources from pesticide and nutrient runoff

  • Measure number of TMDLs implemented to address impairments
  • Measure changes in residue detections and amounts of pesticides in water resources

[Back To Top]


Example Long-Term Impacts and Possible Measures

This table contains all of the Long-Term Impacts and Possible Measures currently on this site.

Long-Term Impacts Possible Measures

Reduce impacts to non-target terrestrial and aquatic organisms caused by pesticides

  • Measure reduced incidence of fish and bird kills due to pesticides
  • Measure reduced toxicity profile for avian species
  • Measure reduced toxicity profile for aquatic species
  • Measure increased diversity and abundance of beneficial species
  • Measure increased numbers of pollinators
  • Measure changed detection of residues in natural environments
  • Assess changes in endangered species listing

Reduce impacts of off-site movement of pesticides, sediments and nutrients on soil, air, and water quality

  • Measure decreased contaminants in water bodies
  • Measure delisting of water bodies with pesticide impairments
  • Measure increased population of native fish, aquatic insects, invertebrates, and/or freshwater mollusks
  • Measure improvements in soil quality, air quality, and/or water quality as a result of adoption of conservation measures
  • Measure increased natural fish habitat
  • Measure increased diversity and abundance of aquatic species

Reduce farmer, farm worker and farm family exposure to and risk from pesticides

  • Measure change in number of cases and severity of worker exposure incidents
  • Measure change in number of pesticide poisonings
  • Measure and maintain baseline cholinesterase levels in workers
  • Measure change in pesticide use from precision ag equipment

Improved profitability of production agriculture through adoption of IPM

  • Document economic impact of farmers/ranchers in certification programs
  • Measure changes in cost-of using IPM tactics (thresholds, modeling, etc.)
  • Measure number of food processors and distributors demanding IPM programs and providing incentives to farmers (e.g., new market access)
  • Measure increased number of Target audiences in certification programs.
  • Measure effect of externalities in cost-benefit analyses
  • Measure number of retail markets that provide incentives to farmers using externalities (e.g., new market access)
  • Measure reduced amounts of inputs (e.g., pesticides, fertilizers and water)
  • Measure change in social acceptance of less than perfect produce

Off-site movement of pesticides, non-target impacts, and other environmental impacts are reduced

  • Measure change of pesticide detections in nearby water bodies or sources (e.g., streams, public water, private wells, etc.)
  • Measure changes in nutrient loading in water bodies
  • Measure reduced number of residential drift complaints
  • Measure non-target impacts of pesticide use
  • Document change in pesticide use in the sample landscapes.
  • Document changes in off-site movement of pesticides, non-target impacts, and other environmental impacts.
  • Assess value of ecosystem services

Reduced exposure to pests and pesticides in federal buildings, public housing will lead to improved health of those living and working in the indoor environment

  • Measure decreased number of cases and severity of pesticide exposure incidents
  • Measure pest complaints, sightings, damage
  • Measure changes in incidence of asthma or other health problems associated with pests or pesticides
  • Measure changes in indoor air quality
  • Measure decreased number of pesticide poisonings
  • Measure reduction of household pest allergens and pesticide residues

Schools /childcare facilities / Public Housing Authorities save money by using cost-effective IPM practices

  • Measure number of schools, districts and/or childcare facilities that save on cost of pest management after training staff in using IPM
  • Measure number of schools, districts and/or childcare facilities that save on cost of pest management after developing or implementing IPM plans
  • Measure number of schools, districts and/or childcare facilities that save on cost of pest management after developing or implementing IPM contracts/bid specifications
  • Measure number of schools, districts and/or childcare facilities that save on cost of pest management after developing or implementing IPM policies/laws or becoming IPM Certified
  • Use surveys, face-to-face interviews, or other means to measure number of schools, districts and/or childcare facilities that save on cost of pest management after adopting IPM policies/laws or becoming IPM Certified

Reduced exposure to pesticides used outdoors in residential and public areas

  • Document complaints about pesticide use
  • Measure reduction of pesticide detections in nearby water bodies or sources (e.g., streams, public water, private wells, etc.)
  • Measure reductions in nutrient loading in water bodies
  • Measure reduced number of residential drift complaints
  • Document reduction in pesticide use in sample landscapes.
  • Measure decreased number of cases and severity of exposure incidents
  • Measure decreased number of pesticide poisonings
  • Measure and maintain baseline cholinesterase levels in workers

Target audiences save money compared to previous approaches by implementing cost effective IPM practices

  • Conduct cost-benefit analyses that include externalities
  • Document cost effectiveness of lower risk pesticides and tactics
  • Document cost effectiveness of precision application technology and equipment
  • Document cost effectiveness of IPM adoption compared to previous approaches
  • Document cost effectiveness of changed aesthetic standards
  • Measure social acceptance of less than perfect landscapes

Improve the health of children/staff and the learning environment in schools and childcare facilities by reducing exposure to pests and pesticides

  • Measure reduced number of cases or severity of pesticide exposure incident reporting
  • Measure reduced incidence of asthma or other health problems associated with pests or pesticides
  • Measure improvement in indoor air quality
  • Measure reduction in pest-related complaints
  • Measure reduced school and childcare facility absences

Reduce off-site movement of pesticides and nutrients into water and improve water quality conditions

  • Measure efficacy of lower risk products in promoting biodiversity
  • Measure changes in levels of contaminants in water bodies
  • Measure delisting of water bodies with pesticide impairments
  • Measure changes in population of native fish, aquatic insects, invertebrates, and/or freshwater mollusks
  • Measure changes in water quality as a result of adoption of conservation measures
  • Measure changes in natural salmon habitat
  • Measure changes in diversity and abundance of aquatic species
  • Document value of ecosystem services

Reduce impacts to non-target terrestrial and aquatic wildlife caused by pesticides and nutrients

  • Measure changes in non-target populations of interest
  • Measure contribution of IPM implementation to delisting of endangered species
  • Measure reduced toxicity profile for avian species
  • Measure reduced toxicity profile for aquatic species
  • Measure increased diversity and abundance of beneficial species
  • Measure increased number of pollinators
  • Measure reduced detection of residues in natural environments
  • Document value of ecosystem services

Reduce pesticide exposure of site staff and land users/resource users

  • Measure changes in number of cases and severity of worker exposure incidents
  • Measure changes in number of pesticide poisonings
  • Measure and maintain baseline cholinesterase levels in workers
  • Measure changes in Dislodgeable Foliar Residues
  • Measure use of personal protective equipment as required by the pesticide label and MSDS

Target audiences save money by implementing cost effective IPM practices on natural resource and recreational lands

  • Document agency savings from use of precision application equipment and technologies
  • Measure effect of including externalities in cost-benefit analyses
  • Document value of ecosystem services
  • Document savings from IPM adoption

[Back To Top]

About Us | Privacy Policy | Contact Us

This website is maintained by the Southern IPM Center, supported by the National Institute of Food and Agriculture, U.S. Department of Agriculture.