Minimising observer variation
As the monitoring result will have a direct influence on the way that a site is managed, as well as resource implications for the land manager, a monitoring project must provide the same result regardless of who does the monitoring, and that result must be the right result.
The main source of error in management monitoring projects is observer variation, i.e. the variation in the monitoring results produced by different observers. This source of error is often ignored in scientific research projects, but can be greater than the error associated with sample size, i.e. sampling error.
Perhaps unsurprisingly, the highest levels of sampling error are associated with recording using ‘subjective measures’, such as recording vegetation cover estimates, in all their different forms. Sampling trials have revealed that monitoring results informed by indirect measures are far more likely to reflect the biases of the individual recorders than the condition of the habitat being assessed.
The most reliable and consistent monitoring results are generated through the use of objective (or direct) measures, such as presence or inferred absence. When applied with species (or assemblages of species) with high detection rates, we can observer variation can be reduced to less than 5%, compared to 35% observer variation when using subjective measures. A document that looks at these issues in detail can be found here
Links to examples of good practice
- Hurford, C. (2006). Minimising observer error: increasing the reliability of a monitoring project. In: Hurford, C., Schneider, M. (eds) (2006) Monitoring nature conservation in cultural habitats: A practical guide and case studies. Springer, Dordrecht, The Netherlands