Monitoring and evaluating policy effectiveness

Consistent and Dependable Monitoring

New Zealand is currently unable to paint a national picture of the state of the country’s waters, largely due to inconsistency in monitoring data between regions. We made a major contribution to the NEMaR (National Environmental Monitoring and Assessment) project that was developed to address this need. A series of reports arising from NEMaR (available on the Ministry for Environment’s website) recommend:

• National reporting indicators for water (we expect these will also be used in regional SoE reporting).
• Variables (attributes of water) to be measured together with measurement protocols and quality assurance (QA).
• Networks of monitoring sites.

For further information, our report (Davies-Colley et al., 2012a), identifies and recommends national freshwater monitoring protocols for core river water quality, river bio-monitoring, and lake monitoring variables.

Recommended regional council protocols and quality assurance methods are described in a second report. A third report (McBride et al., 2013) consolidates the recommendations and costs of making the transition to consistent regional monitoring across the country.

The VMO programme has added value to NEMaR by providing on-going advice to regional council SoE monitoring staff, and presenting overviews at national conferences and hui.

Freshwater Conditions and Trends

Our review (Davies-Colley et al., 2011) of New Zealand’s long-term river sites highlights the stability of the National Rivers Water Quality Network (NRWQN) operations and the value of that stability for tracking condition over time (trends) in the nation’s waters. It also highlights the value of constantly striving to improve data quality and of ‘integrating’ different attributes of water monitoring (notably benthic bio-monitoring integrated with water quality, sediment and flow data). The NRWQN provides a ‘model’ for regional council water monitoring for SoE reporting, and has been used to underpin recommendations in the associated NEMaR project.

Our recent assessments of water quality in New Zealand have indicated degraded water quality in the 40 % of the country’s area under pasture.

Trend analysis (Ballantine & Davies-Colley, 2013) of the NRWQN data (and regional council data) also shows increasing nutrient concentrations, mainly from pastoral intensification.

Māori Cultural Monitoring

Māori cultural monitoring is needed to track the effectiveness of incorporating Māori values, tikanga, and mātauranga Māori of different iwi/hapū into environmental plans and decisions. So we have focused on reviewing, evaluating, testing, and applying cultural monitoring approaches, methods, and indicators with a large number of iwi/hapū groups:

• Te Hā o Te Wai Māreparepa (Mauri Monitoring Framework. Pilot Study on the Papanui Stream)
• Te Uri o Hau (Assessing the Mauri of the Kaipara)
• Rangitāne o Manawatū mandated iwi authority and Tanenuiarangi Manawatū Inc. (Manawatū River)
• Waikato-Tainui and Waikato River Authority
• Ngāti Kahungungu (Te Taiwhenua o Tamatea & Te Taiwhenua o Heretaunga, Hawke’s Bay – Waipukurau–Tukituki–Napier)
• Māori strategic and relations unit Auckland Council
• Tiakina Te Taiao, Nelson–Motueka.

Our fact sheet reviews current tools and methods for cultural monitoring.

Improved Statistical Methods for Reporting

A more appropriate statistical analytical metric (not using "p values" alone) to quantify the "strength-of-evidence" in data has been identified. This has been incorporated into a refined hypothesis-testing/confidence-interval framework (McBride et al., 2013) and the Time Trends and Equivalence freeware, which is widely used by regional councils.

For the next phase of the research we are developing new statistical approaches to make use of ‘null values’ (or values below the detectable level) in monitoring data.

Performance Reporting

The Fresh Water Reform 2013 (Ministry for the Environment 2013) highlighted the need for improved monitoring to enable adaptive management approaches. A Performance Reporting Framework has been developed to help councils improve frameworks and processes to evaluate the effectiveness of fresh water policy. The guide provides councils with a framework and method for undertaking performance monitoring and reporting of freshwater policies. Performance reporting is the mechanism by which decision impacts can be evaluated and reflected on, outcomes reported, and information communicated. It is also a means of facilitating adaptive management. As the approach and principles are generic, the guide can also be used to help evaluate policy effectiveness within other policy issue domains.

Community Monitoring to Support Community Decisions

We are identifying how community monitoring can support collaborative decision-making, and are also investigating the complementary relationships between community, cultural, and scientific monitoring of local waterways.

This is a major area of interest across New Zealand, and seems all the more important because community monitoring seems a logical extension of community collaboration in water planning. Community groups, including iwi, are likely to want to be involved in the whole policy cycle so that they know and can own the fate of ‘their’ water. We see community monitoring as a win-win for regional councils and community groups – the council providing encouragement and technical support to the community group and the community group acting as eyes in the field to extend the council’s monitoring coverage.

The main research focus is to compare the monitoring results of properly resourced community groups with the professional monitoring undertaken by regional councils. Nine streams from Auckland to Nelson are being monitoring simultaneously (same site, date, and time) by community groups and regional councils over an 18 month period. Community groups are using an updated version of NIWA’s SHMAK (Stream Health Monitoring and Assessment Kit) equipment for their monitoring. They measure water quality and algal growth monthly, and stream invertebrates and habitat every six months. Results at the halfway point have been collated and are available. We are working continually with the community volunteers to resolve issues that have reduced accuracy and reliability.

Selected useful findings

Engaging communities in freshwater monitoring

Community monitoring can increase the amount and scope of data on the state and condition of freshwater as well as provide many social benefits and raise public awareness. The aim of this study was to discover the motivation for members of community groups to take part in the monitoring, the benefits they and their communities had gained, whether monitoring would encourage them to engage in council-led freshwater planning processes, and what support would enable and encourage them to continue monitoring long-term. There is also an Ecological Economics journal article outlining the findings.