DAV00140 - Evaluating the impact of the Soil Biology Initiative-II (2009-2014)

Final Report

Project Start Date
26 May 2014
Project End Date
26 October 2014
Supervisor Name
Pauline Mele
Organisation
Department of Environment and Primary Industries
Contact name
Pauline Mele
Contact phone
03 90327083
Region
South
Summary

The GRDC Soil Biology Initiative-II (SBI-II) comprises 18 projects in three themed areas (soil quality monitoring, nutrient management and disease suppressive soils). Evidence generated from SBI-II projects against agreed outcomes were assessed using the monitoring, evaluation, reporting and improvement (MERI) framework (SBI-II 2009-2014 'Harnessing the Biological Potential of Our Soils' Alice Roughley). An ex-post benefit cost analysis (BCA) (Peter Chudleigh, Agtrans) assessed impacts based on assumptions (Impact Assessment: Analysis of GRDC Investment in SBI-II, Final Report Chudleigh & Bath, Agtrans). Both assessments must be considered as each is highly complementary.

Conclusions

SBI-II was designed as a strategic investment based on the 18 contract specifications. This was strongly influenced by the outcomes of an evaluation of Land and Water Australia (LWA)'s soil health extension program 'Healthy Soils for Sustainable Farms' that identified the need for vastly improved technologies to assess soil biology in a crop management context. The evaluation highlights the strong evidence of technical progress and significant growth in research capacity and infrastructure related to soil biology research and development (R&D) as a result of this investment. The evaluation also highlights the innovative and global relevance of SBI-II through evidence of significant attention from the national media (e.g. ABC Landline program) and from the global agricultural and scientific communities. It follows then that the knowledge produced by SBI-II has placed GRDC and the industry in a considerably improved knowledge position compared with that before SBI-II and therefore presents a case that future investment would be profitable.

MERI also highlighted some issues about the effectiveness of SBI-II. In summary, it identified that:
1. The gap in the SBI-II program was the lack of investment in communication and adoption (based on most five year outcomes not achieved); less than 1% of funding went into this area.
2. None of the managers and coordinators was tasked with extension communication.
3. There was an expectation that information would feed into existing networks that in most cases were only becoming established (e.g. regional cropping solutions networks in the west and south).
4. There is a difference between communication and adoption and this difference is not well understood (as evidenced in SBI-II). Adoption refers to end-users applying the knowledge or recommended practice. Active, genuine end-user engagement is implied in adoption. Communications are often in the form of one-way information to passive end-users. Adoption can often be understood as researchers making end-users manage in a new or different way. It must be understood that researchers produce knowledge and/or recommendations to influence practices. They are not usually active change agents. Researchers can provide information in innovative and accessible ways, but advisers, local knowledge and family members have more influence on growers' decisions than researchers (see research study by Greiner).
5. Adoption implies something that people can change. SBI-II is largely a strategic investment which may lead to a mind-set change and eventually practice change.

In terms of a BCA, the SBI-II delivered a benefit cost ratio (BCR) of 31:1. Several projects in SBI-II (themes 2 and 3) have suggested that the farming systems currently used, particularly the use of synthetic fertilisers and chemicals, may inhibit the effective exploitation of soil biology mechanisms. This suggests the potential for more efficacious use of costly inputs. Also, further investment in SBI-II will lead to greater profitability and that reasonable and realistic assumptions can be developed about expected impacts (e.g. management practice changes) from future investment.

Recommendations

The evaluation of SBI-II highlights that the knowledge produced by SBI-II has placed GRDC and the industry in a considerably improved knowledge position compared with that before SBI-II and therefore, together with the findings from the CBA, supports the case that future investment would be profitable. Strong evidence of technical progress, significant growth in research capacity and infrastructure, and of the innovative nature and global relevance of the science undertaken support this.

Both evaluation approaches recommended program design improvements. Though SBI-II was a 'trailblazer' in implementing the MERI framework, the completion of issue prioritisation and program logic with targeted end-users before finalising the business plan, management structure, resource allocation and commissioning of projects would have enabled better alignment of the outcomes in research contracts and the MERI plan. Specific approaches have been suggested such as Decision Programming Language (DPL) to assist future economic evaluation and resource allocation. This would necessitate identifying the various pathways to impact for different investments (timelines, research and commercialisation costs, common intermediate outcomes leading to different pathways, and levels of uncertainties faced). Such an approach would not only improve evaluation of future expected benefits at any point in time, but would also be helpful in analysing risks and in assisting with multiple and sequential investment decisions and priorities where there are multiple pathways to the desired impacts.

SBI-II was designed largely as a strategic investment based on contract specifications. This was strongly influenced by an evaluation of LWA's soil health extension program 'Healthy Soils for Sustainable Farms' that identified the need for vastly improved technologies to assess soil biology in a regional crop management context. A better balance of applied and strategic investment is, however, recommended in order to achieve some short term outcomes and benefits, as well as longer term and potentially larger impacts. Further, the MERI evaluation concluded that investment in science for applied outcomes rarely impacts on grower practices unless there is a more serious investment in end-user engagement. Early and ongoing engagement with end-users through processes such as Participatory Action Research (PAR), as highlighted by the Western Australian (WA) field tour (February 2014), requires investment.

It is imperative that the recommendations and learnings from the SBI-II program are integrated into future investment decisions. Please refer to attached MERI final report (Attachment 2, Alice Roughley, November 2014) and the ex-post Benefit Cost Analysis final report (Attachment 1, Agtrans, Peter Chudleigh, November 2014) for further detail.

Outcomes

The following evidence based benefits were generated and are listed below:
1. The science is progressing strongly and has been found to be innovative, sound and of a global standard.
2. The program has built significant soil biology science capacity, collaborations, networks and research infrastructure.
3. Most contracted science outcomes were achieved (based on two final reports and four projects, including the flagship theme 1 project, due for completion in 2015).
4. Progress is being made across most areas of investment, but there is a low level of achievement for five year outcomes related to adoption. This has been attributed to timing of the evaluation as per point 3 above, the mismatch between contracted project outputs and MERI outcomes set by project leaders and lack of comprehensive end-user engagement and extension.
5. SBI-II could still potentially achieve the MERI 10 year aspirational outcome of 'Increased profitability and sustainability of grain cropping as a result of harnessing the biological potential of soil' with significant investment in further research and adoption activities.
6. If only the benefits delivered directly from SBI-II are considered, the net present value (NPV) of the investment is negative and the BCR for the investment is 0.31 to 1. However, if further investment to build on the new information produced by SBI-II is included, the NPV becomes positive at $6.8 million and the BCR increases to 1.33 to 1 for both the SBI-II and further investment combined.
7. Based on the set of assumptions made, the BCR of this further investment alone was estimated at 2.5 to 1. This would apply if the SBI-II investment is considered as a sunk cost.

Achievements/Benefits

This four month project has delivered a two-pronged approach to evaluate the SBI-II 'Harnessing the Biological Potential of Cropping Soils'. It is comprised of a summary document describing the rationale for selection of these two contrasting approaches and a MERI report and an ex-post BCA.

The SBI-II evaluation process has been rigorous and extensive and has been implemented throughout the course of the program. As such, it has highlighted some strengths and weaknesses in the approach and provides helpful recommendations for future implementation.

Specifically the MERI report highlighted that:
1. Progress is being made across most areas of investment but there is a low level of achievement for applied five year outcomes (these may have been unrealistic due to the largely strategic nature of most project contracts).
2. Most contracted science outcomes were achieved.
3. The science has been found to be innovative, sound and of a global standard.
4. The program has built significant soil biology science capacity, collaborations, networks and research infrastructure.
5. The program could still potentially achieve the MERI 10 year aspirational outcomes with significant investment in further research incorporating a comprehensive end-user engagement program communication.

The ex-post BCA analysis highlighted:
6. If only the benefits delivered directly from SBI-II are considered, the NPV of the investment is negative and the BCR for the investment is 0.31 to 1. However, if further investment to build on the new information produced by SBI-II is included, the NPV becomes positive at $6.8 million and the BCR increases to 1.33 to 1 for both the SBI-II and further investment combined.
7. Based on the set of assumptions made, the BCR of this further investment alone was estimated at 2.5 to 1. This would apply if the SBI-II investment is considered as a sunk cost.

Other research

It is strongly recommended that the three key documents that represent this project be used as reference material for future investments involving a focus on soil biology (refer to attachments).

Intellectual property summary
No commercial information.
Additional information

Attachments

1. Agtrans Analysis on GRDC's investment in Soil Biology Initiative 11.

2. MERI Evaluation report - Harnessing the Biological Potential of Our Soils.

3. MERI - Part 2 report.

4. Brief evaluation of SBI-11.

Published Date
16 January 2018
Disclaimer

← Return to the Final Reports list