Evaluation Process for MLA On-Farm Research and Development

Summary

Meat and Livestock Australia (MLA) engaged Hassall & Associates to:
1. Conduct Triple Bottom Line (TBL) evaluations of two programs within the on-farm research and development (R&D) portfolio at MLA: the Southern Beef and Lamb and Sheepmeat programs. The review focuses on R&D projects within these programs that have been completed or have had substantial outputs by June 2003. The results of these evaluations are contained in Volume 1 of this report.
2. Develop standardised approaches for conducting future TBL evaluations. This will guide how MLA can incorporate TBL thinking and measures into program and project development, evaluation and management. The standardised approaches are detailed in Volume 2 of this report.
TBL evaluations for R&D programs address the financial, social and environmental outcomes achieved through the investment by industry and the Australian Government.  The study was predominately a desk-top exercise. The main finding from the examination of the previous reviews is that the approaches to evaluation are inconsistent. The main sources of inconsistency are whether the benefits:
• Are clearly described;
• Are estimated and/or quantified;
• Consider the baseline (e.g. what would have happened without the research);
• Consider how many people would use the R&Dand
• Consider causality and attribution (e.g. what the R&D actually achieved as compared to other factors such as seasons and higher prices).
For the financial analysis, the net benefit of both programs to industry participants is estimated to be $26 million (over 10 years with a 7% discount rate) with a benefit cost ratio of 1.29. The majority of the benefits are derived from the Lamb and Sheepmeat program ($22.7 million), however, it is important to note that its expenditure is approximately double that of Southern Beef. The majority of the benefits arise from three projects: Lambplan, Sustainable Grazing Systems and Breedplan.  The results are particularly sensitive to assumptions regarding increases to farm cash income, and somewhat sensitive to assumptions regarding the adoption rates. The data available is variable in quality and a significant number of assumptions had to be made regarding potential impact. This results in a low confidence regarding the absolute value of the results.
For the social analysis, the two major findings are the:
- Increased capacity of participants – higher skills, confidence and decision making abilities. Over the period, there has been a total of approximately 20,000 participants in the 13 projects considered. It is likely that there is double up and a number of producers who have participated in more than one MLA project. The extent to which these participants have higher skills cannot be ascertained from the data availableand
- Formation of groups – leading to improved networks and information flow. There have been approximately 250 marketing, supply chain, production, and business groups formed. Again, there are likely to be some producers that have participated in more than one group.
For the environmental analysis, the benefits are mainly derived from the Sustainable Grazing Systems project. One half of the respondents to the ABARE survey reported improved perennial pastures, water usage and a lower weed incidence. Overall, there was an increased recognition of two environmental issues – soil acidity and water quality issues. Whether this is a benefit depends on the distribution of the particular issues. There is no data available about the magnitude of the benefits reported. There is also an unclear baseline – where did people start from and what would have happened without the project/program.