On-farm RD&E framework for M&E

Summary

This project was initiated to undertake a review of MLA LPI priority setting and evaluation processes.  LPI’s objective is that research resources be used efficiently from the viewpoint of industry and community stakeholders. The essential element of any impact assessment process, whether ex ante or ex post, is that it contains a plausible story about the pathway by which dollars invested in research translate into specific outcomes of value to the industry and the community. 
It is impossible to objectively measure the full impact of research activities and rank projects on the basis of a simple score or financial metric. Difficulties that preclude this include:

• Long lags between research activities and impact;

• Uncertainty about impacts on heterogeneous farms

• Uncertainty about the rate and extent of adoption;

• Uncertainty about environmental and social impacts and their value to society.

This is particularly true for project proposals (ex ante analyses) but also applies to ex post evaluations.  Research managers have to weigh these subjective and objective elements of investment proposals in developing their portfolios. It is important that they fully understand the elements of an impact assessment process and the key concepts and tools, often economic in nature, they will be employing in developing these narratives. Sophisticated modelling of some parts of this narrative is not a good substitute for a more intuitive understanding to the whole. Elements in the narrative include:

• A description of the problem and alignment with MLA priorities;

• An identification of the target population for the innovation;

• A description of the science underlying the research and its probability of success given the capacity of the research team;

• What resources will be used in research and extension processes;

• A description of the on-farm impact of the technology in terms of changes in farm practice and changes in unit costs or profit if possible;

• How will the technology be transferred to final users;

• What will be the rate and extent of adoption;

• What are likely environmental and social outcomes

• How will industry and community develop in the absence of this research.
​
This narrative has to be constrained by the pool of resources available for the proposed research activities. There has to be some indication of a likely ‘causal link’ between research activities and industry and community outcomes. 
​In reviewing evaluation processes in LPI the consultant found: 

• A strong recognition that research activities must lead to identifiable industry impacts whether they be economic, environmental or social, in addition to using and developing good science;

• An acceptance of the importance of, and good intuition about, economic concepts and tools used in assessing the impact of research activities  (even if familiarity sometimes remains a hurdle);

• A recognition of the importance of the LPI peer review process;

• Widespread acceptance of the R-M model to estimate economic impacts

However there seems to be room for improvement in a number of areas.

• The impact assessment process for proposals could be made much more transparent.

• Impact assessment processes are facilitated when there is a clear link between a set of discriminatory strategies whose economic and/or community significance is objectively documented and a set of research programs addressing these strategies. Whether this holds for LPI has not been addressed here. 

• Expanding the set of ‘tools’ to estimate the on-farm impacts of technology, e.g. the RM model is based on historical data and could be more forward looking particularly given the wealth of economic data within MLA. Moreover it is weak in capturing whole farm effects from technologies when there are interactions (jointness) between say livestock and cropping enterprises.