There are now quite a number of experimental designs and RCTs (randomized controlled trials) of market information systems measuring revenue gained by rural agriculturalists (farmers, fishermen) and traders. While some show improvements to the prices farmers get for their crops, others disappointingly show a lack of impact. There are also studies of agricultural extension services delivered via mobile phone which suggest that crop productivity is boosted by such services. How might these studies guide what actions are taken (i.e. to provide such services) in the real world?
What is ‘ecological validity?’ It is the relationship between controlled, or lab-based experiments and what unfolds in a non-lab, non-controlled environment (aka the ‘real world’). Out in the world multiple variables interact in different ways. So what can we take away as actionable from an RCT that shows a market information system has resulted in boosted revenues for farmers who used it? How do we reconcile the successes with similarly designed studies that showed a failure of impact?
Recent RCTs of MIS have been designed to provide the ideal circumstances for their success. They do so in the following ways:
1) providing the service for free [see Cole and Fernando 2014, Fafchamps and Minten 2012, Nakasone 2013, Hildebrandt et al. 2013]
2) automatically enrolling farmers and pre-customizing information delivery [Nakasone 2013]
3) providing extensive training on the use of the system to overcome literacy limitations [Hildebrandt 2011]
Why do they do this? Because these studies intend to show what impact information (in isolation from other issues) has on markets, farmer profits, and revenues. That does NOT quite make these studies true evaluations of a particular system in real-world use.
So what does this mean for the real-world use of MIS systems such as Esoko, Reuters Market Light, etc?
For item (1) it means that a successful business model will most likely require that it be provided for free, (at least in the short to medium term) if it is to yield the effects measured by recent studies.
For (2) and (3) it means that service adoption and literacy issues remain significant and substantial barriers. Artificially eliminating them for the purpose of an RCT is problematic because it obscures this fact. Fortunately, many programs in this space are finally becoming more attuned to this reality and offering such services through more user-friendly and accessible IVR or call center models (as Esoko now does). For more literacy-demanding services, training (or infomediaries) will be needed. Where training is needed, the scalability of such services is an issue.
Susan Wyche and Chip Steinfield have a paper that’s just come out evaluating the user interface issues around MIS. They look at the affordances of such services and the challenges they pose for farmers. By contrast, RCTs championed by economists fail to address usability and literacy issues when they treat ‘information’ as the only important variable to isolate. It will likely require more interest and support for researchers with different kinds of training (HCI, social science) to make sure this important issue is not trivialized.
Bibliography:
Cole and Fernando (2014) ‘The Value of Advice: Evidence from the Adoption of Agricultural Practices‘ Harvard Business School Working Paper.
Fafchamps and Minten (2012) Impact of SMS-Based Agricultural Information on Indian Farmers. The World Bank Economic Review.
Nakasone, Eduardo (2013) The Role of Price Information in Agricultural Markets: Experimental Evidence from Rural Peru. Selected Paper prepared for presentation at the Agricultural & Applied Economics Association’s 2013 AAEA & CAES Joint Annual Meeting, Washington, DC, August 4-6, 2013.
Hildebrandt, N. (2011). Teach a Man to Text. Esoko, Insights from the Field.
Hildebrandt, N., Y. Nyarko, Romagnoli, and Soldani (2013). Market Information Systems for Rural Farmers: Evaluation of ESOKO MIS – Year 1 Results. New York University: Africa House Blog.
Wyche, S. and C. Steinfield (2015). Why Don’t Farmers Use Cell Phones to Access Market Prices? Technology Affordances and Barriers to Market Information Services Adoption in Rural Kenya. Information Technology for Development
