Only 1 in 5 evaluations of humanitarian aid were found to have acceptable sampling validity and reliability (most used convenience sampling).1 The Center for Global Development recommended that - to reduce the risk of bias - random sampling is used, accompanied by a clear statement of methods. This is regardless of whether qualitative or quantitative methods were used.
We offer:
- A learning-by-doing approach
- You retain data in a universal format (we keep a back-up copy if you like).
- No expensive software solutions – using freeware whenever possible, and the Microsoft Office suite. We have experience of data collection mobile software Magpi, mWater, Kobo Toolbox / Open Data Kit (ODK).
- We dispense with popular terms which have crept from social sciences and philosophy (mid-term reviews, impact evaluations, counterfactual) and simply apply the methods most likely to give answers to the questions you have with the greatest likelihood of ascertaining attribution (to what extent we can be confident the intervention itself contributed to the outcomes or changes). We would never say “The mid-term review utilizes an informatics tool to analyse a robust literature search service preceding a field-based study incorporating key informant interviews and focal groups.” There are so many things wrong with that statement, the most important being that it does not explain what was done and with whom.
- Survey design and sampling to suit the questions being asked, the time and money available. Analysis is matched to the survey design.
_______________________
1 Goldberg Raifman J, Lam F, Madan Keller J, Radunsky A, and Savedoff W. Evaluating Evaluations: Assessing the Quality of Aid Agency Evaluations in Global Health. Center for Global Development. Working Paper 461, Aug 2017. There were only 37 studies but they were randomly selected and systematically reviewed, so there are likely to be between 10% and 38% of evaluations with acceptable sampling validity and reliability. Possibly non-health programmes would be lower, as health tends to attract public health practitioners with a knowledge of sampling.
