There’s a gathering trend among organizations working for peace (and their donors) to attempt to measure the long term impact (not just the immediate outcome) of initiatives. This desire to understand how an intervention is effecting lasting change on the development or recovery of a community is not exclusive to peacebuilding work. The basic problems with answering questions of impact are also common across the non-profit sector. Most organizations are not set up to gather data beyond the outputs of their own projects, nor do they have the in-house skills (or time allocated) to answer more complex data questions.
That’s where DataKind comes in. In their own words, DataKind “brings together leading data scientists with high impact social organizations through a comprehensive, collaborative approach that leads to shared insights, greater understanding, and positive action through data in the service of humanity.” If that sounds a little vague, take a look at a recent project they undertook for MIX. MIX works to strengthen the microfinance sector by providing information that promotes transparency. In different ways, MIX looks to assess the impact of microfinance on the world’s poor. DataKind collaborated with MIX to analyse data on access to financial services by the poor, with a view to understanding the impact of non-traditional financial service providers.
The collaboration resulted in some neat spatial analysis and interesting conclusions on the ‘invisible market’ of non-traditional lenders. Equally interesting is how DataKind helped MIX to find the data needed for this study. As they explain, typical methods to collect data for this type of analysis can be both time consuming an expensive. Instead, DataKind’s volunteers ran several programs to scrape the internet for publicly available, unstructured data that could feed the study. The automatic searching and processing of this data into a standard format (fusion tables or csv) saved MIX from a time consuming process they likely would not have had the resources to undertake.
Of course, questions on access to financial services are relatively easy to answer. Although we are looking for an answer on impact (are poor people more financially empowered), we are essentially asking an outcome question (are poor people able to access financial services). This begs the question of how we might answer the impact question more directly, not by way of a variable that (presumably) proxies for impact.
It’s this search to answer direct impact questions that makes DataKind’s work with UN Global Pulse particularly interesting. Global Pulse provided DataKind volunteers with the answers they received to a series of questions about happiness asked via SMS to people around the world (the Global Well-being Snapshot Mobile Survey). The survey asks the following questions about well-being:
- How did you feel over the past 7 days? Answer Options: 1 – Great, 2 – Good, 3 – Bad, 4 – Neither good nor bad, 5 – Very Bad
- How many days did you work in the past 7 days? (Free Text)
- Were you sick in the past 7 days? Answer Options: 1 – No, 2 – Yes but did not need treatment, 3 – Yes and received treatment, 4 – Yes and needed medical treatment but did not receive it
- If you had 15 USD, what would you spend it on? (Free Text)
There’s a whole lot more that can be written on whether well-being is the impact all interventions should ultimately aim to promote, or whether it is a proxy for a host of more complex impacts (empowered + safe + well fed, etc). Without delving into this more philosophical question, what is heartening about DataKind’s analysis of this data is that it shows the potential for supporting other organizations trying to assess impact.
So how could DataKind’s support be applicable for an organization working on peacebuilding? Three questions come to mind immediately. First, the organization would need some clarity around the impact questions that need to be answered. What are the fundamental impact questions for peace projects? Are we interested just in people’s perceptions of well-being? Of safety? Are there some objective indicators that can also help? A well-defined, concrete impact statement (often lacking) would need to be the starting point of any analysis (by DataKind or others!).
Second, previous DataKind support projects have not focused on making the causal link between a particular intervention implemented and a broader impact observed. Likely reason: no-one’s asked for that kind of support. Analysis that tries to suggest a causal link is messier, and thus exactly the type of task where organizations would benefit from collaboration with data scientists.
Third, data to answer these type of impact questions is hard to come by and not everyone has the capacity to run an SMS perceptions survey. Most organizations will collect some data directly, but perhaps this could be complemented by non-traditional data sources as DataKind did for MIX. Can we scrape the web for useful data on peace? Could online traditional media, social media and other online ‘data exhaust’ (online searches?) provide some useful inputs?
As organizations working on peace try to get closer to assessing the impact of their interventions, collaborations with data scientists will become increasingly important. DataKind has the potential to become a strong partner to the peacebuilding community, contributing to more robust, evidence based results.