Assessing the impact of security interventions is a challenging – yet essential – endeavour. Without adequate expertise, resources, and political will, impact assessments are unlikely to contribute effectively to rule of law programming, drug control policy, efforts to tackle organized crime, or countering violent extremism (CVE) interventions. Rigorous impact assessment, on the other hand, can stimulate innovation, and improve both the effectiveness and efficiency of security interventions – important for beneficiary communities and financially-strapped donors alike. However, effective impact assessment in volatile environments is challenging: insecurity itself complicates assessment design, implementation, and analysis.
This paper provides an introduction to the challenges of effective security intervention impact assessment. It discusses different measurement tools and methods available to security practitioners and programmers, and offers ideas for strengthening impact assessment in these fields.
Impact assessment is often not prioritized when interventions are conceptualized and planned. It is only tacked on once accountability to donors needs to be demonstrated. This makes programmatic progress difficult to measure, because no thorough baseline studies are conducted when the programmes are set up. Security practitioners are often not conversant with assessment tools and methods, measurement design, data collection, and analysis. Instead of undertaking the hard task of measuring long-term impacts, programmers often resort to measuring easily quantifiable outputs, which fail to provide insights into an intervention’s complex impact. This risks wasting scarce donor money, producing poorly-tailored interventions – or worse, doing real, but unrecognized, harm to supposed beneficiaries.
Many of the challenges discussed in this paper can be mitigated through thoughtful application and combination of different methods and tools. The second part of the paper therefore provides an introduction to basic assessment methods including surveys, experiments, and select qualitative approaches, explaining how they work, their potential advantages and drawbacks, and offering examples of their use in relation to security interventions.
The final part of the paper highlights new possibilities for more accurate, comprehensive, and thoughtful approaches to impact assessment of contemporary security interventions. These include the early integration of impact assessments into programme design, lengthened time horizons, as well as the use of mixed methods, experimental designs, and new technologies.
The paper also points out, however, that reaping the full benefits of the methodological innovations requires a corresponding cultural change, generating greater familiarity amongst practitioners and donors with different approaches to – and benefits from – impact assessment. This should be matched by a recalibration of expectations, time frames, and budgets for impact assessment.
