Drive improvements by avoiding vanity metrics
Jan 8, 2020 - 2 min read
Measuring social impact is normally undertaken to attract funding, but its biggest benefit is driving positive change within your organisation. To unlock this potential, you must avoid vanity metrics.
Vanity metrics is a term used in the startup world. It describes any metric which does not relate to the business' goals and cannot be used to inform decisions. Perfect examples include website views and social media followers. Increasing website views may make you feel good, but it does not measure a business goal and does not offer any guidance on how to improve.
To assess change, you must first understand how well the goals of your organisation are being achieved. Remember, change is not always beneficial, without monitoring your performance, a change could be detrimental. Once performance monitoring is available, changes can be made and their effect on your performance observed. Metrics which tie actions to observed results are called actionable metrics.
In social impact, an organisation's outputs are vanity metrics. Outputs detail what your charity has done or produced. For example, how many interventions or workshops have been conducted. Outputs are easy to measure and are important in understanding your impact as they describe your reach. However, they are not useful for learning and improvement, as they rarely relate to your goals. I have yet to meet a charity whose goal was to run workshops, they are just the chosen medium for achieving social change.
To illustrate the point, consider a charity which performed 100 workshops last year and 90 the year before. Clearly we can say that they are outputting more, but is this good? Are the workshops helping towards the organisation's goals? Are the beneficiaries actually benefiting from these workshops? Without being able to answer these questions, we cannot assess change.
Outcomes describe the result of your organisation's efforts. For example, the improvements to a beneficiary's well-being as a result of an intervention. Measuring outcomes allow a view into the success of an intervention, and as such, there is scope to learn and improve. It is key that the outcomes being measured do relate to your goals, otherwise any efforts to improve them will be misplaced. Tools like theory of change allow mapping from long term-goals to outcomes, ensuring they are aligned.
Finally, to learn from your outcomes, you must understand what actions affect them. For example, if a new intervention is introduced, did it outperform the status quo? To be able to answer this, your outcome reporting must be capable of reporting on different segments of your data. Segments are a subset of your data which have similar attributes. In this example, the data could be segmented based on the type of intervention, allowing comparison between the new intervention and the incumbent. With this capability, organisations can innovate quickly by trialling new approaches and assessing their performance.
To improve as a sector, we must take a leaf from the startup world and focus on actionable metrics. In the case of social impact, measuring outcomes which are defined based on your goals is a great start. The next step is to make them actionable, by understanding how your actions affect your outcomes.
Impactasaurus is a simple tool for soft outcome measuring and reporting. Data within Impactasaurus can be tagged, allowing reports to be generated for different segments. If you would like to take the next step, signup now and see how easy it can be.
