Summary:
- Community Impact Indicators are used to demonstrate how effectively a CoP is achieving its strategic objectives which align to its Shared Value.
- Being able to report on Community Impact Indicators enables a Community to grow and evolve while remaining focused on its strategic purpose. It also provides evidence which can be used to secure funding and sponsorship for the Community.
- Community Impact Indicators are defined on the basis of a Community’s Theory of Change.
Resources:
What are Community Impact Indicators?
While Community Health Metrics are used to track and assess the health of a Community in terms of how well it is performing - for example: member growth, active participants - Community Impact Indicators are used to demonstrate how effectively a CoP is achieving its strategic objectives which align to its Shared Value.
In other words, while Community Health Metrics describe what a Community is and what it does, Community Impact Indicators describe what the Community achieves.
Refer to the article about Community Health Metrics for a detailed comparison between Community Performance Metrics and Community Impact Indicators.
Why are Community Impact Indicators important?
Community Impact Indicators are key to enabling a Community to grow and evolve while remaining focused on its strategic purpose. They are also key to generating evidence which can be used to secure funding and sponsorship for the Community. With this in mind, Community Impact Indicators are a key component in reports aimed at sponsors (for example, senior management), stakeholders, and members, especially the Core Group:
Sponsors and stakeholders - Community Impact Indicators help sponsors and funders in particular visualize and document the measurable impact of a Community of Practice, enabling them to justify the decision to support a CoP on the basis of its potential impact. Later on, Community Impact Indicators help show sponsors that their investments are being impactful.
Members - Evidence of impact is reason for celebration. Celebrating the results of the Community’s efforts galvanizes members into action, especially when members feel acknowledged for their contribution (important tip: be intentional about recognition!). It also helps potential members visualize the value of a Community, providing an incentive for them to join.
Core Group - As the Core Group brings together the members who are most committed to the creation and success of a Community - Community Impact Indicators are particularly important to them as they allow them to make informed decisions about the general direction of a Community. In other words, Community Impact Indicators enable the Community to orient itself in the search for the most impactful path.
“Across key stakeholder groups, [impact indicators] can help measure and track levels of understanding of what the [institution] does, degrees of trust in the institution, and the extent of its influence. Tactically, [impact indicators] can be used to gauge the impact of different engagements with stakeholder groups, providing teams with insights that help [communities] achieve their goals] most effectively.” (“A Guide for the Measurement and Evaluation of Communications at the World Bank,” World Bank Group.)
How are Community Impact Indicators chosen?
Community Impact Indicators should be defined on the basis of a Community’s Theory of Change. The Theory of Change describes a Community’s “journey of impact” in terms of a process of change and how the change happens in the short, medium and long term to achieve the intended impact of the Community. As a Theory of Change defines long-term goals and then maps backward to identify necessary preconditions, Community Impact Indicators can ideally be set to monitor each step in the process of change: inputs/activities, outputs, outcomes, and indicators.
In terms of the Theory of Change, impact indicators focus on outcomes and impact.
Refer to the Theory of Change article for a more in-depth description of the process of designing a Theory of Change for your Community.
Examples of Community Impact Indicators
The following table shows examples of inputs/activities, outputs, outcomes, and indicators.
Inputs/Activities | Outputs | Outcomes | Indicator |
Hosting events – BBLs, High level events | Number of events hosted
Number of participants | Increased level of support to client countries to implement the reform | The number of IR projects being supported by WBG IR Practitioners |
New content posted on the CoP platform | Files, documents downloaded | Knowledge of members increased | Number of files/articles shared
|
Developing publications and tools | Number of publications | Knowledge of members increased | Number of views and/or downloads of publications |
Courtesy of Patrick Kabuya, Senior Financial Management Specialist and Community Lead for the Integrated Reporting Community of Practice at WBG
For another example of Impact Indicators, see the Article titled Community Impact Evaluation using the KSB Impact Framework.
Sample indicators by target groups
As outcomes are defined by the impact on various types of target groups, you can set distinct indicators for each target group. The example below shows impact indicators for members of the CoP, clients of the WBG (other organizations in the international development sector may call this target group “beneficiaries”), and the organization that hosts the CoP (this is in reference to the concept of Shared Value).
Beneficiary | Indicator |
Members | Increase in knowledge Increase in work program Increase in reputation/position/advancement |
Clients (individuals/institutions) | Increase in adoption of IR/ST reforms Increase in investment in programs Ability to tell a better story |
Organization | Improved ability to develop better programs Increase in number of programs in the pipeline |
Considerations when measuring the impact of a Community of Practice
Attribution vs Contribution
Attribution in the context of Communities of Practice can be a challenge: as Communities operate in broad and complex environments, it is not easy to attribute a specific impact - e.g.: a changed behaviour, a new policy implemented - to the activities of a Community.
With so many factors at play, it is very hard to draw a straight line between a single activity and say “that was the key factor that effected the change.” It is better and becoming increasingly common practice to refer to “contribution” not attribution. For example, “increasing the supply of teachers contributed to the increase in reading levels. ”
Short-Medium-Long Term Indicators
Even if you are able to attribute impact, you might not be able to measure it right away. Impact takes time. Also, impact is defined in different ways depending on the time frame: impact in the short term is not the same as impact in the long term. For example, you might be able to improve the teaching skills of a group of teachers in the short term, but it will take a much longer time to achieve the ultimate impact of improving the performance of their students. In designing the Community Impact Indicators for your Community, don’t rush to the finish line, but rather set indicators for each stage in the Theory of Change. This will allow your Community and its sponsors to understand and appreciate the complexity of a Community’s journey of impact as well as to , and celebrate each step of the journey.
Stories
Stories are also very important; they add qualitative data to bolster your Theory of Change and show impact. In addition, when you have sufficient stories, you can identify trends and usefully extract quantitative data from that qualitative data. Refer to the Stories article for a more in-depth description of the value and process of creating Stories.
Reports
Always Report out – publish your results using data and stories for maximum effect. Refer to the article on Reporting for a more in-depth description of creating reports for your community.
