Key Performance Indicators#
We use Key Performance Indicators to measure and track our progress and impact as an organization. These are tracked along with other important metrics in our KPIs dashboard site. Here are links to the major sections of that site:
Goals and principles#
Our KPIs have the following goals:
Signal our impact as an organization
Signal our efficiency as an organization
Be measurable either quantitatively or qualitatively
Be useful for internal stakeholders to guide our strategy and actions
Be useful for external stakeholders to help them assess our success
As a general rule, the strategic objectives that we set should have an explicit influence on our KPIs.
KPIs do not have the following goals:
Assess individual performance
We currently follow these principles in defining our KPIs:
Be measurable at our current capacity. We should not require extra team members or services in order to measure our KPIs.
Start with the basics. We should not get too-complex in our KPIs because we are new to this and still learning how to use them. We should define three or four at most.
Should be automatable. We should be able to automatically generate our KPIs given data we have about our communities. This is because we have limited capacity to measure and track right now.
How are KPIs used#
We use our KPIs in two ways:
When defining our strategy and objectives, they should be tied to an expected impact on our KPIs.
In quarterly reports to external stakeholders we should occasionally highlight our KPIs to provide updates about our progress.
The following KPIs are reflections of our impact as an organization. They represent a positive outcome that is aligned with our mission as a result of our actions.
The number of unique communities served with at least one hub in the last 6 months.
Monthly Active Users
The number of unique users that have used one of our hubs over a 30-day rolling window.
The 10th, 50th, and 90th percentiles of MAUs across all hubs.
See this issue to track this effort: 2i2c-org/infrastructure#1888
Monthly revenue, broken down by category (grants, development contracts, cloud service contracts)
Monthly operational costs
Monthly operational costs, broken down by category (engineering, partnerships and community, executive, fiscal fee, services)
There are a few ways in which we’d like to expand the use of KPIs as we learn more. Here is a quick overview for future reference:
Our KPIs are currently defined at an organizational level, but most decisions about strategy and policy happen within a functional area (e.g.
We may wish to define KPIs within functional areas as well in order to make it easier to tie strategy and policy to an impact on a specific KPI.
Cloud cost per hub user, broken down by hub type (education vs. research): for each hub,
monthly cloud costs / monthly active users.
issues closed - issues opened.
Average time to completing an issue once somebody is assigned.
Average time it takes to deploy a new hub or cluster from scratch.
Community guidance KPIs
FreshDesk Turnaround Time
nbgitpullerclicks per hub
Our metrics above are all “automatable” and quantitative due to resource constraints. However we recognize that much of our impact will be qualitative rather than quantitative. For example, we might run satisfaction surveys with our communities to see how we are doing and ask where we can improve. We will need to grow capacity and skills at running surveys or interviews at-scale in order to assess these more qualitative KPIs
All of our KPIs thus far are focused on the communities we directly serve with cloud infrastructure. However, a secondary aspect of 2i2c’s mission is to support open communities that underlie this infrastructure. We commit to doing most of our work by performing upstream contributions, and to support upstream projects in non-coding ways as well. While we do not currently track our upstream contributions, we aspire to track this information automatically as part of our self-assessment.
We track some upstream activity in our upstream KPIs page.
Our KPIs are currently fairly basic, because they are immediately measurable and quantifiable. They are largely a reflection of usage, though this is not the same thing as impact. Below are some harder-to-measure KPIs that we may consider for future strategic development:
Publications or presentations that have been generated as a result of use of our hubs
Communities served that come from organizations or backgrounds that are historically marginalized.
Cost reduction given to these communities in a way that is still sustainable for 2i2c.