We often get questions about our approach to starting impact management and measurement (IMM) projects. IMM has been growing quickly in recent years, especially with social impact entrepreneurs and startups (some of our favorite clients). Yet, the subject is often nuanced and difficult to completely visualize within your own programs. It’s important to ask all the difficult questions as you get started. In this post we are going to use a favorite client of ours to demonstrate the early stage of our process here at Basil Data.
Before we dive in, we want to emphasize our reliance on the 7 Social Value Principles established by Social Value International. Broadly, IMM can be applied to many social solutions, and at times, be overwhelming. We lean on these principles in order to maintain integrity in our projects. The 7 principles are as follows:
- Involve Stakeholders
- Understand what Changes
- Value the things that matter
- Only Include what us material
- Do NOT overclaim
- Be transparent
- Verify the results
We encourage our clients to maintain these principles while establishing the best strategy for their programs. Learn more about the principles on our About Us page or check out all the great resources Social Value US has to offer.
Now, let’s jump in!
In the first phase of our process, what we call the Discovery Phase, we establish a foundational understanding of the problem and the change our clients are working to accomplish. Whether you are a social impact startup or a small nonprofit, we start each project the same way.
Let’s break down the Discovery Phase while using our friends at Noomee as a reference.
We had the pleasure of working with Noomee when they were selected as a participant for the Rally Social Enterprise Accelerator. Every year, Rally selects a handful of startup companies from around the world that are focused on creating positive social change. Noomee is the developer of a phone application that provides digital therapy to children with ASD or special needs. Their app utilizes ABA (applied behavioral analysis) therapy in the software to help children across the US develop their daily life skills and encourage a higher level of independence.
The Discovery Phase consists of four main components: Theory of Change (ToC), current state research, alignment with global standards and indicators, and identifying impact metrics.
Theory of Change
When we first meet with social impact entrepreneurs, we work to understand the foundation of their mission. This is accomplished by developing (or clarifying) a client’s Theory of Change. This provides Basil Data was a more thorough understanding of the client’s process, target beneficiaries, and overall impact objective. The Theory of Change lies at the heart of every social impact start up and non profit. It should not be quickly glossed over. A few questions we may ask include: What change do you hope to see? What is the current state of your community? How will you measure your contribution (see principles #2 and #5)?
We love working with motivated entrepreneurs working on social change. We were inspired by Noomee right away. Noomee was founded by Gayane Hovhannisyan and Elina Mirzoyan, two occupational therapists based in Armenia. We learned that Noomee was in the early stages of developing their app. And they wanted to identify opportunities in the US to provide digital therapy services to children with ASD, especially underserved communities. Gayane and her team established a general impact goal to increase access to daily therapy methods and accelerate the progress toward independence for special needs children and families.
Current State Research
After defining an impact goal, it is necessary to establish baseline statistics and current programs relevant to the impact objective. Of course, clients serve as the subject matter experts of their startups, they understand the problem much better than we do. Our role is to gather relevant data and provide a clear understanding of statistics related to the problem. This research will help serve as a comparison, or baseline in an absence of a formally established counterfactual or control group. As development continues, and the client continues to grow, the baseline statistics identified in the research phase can be used to more easily monitor outcomes over the years.
The majority of research that Noomee has collected and conducted over the years focused primarily on the proven effectiveness of digital therapy for children with ASD or other special needs. The research we conducted at Basil Data focused more on what the current state of digital therapy in America looked like. What percent of children diagnosed with ASD are receiving therapy services? What percent of children in America are classified as having special needs? How many therapists in the US offer therapy services online? Not all of these are easy questions to answer, but we do our best to establish a current state that Noomee can reference as their internal data continues to grow.
Alignment of Global Standards
Determining the appropriate metrics for your organization can be nuanced. And although global standards have yet to be established for every impact objective, as a sector we are getting closer to standardized references. Among others, SASB, IRIS+ and the Sustainable Development Goals (SDGs) are utilized by more organizations and investors as the requests for better impact measurement standards become more prevalent. Additionally, alignment with these reporting standards and display your motivation for long term impact improvement. Such indicators serve as appropriate comparison metrics for impact investors, donors, and the general IMM community.
Noomee’s focus of providing digital therapies for children with ASD aligns well with the high level Sustainable Development Goal 3: Good Health and Well-Being
SDG Target Indicator 3.8 specifically states: “Achieve universal health coverage including financial risk protection, access to quality essential health-care services and access to safe, effective, quality and affordable essential medicines and vaccines for all.”
Even though ASD and therapy are not explicitly mentioned, Noomee’s mission closely correlates with this target, given their dedication to providing an effective and easily accessible product for underserved populations. The Noomee team can reference this global goal with interested investors as well as stakeholders to simplify their impact message.
Metrics and Indicators
Now that we have a better understanding of overall goals and current state of the problem, the next step toward accurate impact measurement is identifying which metrics and indicators will provide appropriate insight into the client’s progress. We start by considering a range of metrics that could apply to the problem. Internal metrics, sector indicators, as well as any indicators around positive and negative externalities are all considered in order to accurately monitor a program’s outcomes.
In order to maintain a comprehensive perspective, we encourage metrics and indicators within the 5 data categories: Impact, Outcomes, Outputs, Feedback, and User Demographics. The identified metrics and indicators will serve the client even after our Discovery Process comes to a close.
Because Noomee was focused on skill development and independence as a major feature of its application, we defined their metrics based on this target outcome.
- User Demographics are specific to the characteristics of the beneficiary: gender, age, diagnosis, region, family income, etc.
- Feedback metrics would rely on client satisfaction and qualitative improvement. Noomee can use in-app reviews as well as direct surveys to gather this information. Feedback is mandatory in order to maintain an understanding of what matters to your stakeholders (principle #3)
- Outputs provide insight into engagement – frequency and duration – in the application.
- Outcomes, in the case of Noomee, are established to monitor the overall speed of skill development, and will utilize the app’s internal data to show progress.
- Impact metrics are often more difficult to measure, but with Noomee, the long-term plan came down to comparing the level of independence of those who utilize the app to the status quo.
Once the components of the Discovery Process have been completed, Basil Data summarizes all the information we have gathered into a powerful report along with recommendations to move forward. E.g. How to collect the appropriate data, where data will be stored, and who will be responsible for collecting it. This report belongs to the client whether we decide to continue work with Basil Data or start the journey with an internal team.
After the Rally Social Enterprise Accelerator was complete, Noomee took their Impact Summary Report and merged it with their investor pitch deck. They could clearly define their opportunity for impact, impact key performance indicators, and strategy for increased impact going forward.
“Everything needs to be measured, and most importantly, our Impact. With Basil Data, Noomee got started on the right path to measure impact and create tools for making it even better” – Gayane Hovhannisyan (Noomee Founder)
The Discovery Process is a great way to begin your understanding of your company’s impact measurement, but your impact measurement strategy can, and should, continue to be refined and expanded upon as you grow. Your dedication to improving the impact never stops. Similar to a financial return on investment, impact investors are requesting to know the social return on investment. As a social impact entrepreneur, you are motivated to see change in the world, and improving your impact represents change. Without understanding how to measure it effectively, how do you know you’re making the difference you set out to make?
Interested in learning more about Basil Data? Have more questions of impact measurement? Please reach out to us! We would love to hear from you.