Hello, I'm Catherine Klein. I'm a professor in the Management Department here at Wharton and the Vice Dean for Social Impact, and I'm delighted to be speaking with Lisa Hall, Impact Investor at large. Lisa was formerly with Anthos Fund and also the Calvert Foundation and just has a wealth of information and background experience to share with us about Impact Investing and Impact Measurement. So Lisa, thanks for chatting. Thank you for having me here. So your experiences as an Impact Investor, you have heard the channeling this capital to businesses that are having a positive impact in the world and a positive financial return. I'm particularly interested in your thoughts and your experiences around impact measurement. We know how to measure financial returns. There are clear accepted standards, figuring out how to measure impact, and be sure that impact is happening is a real challenge. So, talk with us about what your experience as an impact investor were in trying to wrap your hands around, how do we measure impact? I think this question of how do you measure impact is at the core of impact investing. It's what makes impact investing different than traditional investments. And for me, and based on my experiences, being an impact investor, it starts with the end in mind. That you really have to be clear about your objectives and were always very clear in investing about financial objectives. We're clear about how much liquidity we want, what type of volatility we're willing to take, how much risk we want to incur, but my observation has been, investors are less clear about what their impact objectives are, what type of social returns, what type of environmental impact do they want to have through their investments. And if you're clear about your objectives for impact upfront, then it becomes much easier to measure over time. When you're unclear, then measurement can get very confusing. And I think that in the early days of impact investing, and of course impact investing has been taking place for a very long time, but the term was recently coined. But, in the early days of using this term impact investing, most investors were satisfied with measuring outputs. And they would simply count, count the number of students that had been educated when they made an education investment. They would count the number of patients that have been served by a clinic when they made a healthcare investment and counting outputs is important. I like to think of it as necessary but not sufficient for impact measurement. And we have also thought about a term that broadens impact measurement to impact management where it includes thinking beyond just outputs. And, measurement is a very important aspect of impact management, but setting objectives is the starting place. So, what's the distinction between impact measurement and impact management? So, impact management is a series of activities that includes measurement. It starts with setting your objectives. It includes measuring using metrics and thinking about outputs, but it also includes thinking about what are your target populations, who are you trying to reach, what geographies do you want to cover? It also includes thinking about things like resources. What are the resources that are required to implement the types of measurements and objectives setting that you want to accomplish? It takes into account what the technology is for being able to execute on your impact measurement. So, it's a holistic approach to thinking about impact and creating and delivering impact along the different value chains. Yeah, and you're approaching this, primarily, as an investor from the stance of the investor, right? So, not necessarily as a stance of the social entrepreneur or someone else, and it sounds like a key message is, you can't be thinking about impact measurement at the, after the fact. Alright, we've made these millions of dollars of investments. I wonder if were having any impact? That's a dangerous or discarded strategy. Right. Intention matters, and intention matters because you want to make sure that your investment objectives are aligned with the underlying activities of the fund and of the social entrepreneur who's receiving the capital. Because when there's a lack of alignment around your goal, then that's when things go awry. And, part of aligning the objectives, is thinking about what outcomes are you really trying to accomplish. And, sometimes we get stuck on the outputs and measuring the outputs and don't think about, well, what are the outcomes that we're trying to deliver. And so in the example of education, are you also concerned about the quality of education? Do the students actually learn something? I hope so. Yeah, right. And as an investor, that's our ultimate concern is, did they learn anything through the curriculum or in that classroom? Same with healthcare. Yes, it's important to measure how many patients were served, but do they actually get better? Yeah. And, it's a big challenge I think in impact management because we're not always clear about causality. And the real way to figure out causation is to do randomized controlled studies. Absolutely. And there's a lot of debate in impacts management or impact measurement about the value of randomized controlled trials or are RCT's. They're very costly and they're expensive but you have to have a evidence base to be able to demonstrate that you've really delivered the outcome. And once it's been done, hopefully, it only has to be done once, and then, you can use that study to justify lots of investment and know that you're delivering the outcome that you want to achieve. Interesting. And do you think investors, how important is it to investors to actually see these metrics, to actually see data on outcomes or to actually see an RCT or in your experience or investors, once they've made this decision to go into impact investing, how concerned are they about actually verifying impact? Yeah. No, this is an interesting debate. How concerned are investors truly about impact? And in the early days, I would argue they didn't care so much about seeing the numbers and they were very satisfied with simply getting output measures. And that partly was because there were so few options, right? The universe of impacts investments was small and so there was an assumption that if someone had chosen to do this type of investing as a fund manager or as a firm, they were doing the right thing. That must be good enough. It must be good, right? You're investing in them because they know what they're doing on the financial piece and also on the impact piece. And I think, as the field has grown more crowded, as definitions of what is impact have become blurred, that it's been even more important now to investors, to understand what is the real impact that's being delivered. And when you have a world in which there are products that are being labeled Impact being run by hedge fund managers and you have other products that are providing concessionary returns in very high-risk conflict countries, that how do people compare those two things? Right. And impact measurement and management is a way to be able to compare, and that's when the outcomes matter. And, I think it's also very personalized in impact investment. Particularly, when you're talking about retail investors, individual investors, and what matters to them? For some people, education is the primary path to prosperity and they want to drill down on educational outcomes. For others, it's a belief in healthcare, for others, the belief in access to capital and financial inclusion. And so, as the field gets more and more sophisticated and there is a greater universe of options, impact measurement and that data and understanding how to compare alternatives, and to meet the objective that you're trying to meet, the data becomes critical like that. So, what the real challenges around impact measurement, is it hard to measure the impact of an enterprise? It's really hard to compare the impact of two or 20 enterprises, especially, if they are in different districts. Yeah. Yeah. Right? And so, how do you compare the folks that are making solar lanterns, from the folks that are providing schools, or Ed Tech, to the folks who are doing vertical farming in totally different geographies? What's your thinking about that? I mean that seems to me to be a nut that's like, I don't know that we're ever going to crack that nut very well. Yeah. It's a huge challenge. It's how do you compare apples, to oranges, to mangoes, and bananas? To umbrellas. Umbrellas. It's a huge challenge for impact management and impact measurement. I think that there will always be customized frameworks. And that was the case with the work that we did at Anthos, where we really said, "We need to develop a framework that will work for the values and missions of this fund." And we drilled down on what mattered based on a set of values that was over 100 years old of a family, and one of the things that we looked at was what we called intended results. So what were the results that we were hoping to achieve? What theories of change were relevant for our investors, and how are we going to measure that? And so we did look at outputs, but we linked those outputs to some evidence base. We actually, on that particular dimension, decided to score on base risk and return. What was the risk that you were going to actually achieve your intended results, and what was the evidence that existed out in the universe that supported your theory of change? So that as you generated those outputs, you could be more comfortable that you were going to actually produce the outcome that you wanted. And we embedded that in a broader, customized framework that also looked at things like unintended consequences like what were the offsetting factors that neutralize some of the positive benefit that you were achieving. And so we dug deep into layers, and I think that because it's so hard to compare apples to umbrellas, that it requires some customization. And just customization mean looking at each enterprise, each investment on its own? I think it means applying the same theory of change to each investment, in each social enterprise. And it does mean collecting enterprise-level data. Now, in our case, we were investing in intermediaries. Our expectation was that the funds would collect that data, and it was a high bar that we set. In fact, there were some funds that said, "We're not going to do it because we don't have the resources, and it's not worth it for this one investor to collect that data and information. " And we said, "Well, the next time around, we're not going to invest if you're not willing to reach that bar." So it was a very high bar to say to our funds, collect at the enterprise level data around the impact that they're delivering. We had other funds that reluctantly bought into that idea, that it was important to collect data at the enterprise level, and later came back to us and said, "We are so glad that we did that work." That's interesting. And they build out really robust, impact management systems with the technology to support it, that allowed them to look at their entire portfolio on an enterprise level basis and make comparisons, and conclude rich conclusions that actually benefited that work, that actually made them better investors, that allowed them to see where performance aligned with creating impacts, and they use it for all their investors. They report out to all their investors, and the other investors have found it useful. So it's hard work, and it's not easy to set up. But I think, once it's in place, you actually can make it efficient, and it's incredibly useful. It pays for itself over time. Yes. And it does seem that people get better at it. They get better at learning what information they need. It may not meet all the academic standards of rigor, but it tells them a lot. What's your perspective, your experience with SROI, Social Return On Investment? I think it can be a useful tool. I think there are lots of tools in the marketplace for impact-investing and measuring impact. The one that we looked at was best available charitable option, which does allow you to develop a score and factor in other stakeholders similar to what you would do and coming out with the actual number for SROI. But we found it to be for our purposes, another tool that was helpful coming out with the same type of result. Interesting. I mean, the thing that I have observed looking at SROI was like, "Wow, there are a lot of assumptions that one has to make to come up with a number." So if I'm trying to judge how much does my organization, my company responsible for this outcome, numbers look better if I say, "I'm 90 percent responsible." That if I say, "I'm 83 percent responsible and calculating," that just seems extremely difficult. No, I think that that's true. It's very subjective. Some measurements of impact are subjective, and I think that that's okay. Well, that's interesting. I think that we have to be willing to allow in a certain level of subjectivity into the analysis. There does still need to be quantitative data. You need an evidence base. But I think it's also true that we sometimes have a sense of, is this generating good, or is it ultimately negative for a community, for individual? And that we have to be comfortable with it. I think it is as much an art as it is a science. We had both quantitative and qualitative measures in the framework that we developed in the More Than Measurement project, and I think one of the things that we haven't touched upon is measuring sustainability. It's an area where there is a lot of information, a lot of very quantifiable information around carbon footprint and carbon emissions and a lot of evidence base for things like solar lantern. That's great, and it's very data-driven. I think we also have to be willing to go out into the field and see how these are being used. One of the most informative experiences that I had recounted to me was with Clean Cookstoves, and someone going into the home of a woman who had been sold a Clean Cookstove and was using it as a storage closet. So if you were counting it outputs, you were counting that one Clean Cookstoves had been sold. And, of course, Clean Cookstoves generate a lot of good in the world, in the universe, but it's more complicated than just counting that unit sold. A friend just sent me a piece, and I think the Washington Post, on the challenges around cook stoves. I mean, so much promise but really difficult around a lot of those issues, around implementation and cost. So cook stoves are good example of the challenges. Adoption is a key component, particularly in emerging markets, where the cultures are very different than the cultures of even sometimes the social entrepreneurs who are introducing the product. And cooking, in particular, we all know it's a traditional behavior that goes back generationally. You talked earlier about impact risk. What is impact risk? So the way that we think about impact risk is the risk of whether the impact will be generated or not. It's very closely connected to evidence base, and when we thought through, when would you take impact risk? When would you invest in something where the risk that it might not deliver impact is pretty high? You would do that, or we decided that it was worth doing that when the upside was huge. A great example is nuclear fusion, which has not been proven, but the upside of success in that arena of being able to create energy without any threat to the environment, without any use of fossil fuels is really high, and we did not invest in that. But it's a good example of where the evidence base is practically non-existent. Interesting. But the upside is so high that you might be willing to take that impact risk. And when you're looking at impact risk, is this an area where prior research can be really helpful? Absolutely. I come out of the early career in affordable housing, and to me, the best example was always lead paint. And early on, there were studies done that showed when you remove lead paint from housing where it's been, that health improves, asthma goes down, learning of children goes up, that are all of these spectacular benefits, and it was one or two RCSs, Randomized Controlled Studies, that demonstrated the impact of removing the lead from homes, and that's all you needed. All of a sudden, there to be a huge crusade to remove lead paint from apartment housing. It goes back to your point about an RCT can be worth it, very expensive but galvanizing. Yes. It really can. Interesting. Thank you so much for talking with me. This is great, really informative about how impact investors are thinking seriously about impact measurement. Well, thank you. It's such a pleasure to be here. It's such an important topic for the field of impact investment. I really appreciate the opportunity to speak with you. Great. Thank you.
