Jane: Hello, I'm Jane Goodland and this is the LSEG Sustainable Growth Podcast, where we talk to leading experts on a wide range of issues that touch on both the worlds of sustainability and finance. This week I had a really good conversation with Florian Berg, who's an academic focusing on sustainable investment with a specialism on ESG ratings. He's also the co-founder of the intriguingly named Aggregate Confusion Project. We covered the Good, the Bad, the Ugly of ESG ratings. So, let's find out what Florian had to say.
Jane: Hello, Florian. Thank you so much for joining us today. Now, you're a research scientist at MIT and you're focusing on ESG ratings. So, tell us more about your work and how you came to be doing what you do.
Florian: A long time ago I wrote my PhD about a very obscure niche topic, and that was the link between ESG performance and stock performance that was in, I think I started in 2009. And yeah, at the time everyone warned me that this would be the end of my career. And it's somewhat was actually because I didn't find a job. So, I went into a very traditional hedge fund in London, worked there for two years. But then I met actually Roberto Rigobon at MIT, and he worked a lot on measuring inflation and wanted to transition to measuring what he called at the time ethical behaviour of companies. And so, I said, yes, there is such a thing already. It is called ESG, but the data has a lot of problems. So how about we actually try to improve that?
Jane: And so that's how you come to be doing what you're doing. So, some of our listeners will be familiar with ESG ratings. And many of us will know that they've changed a lot over the last few decades and really, I guess from certainly from my early days of using this type of research, it was predominantly for ethical screening purposes. So, walk me through how you see the evolution having taken place.
Florian: Yeah, that certainly. That was before my days when I started thinking about that.
Jane: You make me feel old, Florian.
Florian: It was in 2009 and in 2009 we already or at the time I was a quant at Amundi and I was focused on ESG investing. And at the time we already called it ESG investing or ESG ratings. And so, at the time it was very much we looked at this from a double materiality perspective. I'm not so sure that the term was around at the time.
Jane: No, not back then.
Florian: Yeah, that I think came later. But it was certainly this philosophy and then this evolved over time. And for some it became more focused from on the financial material perspective. And those terms actually appeared, and it became definitely much more mainstream. So, no one would ever tell me that this is the end of, my career.
Jane: No, no, absolutely not. Am I right in thinking that we've seen quite a lot of changes in terms of some of the guess who's providing those ratings? So, I remember way back when it was very sort of regionally dispersed and there was the research houses focused on kind of particular areas like Europe or even single country. But then that changed, didn't it, in terms of we've seen a lot more consolidation in the marketplace and entrance of some of the bigger providers.
Florian: Yeah, at the time when I started, it was just little startups providing ESG ratings. And now actually when you look at the big financial data providers, they all have an ESG rating. And so that actually and most of them actually bought those little startups. So those little startups are still around. And by that actually they shaped the whole field of ESG quite a lot because they were starting their work in a certain culture in with certain views from the time that people started it and from also from the backgrounds of those people that started those companies or those ESG raters, they actually shaped also the methodologies. And it's really fascinating when you look at the different ESG ratings, how there's often this background from the people that started it so still alive and present in those ratings, even though all of those ratings had several methodology changes in between.
Jane: Yeah. And that takes me on to the area where I know that you specialize, which is really about thinking about the fact that there's quite a lot of dispersion and variation between ratings and indeed your I love, the name of your projects. It's called Aggregate Confusion Project, which I think is just a brilliant name for everything ESG, actually, because we're all a bit confused, aren't we? To be honest. So tell me about this. Tell me about why they differ so much. Are they measuring the same things? Are they the same thing but different? And why do we have aggregate confusion?
Florian: Yeah. So, we actually when we started six years ago, when I joined MIT, we really set out just first to understand what's actually going on in the market. That was Julian and Roberto. And so, we approached six different raters, downloaded the data and compared them, and we saw that there was a lot of aggregation in the data from the raw data to the issue indicator level data and then the dimension E, S and G, and then they all aggregated on an ESG level, and they were fairly similar in that aspect. So, in that sense, we there was a lot of aggregation, there was a lot of data, and we got a little bit confused. So, we called it aggregate confusion. So first we actually tried to explain really why there's so much difference with firm characteristics. Does it depend on different industries? Does it depend on the size of companies or other characteristics? And we couldn't find anything. So, we thought maybe the reason is just different. And that is because on the one hand, those, ESG raters, they define ESG differently and on the other hand, even if they agree on how to define it, they still come to a different conclusion because they will use different data sets and they will feed the data from a different angle and then shape a different opinion because of that. And so basically this is really the reason why there is so much divergence. So less than half is explained by the definition of sustainability. So, do I actually assess a certain issue or not? Everyone probably agrees that D&I issues are very important part of ESG, but electromagnetic field radiation might be not. That really then depends on the rater. Some do integrate it, some don't. And then once you actually agreed on what you assess, then it's also the question on how important those topics are or issues are to each other compared to each other, for example, electromagnetic fields might be less important for most of the people than DI issues. And so, this is really this definition of sustainability. And then on the other hand, which creates a most of the divergence is measurement, noise, or measurement. But that hints measurement, noise. And this measurement divergence is because what you really want to measure for example, regarding DI is this true discrimination within a company? How do actually historically disadvantaged groups are treated? But in the end, you only observe certain proxies like salary pay gap or sexual harassment cases, or how many promotions do historically disadvantaged groups get over non historically disadvantaged groups? Then we have to once we have that data; we have to aggregate it. We have to form our opinion. And these opinions certainly diverge a lot, which is very normal about.
Jane: I was just going to pick up on something that you were talking about there, which is really about this what it is you're measuring and whether or not that's a reasonable proxy to what's really going on. And I know that, there's some things which are very difficult to get a line of sight on because actually just the disclosure is not sufficiently, available, right? So, sometimes a proxy might be, the presence of a particular policy. Let's carry on the theme of diversity and inclusion. You might have a policy on diversity inclusion, but its application may be pretty limited. So, it's this notion between is like what to actually measure is simply having a policy good enough. And in many cases, we've seen that you can have a policy all you like, but it doesn't really actually make a difference in terms of practice. So, tell me about how much that is important in terms of the overall usefulness of the ESG ratings.
Florian: I think this was the start. At the start it was really easy. Curators were mainly gathering information because this was just not possible easily at the time, and this was something that little startup could scale. And it would be very costly for investors to have also just people gathering that data. That obviously changed with the advent of AI and especially generative AI. In the future it will be very easy and very costless for people to gather different data points. So, I think the reason why what's so important about ESG raters is that here they really, for example, when a company has a policy, they really need to figure out and decide how serious the company is about this policy. So, I call it intentionality. For example, yeah, DI, is the company really reducing discrimination or is it just hiring people from historically disadvantaged groups? Regarding CO2 emissions, the CO2 emissions of now are not necessarily that important. The thing that is really important is how much does the company want to reduce CO2 emissions in the future? Are they actually doing everything they can? In some industries it's harder, some industries it is easier. So, are they doing everything they can to in a reasonable way to reduce their CO2 emissions? This is one aspect, and the other aspect is context, what I call context reality, and that's regarding DI issues that's also very important. For example, if you have a rise in sexual harassment cases in a company that if it's a very if the company had zero sexual harassment cases and then there's a strong rise, it might be actually a positive thing because women are allowed to speak up. They were allowed to file complaints. There may be things put in place so they can, and they can do so safely and that's why they report those sexual harassment incidents. And so, this this data always has to be put in context. Also, pollution regarding polluting upstream is much worse than downstream. And I think this is also this is where ESG raters really have the added value actually in the ecosystem.
Jane: When you're talking, it's really bringing into focus the fact that ESG research and maybe the ultimate kind of score or rating or number or grade or whatever it is that results in, the real value is in the underlying research in terms of what it can tell the user about that particular entity, whether it's a company or whether it's some sort of government or whatever the ratings on. And so, I think for many years people have been so keen to say take that top aggregate rating and say, show me the alpha or, where's the performance coming from. And so, what would you say to those people? Are they looking for something that doesn't exist?
Florian: Yeah. So, the alpha is a very good question. It's still a very debated topic in academic research. And so, the idea is why is alpha is that there's a mispricing. So, markets do not take into account ESG information sufficiently. And that's why if you do, the idea is then you generate alpha. Now, if you look at most of the markets, they have so many ESG ratings, and there are so many people that have basically all big asset managers by ESG ratings. So, I don't think that story necessarily holds. Then when you when you actually integrate. So, there's two ways of integrating is to use either because of risk or because of mispricing or you use it as a preference because you just say, I want to hold stocks that have a good footprint or that companies that treat women in a better way. And of course, you have the right to do so. And there's there are lots of people that think like this, but this if then you would buy those stocks and that would push up the price, then you would face lower expected returns. In financial theory, that means you would actually have a lower return in the future in this fund. But because you have basically you have to pay for the fact that you have better companies that you that you like. And so what in a lot of studies actually that show academic studies and I think there are some meta studies that count more than 3,000 studies where people regress sort of stock performance and accounting performance on ESG. But so far, I've never really came across a study that really controls for all the omitted variable biases. And here a large omitted variable bias would be, for example, inflows. We had a massive shift in preferences in the last five, six, seven, eight, nine, ten years where people actually started really buying those ESG stocks, high ESG stocks, and that would push up then the price is higher. So, it might have been that all those studies and I'm not saying that I have the answer to it, but it might have an alternative explanation is, and a lot of people think that's actually the more likely explanation is that those stocks got pushed up, those high ESG stocks. And so, the prices became went up. And by that also the realized returns over the last eight, nine, ten years that you see. But that would mean once this preference shift levels off and that will happen at one point, then you would actually those funds would actually see a lower return going forward.
Jane: So in terms of if we're thinking about kind of ESG ratings and where we've got to lots of evolution over the last few decades, and we've got to this point where ESG ratings are used by many, many investors. How would you summarize what ESG ratings are good for? Because we've touched on some of the challenges associated with them and some of the divergences. So, what should people think? You know, what are they good for? And then we can think about what they're not so good at.
Florian: Yeah, I think they are great for expressing your preferences. If you think if you don't in some ways also to bring the issues a little bit closer to you because without ESG ratings, you don't know what companies actually do in their supply chain and or in terms of CO2 emissions. So, and you might actually disagree in very strongly so especially academia thought for a long time that the only thing that matters is financial return in your investments. But I don't think that has ever been true. And here's just a simple example. You have one investment that gives you 10% and the other 10.5% in a year. And with a 10.5 investment, you have to watch a video where you see little kids in a mine using playing around with mercury. And you know that they will not survive for very long because they exposed to certain heavy metals, in a very excessive way. You would think very, very few people in the world would take the 10.5 return investment. So, in this sense I think academia got it a little bit wrong. But that has changed in the last five, six years. There's actually more and more papers that show that people do have this what we call non-pecuniary preferences. And so, they're very good for hat actually just to express your preferences in your portfolio choice.
Jane: Is that because, is that because actually quite a lot of them are backward looking, so they don't really have a predictive power? Or do you think they do have predictive power?
Florian: That really depends on the on the rate. I do think going forward, as I said, this is really this transition where ESG raters used to be just data gatherers and now they actually have to have more and more opinions, not only on the ESG rating level, but also on the different issue level. I think this will then be more and more about going forward. And some raters are more there, and others are more in the looking backward. So, it really it really depends. But I mean that you have to unfortunately, even as a small investor, if you want to express your preferences, you have to actually figure out on your own. There's a lot of written about the different methodologies. But yeah, and if you just buy a retail fund, then the, the investment manager will actually take care of this for you.
Jane: And so building on that, what are ESG ratings not good for? What should people not expect to get from using an ESG? Or when I say people investors, basically?
Florian: Yeah. I think one thing that is very misunderstood and when you look under the hood, it's very clear that they're actually very bad as a risk rating. And here I mean, really specifically the ESG rating. I think if on the issue level, it's very this is a different picture. So, the ESG rating is very cannot use cannot be used as a risk rating is because they're all actually when you look at all the methodologies from all the big providers, they're all aggregated in a linear way. This is just a weighted average between the issues. That means their trade-offs. Or put differently, you can calculate how much a company can pollute more in CO2 by hiring X amount of people, from historically disadvantaged groups. And that shouldn't be the case. That's really an immoral question to ask.
Jane: So just to clarify on that a little bit. So, what we're saying is, is that underlying an overarching ESG rating or score, what's underneath all of that are separate judgments about, how it how a company manages its human capital, how it manages environmental issues, how it sits on governance, all that sort of stuff. And all of those things are just aggregated up into a single measure. And what you're saying is, is that single measure is not a very effective, overarching indicator of, say, risk. For example, you've got to look below the top level.
Florian: Yeah. For example, it's really odd that you, anyone would think that you can compensate the risk of oil spills for ExxonMobil with discrimination. If ExxonMobil goes to an investor conference and says, Oh, don't worry about more oil spills because we have a great DI performance, everyone would start laughing. Right. And that's but this is actually what is about the those aggregate ESG ratings. The problem is here that it's really this weighted average that you could actually create different ESG ratings where you that are more risk focused, but you would need a different aggregation function for that. Then where they're actually good is that in the dis aggregate level, so, on the issue level or indicator level that is then a different perspective, right? Because then you actually know what the ESG rating itself can tell you is that there's a problem right somewhere, or that the company overall is actually fairly good. But then when you dig a little bit deeper and you look at the different issues, then you see, Oh yeah, maybe ExxonMobil has some problems with oil spills, but they're good at DI discrimination. So now we actually bring that, mix that with our financial analysis and look at what that actually means for our portfolio.
Jane: Makes sense. So, it strikes me that there's a whole lot of effort going into measuring ESG and thinking deeply. And there's people like you in universities thinking deeply and doing doctorates on ESG. So, we're measuring the life out of ESG. My big question is, is whether or not it actually creates real world impact, because a lot of people who work in sustainable investment and sustainable finance, fundamentally are driven by trying to create real world outcomes. So, tell me, Florian, you're the wise one. Is all of this measurement making any difference?
Florian: That is a great question. And, you know, in my environment, I have a lot of people that are very critical about ESG. And this is actually the discussions I cherish most because this is where I actually see what the big questions are when we discuss it. I think the big main overarching question of all this sustainable investing is does it crowd out policy? And that's academic speech for saying the fact that we all think that the private markets take care of, for example, bringing the CO2 emissions of companies down through the sustainable investing. Does that prevent or give an excuse to regulators to not have slap a price on CO2 emissions, a tax on CO2 emissions? Because that would definitely have a much bigger impact and much more needed than just all this. Yeah, CO2 focused investing. I think this is the big question but can also go in the other direction. And I do think there's also some signs that it's crowding in policy. So doing the opposite because we see actually that some companies in some industries, they take a position as a leader because they are pushed by the sustainable investing, and they show that it's actually feasible to reduce CO2 emissions and the others maybe just don't want to. And so then the regulator can actually look at the leaders and say, Oh yeah, actually we can regulate. And then he starts regulating or she starts regulating afterwards. So that could also happen. And I think this is really one of the big questions. And then the other question is, just because you identify companies that are sustainable, and you put them in your portfolio doesn't necessarily mean that it translates in real world impact. And for example, you, Jane, you are a socially responsible investor, ESG investor. So, you buy a stock because you think they have a great ESG performance. I am a traditional investor, and now I see that the stock has a higher price because you bought it. And I just think this is way too high. So, I sell my part in the stock. So now actually the the stock, the certificate or the digital number just exchanged hands. So, you now own this own this stock. But in the real world nothing has changed because the price wasn't affected. So, I think there's still a lot of confusion around company level impact in portfolios. So, the footprint that an investor owns and actually the change this investor generates. And so, this should really actually I think they should funds should be more transparent because there are ways of getting companies to change. Right? There's investor activism. You can vote at shareholder meetings. You can also just directly engage with those companies. So, I do think there is actually yeah, there's much more transparency needed. And unfortunately, some of the biggest regulation, SFDR, in that way, doesn't really contribute to reduce this confusion.
Jane: That's probably a whole other podcast Florian. So let me take you to a closing question if I may. So, we've been talking about the use of ESG ratings, principally with as investors as the primary users. Is that where it stops, or are companies starting to use these ratings more as kind of ways of doing business? And is this something you're looking at?
Florian: So for we haven't looked at it because we got so busy with the measurement of these ratings themselves. But this is definitely on the agenda. And I do think I'm actually surprised that those rating haven't popped up on my Amazon feed when I want to buy a product. And I think that's still a lot of things that can happen here because this is then an aggregate ESG rating. It's not a risk measure. It's a preference. Right. That it would be very useful also for companies to assess the supply chain risk would be very useful.
Jane: Which is starting to happen, right? I'm starting to see sort of that being a more systematic use of ESG ratings and supply chains. But I think good point. Like you say, it doesn't seem to have really filtered through into certainly retail consumer preference signposting. And it's only starting to find its way into B2B business relationships, I suppose so. Well, it sounds like there's more work for you to do. Florian So you're going to be busy for a long while yet on these ESG ratings? I think we're out of time, unfortunately, but it's been a really interesting conversation and I think it just goes to show how, we can't look at any of this in isolation. And certainly, we can't jump to conclusions at of a single measure. It's really about getting underneath the bonnet and really seeing what information is feeding that overall view. So, thanks so much for your time. It's been enlightening and look forward to reading more of your research in the future.
Florian: Thank you, Jane, for having me. It was a wonderful morning here.
Jane: So that's it for this week's episode of the LSEG Sustainable Growth Podcast, which I hope you found as insightful as I did. And a big, big thanks to Florian for sharing his expertise with us. Now, you might have seen that we have a poll question this week, so if you've got a second, please do let us know your thoughts on that. And if you're not already following us, then please do so. And don't forget to rate us on Spotify, Apple Podcasts, or any other platform. If you've got questions, comments, or someone you'd like us to talk to next, then do get in touch by email at
[email protected]. That's all from me but watch out for another episode very soon.
