How ESG Disagreement Affects Portfolios

29 SEPTEMBER 2023
BY: Lars Qvigstad Sørensen, Stig Roar Haukoe Lundeby and Paul Ehling.

Which companies are sustainable? While most would agree that a hypothetical company mining coal while violating human rights should not be considered sustainable, reality is often not that black and white. Does Meta have good or bad ESG properties? Is Warren Buffet's role as chairman and CEO an impediment to a good governance score? And what about fracking? Should companies extracting oil and gas through hydraulic fracturing receive lower environmental scores than companies engaged in traditional forms of exploration and production? These are questions that the ESG rating agencies must deal with when assigning ESG scores. 

Perhaps unsurprisingly, ESG rating agencies tend to disagree. The disagreement is substantial even for some of the world's largest corporations, which are likely subject to extensive research and analysis. To visualize the disagreement, we standardize ESG scores of each rating agency to have a mean of 0 and a standard deviation of 1. Figure 1 illustrates that the disagreement can be considerable, e.g., the difference between the score awarded to Meta by FTSE and MSCI is 4.7 standard deviations. How is it possible that the assessment of one of the world's largest companies differs by so much between two large vendors?

Conceptually, we can think of two different kinds of ESG ratings: (1) a rating that gives a high score to companies that have positive environmental, social, and governance impact on society at large, or (2) a rating that gives a high score to companies that face low financial risks from its ESG practices. The first kind of rating would perhaps be more closely in line with what most people think of as "sustainability." However, most ESG rating agencies actually claim to assess companies based on their financial risks from ESG practices, i.e., the second type of rating. The distinction is important, because many ESG issues are inherently related to externalities which will only materialize as financial risks to the company to the extent that they become internalized. For example, a coal company should not get a low environmental score unless you believe it will face stricter regulations, tax regimes, or consumer boycotts that are financially material. On the other hand, a small hotel that prides itself on its environmentally friendly practices but is expected to be inundated by rising sea levels in the near future, might get a low environmental score.

Clearly, assessing the financial risks a company faces on ESG related issues is made difficult by several factors. First, the ratings to a large extent rely on non-standardized, self-reported data. Second, we have relatively short time series of data, which makes it difficult to investigate the relationship between any given ESG issue and financial risks using statistical tools, which forces rating agencies to use their own subjective judgement about the likely relationship. Third, the channels through which ESG issues become financially material can differ a lot across issues. For instance, human rights violations might primarily manifest as financial risks through consumer boycotts, whereas poor labor practices might primarily be costly due to less productive employees. Consequently, trading off performance on different issues is not trivial.  Each of these factors leaves a considerable room for subjective judgements from specific rating agencies, which contribute to large disagreement. FTSE, for example, has apparently taken issue with Warren Buffet being both chairman and CEO, whereas Sustainalytics does not seem as concerned. On the other hand, Sustainalytics seems more concerned with fracking than other rating agencies and has given very weak ESG scores to exploration and production companies involved in such extraction processes.

Figure 1: S&P 500 Stocks with the Largest Range in ESG Ratings

To get an impression of the overall disagreement, Figure 2 reports correlations among rating agencies for companies in S&P 500. This figure shows that the lowest correlation between a pair of rating agencies is 0.21 between MSCI and Sustainalytics. The figure also illustrates that computing the average score across vendors increases correlations, as expected.

Figure 2: Correlations Between ESG Scores

In an influential paper, the divergence in ESG ratings illustrated above was labeled aggregate confusion. While this confusion is by now well known, the implications for portfolio choice have received little attention. One possible reason for this is that to answer that question, one needs to define a portfolio strategy. To assess the implications for portfolio choice, we examine to what extent the optimal portfolios differ for an investor who maximizes ESG subject to limiting the tracking error versus S&P 500 (see Portfolio Choice with ESG Disagreement: Customizing Sustainability through Direct Indexing).

For each rating agency, we construct an optimal portfolio and find that the ESG rating divergence gets exacerbated. In this regard, Figure 3 shows optimal portfolio weights for the stocks with the largest differences between any optimized pair when tracking error is capped at 1 percent. For example, using ESG scores from FTSE results in a positive active weight exceeding 2 percent for Johnson & Johnson, whereas using ESG scores from MSCI implies a negative active weight. The difference in optimal weight is substantial at 3 percentage points. For Johnson & Johnson, the signs of the active weights are consistent with the signs of the ESG z-scores for the two vendors. Those different ESG scores could again stem from different views on the lawsuits the company is facing due to the allegations that its talc-based baby powder has caused cancer.

Figure 3: Stocks with the Largest Difference in Optimal Portfolio Weight for 1 Percent Tracking Error

Importantly, the correlation between active weights of the optimal portfolios is on average less than the correlation between ESG ratings from different vendors displayed in Figure 2. Furthermore, we find that the tracking error between any two optimal portfolios is always greater than between an optimal portfolio and the S&P 500 benchmark, which is used in the constraint in the optimization. The interpretation is that two ESG-optimized portfolios are more different from each other than any of those optimal portfolios is different from S&P 500. This in turn implies that an ESG-optimal portfolio is an elusive concept: ESG is not uniquely defined and different funds carrying an ESG label may look nothing alike.

The large disagreement between rating agencies clearly represents a challenge to an ESG investor who is agnostic about which rating is "correct." To mitigate this challenge, we propose two simple remedies. First, the investor can use the average ESG rating across rating agencies rather than an individual rating. Second, the investor can penalize ESG risk as measured by the variance in the standardized ESG scores across vendors. Intuitively, all else equal, this amounts to taking large positive (negative) active weights in companies where all rating agencies give a high (low) score, and small active weights in companies where some agencies give a high score while others give a low score. Our proposed remedy, detailed in the paper along with corresponding results, would allow investors the possibility to control their exposure to ESG uncertainty and can be implemented in regular mutual funds, direct indexing solutions, or other investment vehicles. 

ESG rating divergence and its implications for security selection are pertinent issues that matter for academics, investors, and regulators. With continued growth in sustainable investing, as illustrated by the growth in UN PRI signatories and their assets under management, ESG assessments are likely to retain or even increase their importance going forward. As our work has highlighted, the choice of ESG rating agency matters a great deal for which securities get selected in an ESG-optimized portfolio. The consequence is that ESG funds using ESG scores from different vendors may differ markedly. Therefore, investors pursuing ESG goals must make sure that the scores used are aligned with their sustainability objectives, as it is getting increasingly clear that ESG is in the eye of the beholder.