Post #78
May 1, 2024
Tamara O’Brien
FW writer Tamara O’Brien digests the issues from Claire’s recent webinar on the uses and abuses of AI in sustainability reporting. Corey Walrod, from the ISSB’s technical staff, gave the inside track on the new IFRS sustainability standards; Diana Rose, ESG Research Director at Insig AI, explained the pros and cons of common types of AI and how corporate reporters should approach them; while Adrian Clark of RBC BlueBay explored the uses and limitations of AI in making judgement calls about potential investments.
I’m indebted to The New Scientist for this arresting image: “A cat that is both living and dead may be hard to swallow.” The cat, of course, is Schrodinger’s. The indeterminate feline made its appearance in an article about what quantum physicists call the ‘many-worlds’ interpretation of reality. Think multiverse on steroids.
I won’t go into that here, but it did make me think what might occur when AI meets sustainability regulation. The infinitude of perspectives! The notion that regulation can both obscure and illuminate! The fractal consequences of interoperability in parallel jurisdictions! And the most fundamental question of all: what is real?
These were the themes hovering around last Thursday’s webinar. I, of course, was in the happy position of observer, but those whose day job brings them up against these issues proved more than up to the task of making sense of them.
Claire prefaced proceedings with observations she has made from what’s now over a year’s worth of intensive investigation into the use of AI in corporate reporting.
She boiled said observations down to a useful shorthand. ‘Good AI’ is when reporters use it as a tool for filtering key information from the vast quantity of datasets available, that would be impossible for humans to process within the reporting cycle – and let’s not forget there may be more of this with the new standards. ‘Bad AI’ is when reporters clutch at it to do the difficult job of writing the narrative. Bad because narrative, supported by data, is the stuff on which investment decisions are made. A company’s narrative, of which its annual report is the foundation, first of all must be accurate (and we know AI can find this a challenge). And second, in the context of the annual report, it’s supposed to represent the opinion of management and the Board, and should reflect the purpose, values and sustainability of the company as whole. Which, Claire reminded us, is the point of corporate reporting. If we can’t really trust what companies are telling us, we might as well all pack up and go home.
The panel, carefully chosen from the commercial, regulatory and investor worlds, were asked not only for their diverse expertise, but their ability to use their imagination alongside their reasoning abilities. Something that’s essential for emerging fields like AI and new sustainability reporting requirements.
Filling the black hole of ESG reporting
The first speaker, Corey Walrod, is a finance professional who’s been deeply involved in developing the IFRS’s sustainability disclosure standards, through its International Sustainability Standards Board (ISSB). When ISSB asked report preparers, users and other stakeholders what they’d want to see in a global baseline of sustainability-related disclosures – analysts and investors made a strong call for decision-useful and comparable information; everyone wanted an end to the spiralling growth of voluntary sustainability initiatives over the last 20 years; and they also wanted the streamlining of disclosure points and terminology. All of which are being baked into the standards. Ultimately, the IFRS aims to bridge the gap between traditional financial statements and sustainability information; meeting the needs of investors and the wider stakeholder audience with more connected, integrated content.
It’s all relative
Picking up on Claire’s concept of Good/Bad AI, Diana Rose, ESG Research Director of technology company Insig AI reminded us that it’s not the tool that’s good or bad, but how humans use it. And to use AI responsibly, we must first understand how it behaves. So: LLMs, large language model systems, are the basis of many tools currently on the market. This technology has been trained to process language using artificial neural networks. They perform gazillions of computations that eventually form a statistical model of a language’s pattern, such as syntax. They can classify and generate text by making predictions based on those statistical patterns: in such-and-such a context, this word is most likely to follow that word.
Sounds a bit like the infinite-monkeys-writing-Hamlet theorem. But now, thanks to computers, we have enough monkeys. So what’s next?
Machine learning
We don’t normally go into technical details in FW blogs, but Diana’s explanation of the fundamentals of AI was so clear and helpful, I thought it worth the detour. She introduced us to the two main natural language processing (NLP) technologies: BERT (Bidirectional Encoder Representations from Transformers, to its friends) and GPT.
Google developed BERT in 2018 and it’s been driving Google’s search engine ever since. It’s trained by making it predict masked/hidden words, based on the context given by the words either side of it in a sentence (hence bidirectional). Good at: understanding context, sentiment analysis, classifying text.
GPT was developed by OpenAI around the same time, and is behind the now-familiar ChatGPT released in 2022. GPT is trained on predicting the next word in a sentence, given previous words. So it only goes in one direction. Good at: generating natural-sounding text, summarising, translation.
Similar beasts then, but quite different in how they’re used. Which is where good and bad AI comes in.
At InsigAI, Diana uses BERT to help clients analyse sustainability reports and make evidence-based decisions. ‘We designed these tools because we recognised that, until ISSB is adopted properly, the volume of disparate reports being published made it increasingly difficult for humans to read and digest – let alone synthesise and analyse and do what humans do best – without the aid of technology. It was becoming a barrier to good due diligence and research. The bar was also being raised by emerging regulation in sustainability reporting, and the noise around this was starting to present a risk of greenwashing.’
With BERT, subject matter experts label tens of thousands of sentences with their relevance to a certain topic – climate change, for example. (Diana herself helped train 15 NLP models on ESG topics.) Data scientists use this marked-up material to produce models called classifiers. You then pass fresh text, such as a report, through the ‘climate change’ classifier, and each sentence is scored by its probability of being relevant to climate change.
Downside? The classifiers are only as good as the quality and quantity of the material they’ve been trained on. Which is down to human choice, and the inevitable bias that comes with that; even when those humans are SMEs. Still, these are brilliant research tools. Companies preparing reports can look at where they have gaps compared to their peers. Investors looking at sustainability risks and opportunities can identify macro trends. Regulators can use them to assess the impacts of policy interventions and investigate greenwashing.
At InsigAI, this is the point at which BERT bows out. It’s now up to humans to analyse, interpret and assimilate this filtered and classified data.
And what of our friend GPT? Here Diana struck a much more cautionary note. Insig AI are testing the waters to see what ChatGPT might be reliably capable of, and what it can do that BERT can’t.
‘ChatGPT is quite good at navigating, synthesising and summarising information. But it’s vital that you set strict parameters on the data source, such as “only look at these sections of the corporate report”. And that you word the question very carefully. Even so, the same query on the same data can produce a very different answer.
‘This is where we have to use our imagination. Going beyond BERT, Chat GPT can summarise and structure information, it can tabulate and reformat, and could be suitable for generating draft templates for more formulaic documents – but we have to be careful. Basically if it’s not Good AI, it’s Bad AI.’
Crystal balls
Humans have always felt that the ability to predict the future is the one thing that would make us richer, safer and happier. Imagine how different the world would be today if 10 years ago we’d known there’d be a global pandemic.
So it’s no surprise that AI’s potential in this regard is being explored with enthusiasm in the investor sector. And when it came to the investor viewpoint, Adrian fully endorsed Corey’s emphasis on ‘decision-useful’ information. An investment manager’s decision to buy, sell or hold a security, he explained, depends on their view of the future, and whether that security is going to be worth more or less than it is currently. The annual report and other company communications – plus the manager’s own research and information from the media – feed into that analysis. Beyond that, myriad sources from meme stocks (shares that have gone viral on social media) to Reddit chatrooms, all drive views and narrative about a company.
But in this morass of information it’s what the company says about itself, in numbers and the analysis of those numbers, that is the bedrock of reliability. For Adrian, how companies arrive at those numbers doesn’t matter, as long as they’re accurate. And if Good AI can deliver this accurate data, great – although he wouldn’t trust GPT-style LLMs to do so.
And he certainly wouldn’t substitute AI for the human judgement and intelligence that analysts use to pull out decision-useful detail for investors and shareholders; or that business leaders use to explain their thinking through reporting on everything from strategy and risk to purpose and ESG. I’m sure analysts can sniff out inconsistencies in that story in a second.
Question time
Amazingly the 45-minute webinar was nearly up, and if I understood quantum theory better, I’d make some cute observation here about the Problem of Time. Suffice to say there wasn’t much left of it for audience questions. One that did squeak through was about change management. What’s the best way for us to encourage the good use of AI, and discourage the bad, within our firms?
Claire recalled that when she was doing her AI focus groups last summer, she’d mentioned that of course, you shouldn’t put confidential information into a public chatbot like ChatGPT. And while a lot of the corporate reporters present nodded in agreement, more than a few people said, ‘Oh, I never thought of that.’ So there’s a clear and present danger that companies need to be on top of.
The panel was in agreement. When it comes to data, if you’re satisfied that the tool you’re using will deliver something that’s accurate and useful, whether in your reporting or operations – and you’ve managed the risks that it can’t – go ahead and use it. And to make sure you are satisfied, test-drive different models, with the oversight of your SMEs. Also, be prepared to say ‘Nothing is suitable for us right now’. Efficiency isn’t everything! Corey added that the leadership must set the tone: AI tools simply help management tell their story, and how they’re preparing for the risks facing them. There’s a huge cultural element in incorporating them in a way that adds rather than subtracts value.
AI is not I
In her explanation of BERT and GPT, Diana had concluded on a note that really chimed with me. The raw material of these tools is language, and that’s such a uniquely human power, imbued with connection, emotion, cognition, culture, artistry, redolence. So when you use something like ChatGPT, you have to keep reminding yourself that the friendly, human-like sentences appearing before your eyes are literally mindless. As Diana put it, ‘It’s not intelligent, it’s just code.’
Maybe that’s where the real test of imagination comes in. We and future generations will have to adjust our minds, emotions and reactions in new ways, according to whether something is human/not human.
I’m not sure we’re ready. Can we have another few millennia please?
------
Join the discussion
If you’re interested in corporate reporting, and would like to ask questions of the people involved in shaping and delivering it – please sign up for our quarterly Practical Perspectives on Reporting webinars.