How worried should we be about AI?

We brought 150 chairs, non-executive directors and governance professionals together in London on 30 September for our first Governance Summit.

Pippa Begg, our CEO, opened the afternoon by setting out why the question won’t wait. A couple of weeks earlier, Jensen Huang of Nvidia had told the world that artificial general intelligence was here. “New models are pushing the boundaries on cyber security, breaking free of their sandboxes and frightening everyone, including their developers, with their capabilities,” she said. “We’re underestimating the pace with which it’s developing. It’s moving far faster than most of us realise.”

Four speakers answered her question over the course of the afternoon. Rob Hornby, co-CEO of AlixPartners. Begg herself. Kim Crawford Goodman, CEO of Smarsh. The polling expert James Kanagasooriam. Between them they put the danger in four different places.

Governance leaders put their own AI questions to the speakers after each session. 

Rob Hornby: “We don’t have an ability to govern what we’ve created”

Rob Hornby, co-CEO of AlixPartners, has spent his career in technology and now co-runs one of the world’s leading consultancy firms. He played down the idea that AI is a threat to human survival.

"I think there’s a great deal of very irresponsible speculation that takes place, sometimes for very cynical reasons.," he said. “My take is that no model is capable of having enough influence in the real world to threaten our species.”

He acknowledged they could do considerable harm, and said the incidents reported over the past year came from a predictable place. “Almost everything that is going wrong is actually within the frontier labs themselves,” he said. The labs, he added, have built something they don’t know how to fully control. "It's a bit like having a rabbit in a hutch in your living room, where you left the door of the hutch open, and your living room door open, and your back door open, and your garden gate open. And then you were astonished to find the rabbit was in the forest."

Asked how worried he is personally about AI, Hornby pointed to his own preparations. “I did buy an emergency radio, some tinned food, and bottled water. But that was because I was worried about power cuts.”

AI isn’t ready to take responsibility — but nor are we

Having taken the extinction question off the table, Hornby focused on a different, as yet unsolved, problem: nobody has worked out how to govern what we’ve built.

“We’re not fully in control of what we’ve built. And what we’ve built is powerful. It has some ability to operate in the real world. And it has absolutely no morality or responsibility that it exercises over itself. It’s all guardrailed. It’s all from us. We don’t have an ability to govern what we’ve created. And that’s what we need to fix.”

Board Intelligence’s Board Value Index found that 98% of directors say their board has engaged with the question of which decisions should stay human-led and which could be delegated to AI. Only 49% have moved from discussing their approach to implementing it. Most boards have had the conversation, but only half of them have done something about it.

Hornby segments artificial general intelligence into four distinct categories. The most sophisticated of these considers moral vision and social conduct, and on that measure, he said, “AI is nowhere. It’s not even started.” His conclusion for boards follows from that.

“AI should not be making consequential decisions on its own, because it has no ability to take responsibility or process an intrinsic moral position.”

Pippa Begg: “What worries me is bad adoption”

Board Intelligence CEO Pippa Begg approached the question from a different perspective. "It's not the pace and the potential that worry me," she said. "What worries me is bad adoption."

Begg’s first concern is that boards consistently underestimate what the technology can already do. "The mistake we're making is that we're judging AI on what it could do 18 months ago. We need to judge it on what it can do today and assume that's the floor, not the ceiling."

She pointed to how people are already using AI tools. ChatGPT's single biggest use case is companionship, and 39% of young people report using chatbots for emotional support. A board still debating whether a chatbot can handle call centre queries is asking a question the market answered some time ago.

Her second concern is what bad adoption costs. When anyone can produce large volumes of information at almost no cost, good thinking becomes scarce and more valuable. "AI slop is all around us. Information, words, images, data have never been cheaper. Judgement, clarity, and curiosity are in short supply. High-quality thinking has never been worth more,” Begg added.

A board paper that AI has padded out reads fine and decides nothing. We built our own AI tools around that risk.

Report Writer structures a paper around the questions the board needs answered, which is the discipline set out in our board reporting methodology, the QDI Principle. As a writer drafts, it spots gaps in the argument, pulls insight out of the supporting material, and shows what to fix. Writers get a paper that has done the thinking, and they get it in hours rather than days.

Pippa Begg closing the day with the Ulysses contract, and an instruction to put quantum on tomorrow’s agenda. 

The performance gap between good and bad directors will widen

Begg's sharpest prediction concerned the people around the table. "Over the next few years, board members are going to start losing their jobs because of AI. Not because AI replaces them. But because AI will expose the gap between the directors who add value and the ones who don't."

She described the two ends of that gap. "The great director turns up with deeper research, sharper questions, and better ideas. The lazy one finds it easier than ever to wing it, having read a summary of the papers and asked AI which questions to ask."

Her analogy for the second group was unsparing. "It would be like going to university, refusing to read a book, and expecting to come away with a first-class degree."

The underlying problem here is one of format rather than effort. Directors work hard, sit on several boards, and prepare to make big decisions under real time pressure. The average board pack now runs to around 220 pages, and no document of any length can anticipate every question a director will bring to it, pull in every relevant external source, or recall what the board decided about the same issue four years ago. In our Board Value Index, 84% of directors say the quality of the information they received has caused a delayed, rushed or poor decision in the past six months.

Directors can get more out of their preparation time by using AI tools that let them interrogate the pack directly, question by question, and pull in context the pack doesn’t carry. Begg went further. She expects investors, regulators, and other stakeholders to start asking whether a director who refuses to use them is fulfilling their fiduciary duty.

Kim Crawford Goodman: “AI is really a partner to human ingenuity”

Kim Crawford Goodman runs a global software business, Smarsh, that serves regulated industries, and she has watched several technology waves arrive. She acknowledged the worry and then moved past it.

"We are talking a lot about the dangers, and those dangers are real. We have to be thoughtful about it. But there is so much opportunity for value and good."

Her warning was aimed at leaders who treat buying the technology as the hard part.

"I see so many of my customers who will listen to the power of AI, might even buy it. But then when you go back in and talk to them in six months, twelve months, they're still using their traditional methods, because they feel really comfortable with their traditional methods. They've not put the human ingenuity into how you have to transform the process, the workflow, the outlook."

That philosophy recurred through her session. "AI is really a partner to human ingenuity," she said. The organisations she sees pulling ahead are the ones whose people redesigned what they do around the tools. The ones falling behind bought the software but kept the same process.

Her advice to boards is shaped by a career spent watching this pattern repeat. "Every technical wave that comes, the winners are truly innovative." The rest, she said, lean on their reputation and their existing position, and you can watch them falling behind.

James Kanagasooriam: who automation hits hardest

Political polling expert James Kanagasooriam spends his working life trying to measure, understand, and forecast public opinion. He moved the question outside the boardroom, towards what AI does to the societies all companies operate in.

He has modelled exposure to automation by constituency and by occupation. "In all the left behind areas, all the areas that were very strongly represented during the UK Brexit referendum, they're actually all the areas that are least exposed to automation. The opposite is true of ‘Remain’ areas." The places most exposed are graduate-heavy, often in cities, and often wealthy.

Kanagasooriam links that to political instability. "When you get, as it were, excess elites, where you get people who previously were having it good suddenly feeling left behind, you get political volatility."

He pointed at somewhere this is already visible. "If you're wondering why the situation in data centres has got so volatile in the US, even in areas like Texas: this is lots of people who are downwardly socially mobile, who have lots of letters after their name, who are looking at a life that is structurally much less well off than their parents."

His read on what determines whether a society can absorb this is blunt. "The answer is growth. But the problem is that to do a lot of the things that produce growth is, in the short term, unpopular."

The people James describes work inside these companies. A director signing off an automation programme is making decisions about graduate-heavy, well-paid roles in their own organisation, and about the workforce they will still need in five years.

So, how worried should we be?

Experts like these may approach the question from different angles but they’re agreed on one thing: that we have more to fear from moving too slowly than from the technology itself. With that in mind, Begg offered one clear takeaway for the 150 governance leaders in the room.

"The thing we mustn't forget is that we have agency. If we step back, this technology will be done to us, and we'll spend the rest of our careers managing the consequences. If we lean in, we shape what it's used for and where the limits sit. This is not a job for the technology team or politicians. That’s governance.”