When AI Capability Moves Faster Than Governance
When AI Capability Moves Faster Than Governance
A provocative question for boards of commercial, public and governmental organisations: Is your organisation increasing its AI capability faster than its ability to understand, govern and safely absorb it?
And there may be an equally important second question: Is AI capability developing faster than the human and organisational capabilities needed to use it wisely?
In his new essay, We Must Pace the Frontier, Anthropic CEO Dario Amodei argues that AI capabilities may now be advancing so quickly that safety mechanisms, governance and institutions risk falling behind.
His argument is primarily about frontier AI. He points to the growing ability of AI to contribute to the development of the next generation of AI and argues that this recursive self-improvement could significantly accelerate technological development. His conclusion is not that AI development should stop, but that advances in capability need to be matched by advances in alignment, evaluation, interpretability, operational reliability and governance. This raises an important question far beyond frontier AI companies.
What does it mean for boards?
Capability, safety and governance need to advance together
Boards increasingly need to understand not only whether their organisation is adopting AI, but how quickly its capabilities are evolving. AI is moving from tools that support individual tasks towards systems that can reason, act, collaborate with other systems and operate with increasing degrees of autonomy.
That changes the governance challenge.
It is no longer enough to ask whether the organisation has an AI policy, a risk framework or appropriate controls. Boards should increasingly be asking:
- How quickly are our AI capabilities actually advancing?
- Is our ability to understand, test and oversee those capabilities developing at the same speed?
- What evidence would make us comfortable moving to the next level of autonomy or deployment?
- Where should there be explicit checkpoints before capabilities are scaled further?
- How are we considering the environmental and social consequences of AI as its use and capabilities scale?
- And are we governing only the risks, or also ensuring that responsible governance enables innovation and value creation?
But there is another capability gap
A recent MIT report on AI in teaching, learning and research training adds another important dimension. Although written in the context of education, its implications extend far beyond universities.
MIT raises the risk of what it calls cognitive surrender: when people increasingly hand over thinking to AI rather than using AI to strengthen their own ability to think, learn and exercise judgment. This matters for organisations too.
AI can increase productivity while simultaneously weakening some of the capabilities on which organisations depend. Employees may produce faster analyses, better presentations, more code and more polished recommendations without necessarily developing the underlying expertise required to question assumptions, identify errors or exercise independent judgment.
The result could be a paradox: organisations become more productive while becoming less capable.
MIT therefore argues for augmentation rather than simply automation. The relevant question is not only what AI can do for us, but what humans should continue to learn, practise and become better at doing because AI exists.
For organisations, that raises fundamental questions about expertise, learning, leadership and organisational design.
Where should AI automate work? Where should it augment people? Which capabilities do we deliberately want humans to retain and strengthen? And how will tomorrow’s experts develop if AI increasingly performs the work through which today’s experts learned?
Keeping humans capable of being in the loop
This moves the discussion beyond the familiar principle of keeping a human in the loop. A human presence provides little protection if that person no longer has sufficient understanding, experience or confidence to challenge what the AI proposes. The challenge is therefore not simply keeping humans in the loop. It is keeping humans capable of being meaningfully in the loop.
This has implications for professional development and organisational learning. Some of the tasks AI can most easily automate — first analyses, research, drafting, coding, preparing alternatives and working through difficult problems — have traditionally also been how people develop expertise. What looks like inefficiency can sometimes be learning.
Organisations therefore need to consider not only the productivity created by AI, but what happens to the pathways through which judgment, experience and expertise are developed.
This also applies to boards themselves. AI can help directors summarise board papers, analyse competitors, explore scenarios, generate questions and challenge assumptions. Used well, this can significantly augment board capability.
But there is another possibility. If directors increasingly outsource reading, interpretation and questioning to AI, a board could appear better informed while gradually weakening its capacity for independent judgment.
Human interaction becomes more, not less, important
There is another important insight in the MIT work. As AI makes individual knowledge work easier, human interaction may become more valuable rather than less.
Organisations do not create value simply by aggregating individually productive people. They depend on discussion, challenge, trust, collaboration, creativity and collective sensemaking. AI may allow individuals to work increasingly independently. But if efficiency comes at the expense of dialogue, apprenticeship and collaboration, organisations risk creating more productive individuals but weaker collective capability.
This should concern boards because culture, leadership pipelines, succession and organisational learning are all affected.
Governance should not simply follow innovation
The traditional pattern in many organisations has effectively been: Innovate → deploy → govern.
As capabilities become more powerful and the pace of change accelerates, that sequence becomes increasingly problematic.
We may instead need something closer to: Explore → govern → test → learn → scale → govern again.
Amodei proposes a similar principle for frontier AI: when systems reach particular capability thresholds, advancement should be accompanied by evidence that corresponding safety requirements have been met.
There is a useful principle here for corporate boards. Governance should increasingly be dynamic and capability-based rather than something designed once and periodically reviewed. But the capability being governed is not only technological capability.
Boards increasingly need to consider four capabilities evolving together: AI capability → Human capability → Organisational capability → Governance capability
If AI capability accelerates while the other three lag, a widening capability gap emerges. The answer is not necessarily to slow technological development. It is to accelerate the organisation’s ability to understand, learn, adapt and govern alongside it.
This is about taking risk, not only avoiding it
There is also a danger that discussions about responsible AI become primarily discussions about restraint. Boards have a wider responsibility. They need to help their companies take appropriate risks, not merely avoid risks.
Strong governance should create the confidence to experiment, invest and scale while recognising when capability is beginning to outrun understanding. The board therefore has to hold opportunity and risk simultaneously.
- Where should we accelerate?
- Where should we experiment?
- Where do we need more evidence?
- Where has the organisation developed sufficient capability to scale?
- And where might we need to pause before moving further?
This is increasingly part of strategic leadership, not simply risk oversight.
Building Dynamic Board Capabilities
This connects closely with our research on Dynamic Board Capabilities.
Our research identifies three capabilities that boards need in rapidly changing environments: Sensing, Pivoting and Aligning.
- Sensing helps boards identify emerging opportunities and threats.
- Pivoting turns those insights into strategic choices and possible shifts in direction.
- Aligning connects those choices with the organisation, resources and execution needed to create results.
AI provides an increasingly powerful test of all three. Boards need to sense technological developments early enough to understand their implications. They need to pivot as new possibilities challenge existing assumptions, strategies and business models. And they need to align technology, people, organisational capabilities, governance, investments and safeguards so that technological potential can be translated into sustainable value.
This suggests that boards should not measure AI progress primarily through adoption rates, number of use cases or productivity improvements.
They should also ask:
- Is AI making our organisation more capable?
- Are our people developing stronger judgment and expertise alongside AI?
- Are we creating new ways for the next generation to learn and develop?
- Are we strengthening collective sensemaking and collaboration?
- Do we know which capabilities we want AI to replace, augment or leave deliberately human?
- And is the board itself becoming more capable of governing an increasingly AI-enabled organisation?
The challenge is therefore not simply keeping governance up with AI. It is continuously increasing technological, human, organisational and governance capability together. As AI becomes more capable, competitive advantage may not belong simply to the organisations that adopt it fastest. It may increasingly belong to those that learn fastest — those that know what to automate, what to augment, what to preserve, when to accelerate, when to challenge and when to pause.
For boards, that is becoming a strategic leadership responsibility. Perhaps the most important question is therefore no longer: How much AI are we using? But: Is AI helping us build a more capable organisation?
Continue exploring
Read Dario Amodei: We Must Pace the Frontier
Why rapidly advancing AI capabilities need to be matched by safety, evaluation and governance.
Read MIT: AI and Education Report
Why increasingly capable AI requires us to rethink learning, human capability, judgment and the relationship between augmentation and automation.
Read our research on Dynamic Board Capabilities
How boards can strengthen their ability to sense change, pivot and align as the environment evolves.
Upcoming events and programmes
The China Factor in Europe’s Competitiveness — 18 September
A board perspective on China’s impact on European competitiveness, technology and the net-zero transition.
European Boards Growth & Resilience Forum — 7 October, Amsterdam
Connecting sustainability, technology, geopolitics and competitiveness to strengthen European growth and resilience.
Boards Oversight of Responsible AI for Value Creation — October Cohort
Helping boards govern AI strategically and responsibly while enabling innovation and value creation.
Navigating Uncertainty — starting 22–24 October, Ljubljana
A six-month journey for experienced leaders navigating transition, uncertainty and their next portfolio of contribution.
About Digoshen
This blog post was originally published on the Digoshen blog and the personal blog of Digoshen founder Liselotte Engstam.
At Digoshen, we help leaders and boards navigate an increasingly complex world shaped by AI, digital transformation, sustainability, geopolitical change, and evolving workforce dynamics.
Through research, executive education, thought leadership, coaching, and international networks, we translate emerging trends into practical insights that strengthen leadership, governance, innovation, and long-term value creation. We believe that the future belongs to organizations that successfully combine technological intelligence with human wisdom.
If you are a board member, executive, or leader interested in shaping the future of your organization, we invite you to explore our articles, research, podcasts, learning programs, and international communities.
You are also welcome to discover the Boards Impact Forum, chaired by Digoshen founder Liselotte Engstam, and Novisali, her artistic practice exploring leadership, reflection, and human experience through art.
Find more insights on the Digoshen website, explore Liselotte Engstam’s research on Google Scholar, and follow Digoshen on LinkedIn.