Mistakes Boards are making
While 88% of organisations now use AI in at least one business function, board governance hasn’t matched that pace. Only 39% of Fortune 100 companies disclosed any form of board AI oversight as of 2024, and 66% of directors report their boards have “limited to no knowledge” about AI. This governance gap creates a dangerous disconnect: while executives champion AI’s transformative potential, they remain blind to its unique failure patterns.
AI doesn’t fail like traditional software: it fails like judgement, advice, and authority, exposing weaknesses in oversight that boards never anticipated. The most significant risks aren’t technical; they’re governance risks rooted in seven common assumptions that feel reasonable but prove fundamentally incompatible with AI’s probabilistic, fluent, and dynamic nature.
Recent failures illustrate this pattern: New York City’s AI chatbot advising entrepreneurs they could serve rodent-nibbled cheese to customers, McDonald’s drive-thru AI ordering 260 chicken nuggets for confused customers, and Air Canada being forced to honour incorrect refund information from its chatbot. These aren’t isolated technical glitches, but predictable outcomes of flawed governance assumptions.
For boards serious about AI governance, understanding and correcting these seven assumptions represents the critical difference between AI as a strategic asset and AI as a reputational liability.
The Seven Invalid Assumptions Undermining AI Success
The following framework identifies the most dangerous assumptions boards make about AI oversight. Each assumption appears reasonable within traditional software governance but creates specific blind spots when applied to AI systems.
| Assumption | Why It Feels Reasonable | The Core Flaw Why It Is Wrong | Real-World Consequence |
|---|---|---|---|
| 1. “If it’s deployed internally, the risk is low.” | Internal tools feel controllable and aren’t exposed to the public. | Internal use doesn’t mean internal impact. Internal chatbots and decision-support systems influence staff judgement and behaviour, often invisibly. | Staff act on incorrect or incomplete advice, and the organisation cannot trace how decisions were influenced, leading to untraceable operational errors. Example: NYC’s MyCity chatbot gave city staff illegal business advice that could have been passed to entrepreneurs. |
| 2. “We can put guardrails around it and that will be enough.” | Policy documents and system prompts feel like reliable control mechanisms. | Guardrails manage behaviour; they don’t guarantee outcomes. AI operates probabilistically. Guardrails reduce risk but cannot eliminate the inherent small chance of failure. | The system behaves correctly – until the edge case scenario that matters most, such as a critical regulatory or safety breach that wasn’t anticipated in the guardrail design. |
| 3. “If it’s wrong, someone will catch it.” | The human-in-the-loop model provides reassurance and appears to offer final oversight. | Fluency isn’t accuracy – but humans treat it as authority. People trust confident-sounding systems more than they realise, especially under time pressure. | Errors pass through human review precisely because AI output sounds confident and reasonable, leading to the publication of authoritative falsehoods. Example: Air Canada’s tribunal case, where staff relied on chatbot advice despite company policies. |
| 4. “This is an IT issue, not a board issue.” | AI appears to be software, and software traditionally falls under IT department oversight. | When AI speaks, it speaks for the company. AI affects advice, decisions, and external representations across legal, finance, and operations – not just technology. | No clear accountability when problems arise because governance structures never adapted to technology’s cross-functional impact. |
| 5. “The vendor is responsible for the AI’s behaviour.” | Outsourcing technology feels like transferring associated risks to external parties. | You can outsource technology; you cannot outsource responsibility. Regulators, courts, and customers hold your organisation accountable for outcomes, regardless of vendor relationships. | Contractual protections fail to shield organisational reputation or regulatory exposure when third-party models cause harm. Legal liability remains with the deploying organisation. |
| 6. “If it’s compliant today, it will remain compliant.” | Traditional systems behave consistently once they pass initial compliance review. | AI compliance is a process, not a checkbox. AI systems drift as data, prompts, and usage patterns change, altering model output over time without explicit updates. | Systems that passed initial compliance reviews quietly become non-compliant or biased through normal operation, with no internal alerts indicating the change. |
| 7. “We’ll know when it becomes a problem.” | Executives expect major issues to escalate through standard internal reporting channels. | AI failures are often subtle, cumulative, and discovered after impact. Systems fail in ways invisible to internal dashboards, manifesting through external consequences. | Issues surface via customer complaints, regulatory enquiries, or media attention, by which time the risk has evolved into a full-blown reputational crisis. |
From Recognition to Action: The Board’s Governance Mandate
Recognising these flawed assumptions is only the first step. Current research shows only 12% of companies feel very prepared to assess AI governance risks, with 42% lacking policies for employee AI use. The boards that successfully navigate this challenge share a common approach: they treat AI as a probabilistic partner in decision-making rather than deterministic software.
Effective AI governance requires abandoning the assumption-based approach for a framework built on four evidence-based principles:
1. Impact-Based Oversight
The Principle: Focus on potential impact rather than deployment location, complexity or cost.
Example Implementation: Classify AI systems by their decision-making authority and potential consequences, not by whether they’re internal or external. A chatbot providing legal advice to staff carries the same risk as one serving customers directly.
Board Action: Establish impact thresholds that trigger governance protocols. High-impact systems require board oversight regardless of technical architecture.
2. Probabilistic Risk Management
The Principle: Protect the downside; design for detection, mitigation, and rapid response.
Example Implementation: Build monitoring systems that detect when AI output deviates from expected patterns. Create rapid response protocols for when, not if, failures occur.
Board Action: Require management to present failure scenarios alongside success metrics. Ask: “What’s our response when the 0.1% probability event happens?”
3. Clear Accountability Mapping
The Principle: Define who remains accountable for AI decisions, regardless of vendor contracts or departmental boundaries.
Example Implementation: Create decision maps showing who owns outcomes when AI influences choices. Ensure contracts specify accountability rather than just technical performance.
Board Action: Demand clarity on who answers to regulators, customers, and courts when AI-influenced decisions cause harm. Accountability cannot be delegated to algorithms.
4. Continuous Compliance Monitoring
The Principle: Treat compliance as ongoing monitoring, not one-time validation.
Example Implementation: Deploy systems that track model drift and output quality changes over time. Establish trigger points for compliance review when systems evolve beyond initial parameters.
Board Action: Require regular compliance health checks that focus on output patterns, not just technical specifications. Ask for evidence of continued performance, not assumptions.
The Path Forward: Governance That Matches the Technology
The future of AI governance lies not in controlling the technology but in managing the assumptions and governance around its use. Board oversight of AI has tripled to only 48% of Fortune 100 companies in 2025, but oversight alone isn’t sufficient – it must be the right kind of oversight.
Boards that succeed in AI governance share a crucial insight: they govern probabilistic systems probabilistically. They plan for failure modes they can predict and build detection systems to protect against those they cannot. They understand that AI’s value lies in its fluency and speed, while its risk lies hidden.