Galdren

Category: Crisis

  • AI Risk Governance: What Boards Need to Know and Do

    AI risk is different from technology risk in three important ways, and boards that treat it as “another IT matter” will discover this difference the hard way.

    First, AI risk is legally novel. Air Canada learned this in 2024 when a court held them liable for their chatbot’s false advice to a customer, establishing that companies are responsible for what their AI systems communicate. Board members who assume their existing liability frameworks cover AI decisions should verify that assumption with their legal counsel.

    Second, AI risk is regulatory and accelerating. The EU AI Act entered into force on 1 August 2024, with prohibitions on specific AI practices applying from 2 February 2025 and high-risk system obligations applying from August 2026. General-purpose AI models, including foundation models, have faced their own compliance obligations since August 2025. Any board with operations or customers in the EU is already inside this regime. Australia is developing mandatory guardrails for high-risk AI applications. The UK, Canada, and Singapore are all advancing their own frameworks. Boards have a narrowing window to build compliance capabilities before enforcement intensifies.

    Third, AI risk compounds invisibly. A biased model, a shadow AI tool used by a single business unit, or an agentic system making decisions outside its intended scope may produce no visible signal until the harm is substantial. By the time the problem surfaces – through a regulatory inquiry, a customer complaint, or media scrutiny – the exposure has been accumulating for months or years.

    These three characteristics mean that AI risk requires active board governance, not delegation with periodic updates.

    Setting AI Risk Appetite: The Non-Delegable Responsibility

    Risk appetite defines the level and type of AI risk the organisation is willing to accept in pursuit of its strategic objectives. Without a clear appetite statement, management cannot make consistent decisions about which AI applications to deploy, which to restrict, and which to prohibit entirely. The result is governance driven by individual preferences rather than organisational policy.

    A useful AI risk appetite statement addresses several dimensions:

    Use case tolerance. What categories of AI-assisted or AI-driven decisions is the organisation comfortable with? Customer-facing communications? Credit or underwriting decisions? Medical or clinical support? Autonomous operational decisions? Each category carries a different risk profile and may require different levels of human oversight.

    Data risk tolerance. What types of data can AI systems process, including those operated by third parties? What are the minimum standards for data residency, privacy protection, and consent?

    Error tolerance. What rate of AI error is acceptable in different contexts? An AI that recommends products with 95% accuracy may be excellent in a marketing context and wholly inadequate in a clinical one.

    Shadow AI tolerance. What is the organisation’s position on unsanctioned AI tool usage? Zero tolerance with enforcement mechanisms? Managed tolerance with a disclosure and review process? The answer has significant implications for both risk exposure and employee behaviour.

    Agentic action tolerance. What categories of autonomous action can AI systems take without human approval? What decisions must always involve a human?

    The board should approve the AI risk appetite statement, review it at least annually, and ensure management translates it into operational policy. The statement should be a living document, as AI capabilities evolve and as the organisation’s experience deepens, the appetite should evolve with it.

    Defining Ownership: A RACI Matrix for AI Risk Management

    Clarity about who does what is how accountability is created and maintained. Organisations without clear AI risk ownership typically discover this during an incident, when everyone assumed someone else was managing the risk.

    A Responsible, Accountable, Consulted, Informed (RACI) matrix defines ownership across the AI risk lifecycle. The following structure provides a starting framework:

    Role/StakeholderResponsibleAccountableConsultedInformed
    Board of DirectorsAI Risk Governance, Risk Appetite, Strategic AlignmentC-Suite, Key Stakeholders
    CEOStrategic Execution, CultureOverall Business PerformanceBoard, C-SuiteAll Employees
    CAIO / CTO / CIOAI Implementation, Technical GovernanceAI Programme Success, Technical RiskBusiness Units, CRO/CISOBoard, C-Suite
    CRO / CISORisk Assessment, Controls, MonitoringEnterprise Risk Posture, AI SecurityCAIO/CTO/CIO, LegalBoard, C-Suite, Business Units
    Legal & ComplianceRegulatory Adherence, Ethical GuidelinesLegal & Ethical ComplianceCAIO, CRO/CISOBoard, C-Suite, Business Units
    Business UnitsAI Use Case Identification, Day-to-Day OperationsLocal AI Risk ManagementCAIO, CRO/CISOC-Suite

    Three practical observations about making this work.

    First, the accountability column should contain people’s names, not just role titles. Accountability without a named individual is diffuse and ineffective.

    Second, escalation paths need to be specified before they are needed. When a business unit identifies an AI risk that exceeds their local authority to manage, who do they escalate to, by what mechanism, and within what timeframe? Escalation paths that are unclear in practice are impossible to navigate under pressure.

    Third, the RACI should be revisited when roles change. The appointment of a Chief AI Officer, a restructure of the technology function, or a significant expansion of AI usage all represent moments to confirm the matrix remains accurate.

    The Regulatory Landscape: What Boards Cannot Afford to Miss

    AI regulation is no longer a future consideration. It is a present operational requirement for many organisations, and the compliance window for others is closing.

    The EU AI Act is the most comprehensive AI regulatory framework currently in force. It applies to organisations that operate in the EU, offer products or services to EU customers, or whose AI systems affect people in the EU. The Act takes a risk-tiered approach: certain AI applications are prohibited outright (including social scoring by governments and most real-time biometric identification in public spaces), high-risk applications face substantial compliance obligations from August 2026, and limited-risk applications carry transparency requirements. General-purpose AI models face compliance obligations that have applied since August 2025. Boards of organisations with EU exposure should have received a legal opinion on their obligations under this Act. If they have not, that is an immediate action item.

    Australia’s AI governance framework is evolving. The federal government has published voluntary AI safety standards and consulted on mandatory guardrails for high-risk AI contexts. Sector regulators including APRA and ASIC have issued guidance on AI use in financial services that creates obligations for entities within their remit. Organisations should not wait for mandatory requirements to build their governance infrastructure, the frameworks being published now signal the direction of future obligations.

    Sector-specific obligations often extend further than general AI legislation. Healthcare, financial services, government, and legal sectors all face AI-related requirements through their existing regulatory frameworks that predate dedicated AI legislation.

    Liability exposure through existing law is also significant. Consumer protection laws, privacy legislation, anti-discrimination statutes, and professional liability frameworks all potentially apply to AI-generated decisions, even where specific AI legislation does not. The Air Canada case was decided under existing consumer protection principles, not under AI-specific law.

    Boards should require management to produce an annual regulatory risk assessment that maps the organisation’s AI applications against applicable obligations and identifies compliance gaps. This assessment should inform both the risk appetite statement and the compliance programme.

    Integrating AI into Enterprise Risk Management

    AI risk does not sit alongside other organisational risks, but runs through them. A single AI failure can simultaneously generate operational, reputational, strategic, financial, and compliance consequences. Governance is stronger when AI risk sits within existing frameworks rather than operating as a parallel system that the board rarely sees.

    Established frameworks including the NIST AI Risk Management Framework and ISO/IEC 42001 provide practical methodologies for this integration. Both are complementary to existing enterprise risk management structures, not replacements for them.

    Integration involves five elements that go beyond policy alignment:

    Shared risk language. AI risks should be described using the same terms, such as likelihood, consequence, risk rating, that the organisation uses for all other risks. This enables consistent comparison, prioritisation, and resource allocation.

    Consolidated risk register. AI risks should appear in the enterprise risk register, not in a separate AI risk log the board rarely sees. Significant AI risks belong alongside cyber risk, credit risk, and operational risk.

    Integrated assurance. Internal audit, external audit, and risk review functions should incorporate AI risk into their scope. Organisations that have not extended assurance activities to cover AI systems have visibility gaps.

    Risk-adjusted approval. Capital investment decisions, product launches, and operational changes involving AI should go through the standard risk approval process, not a separate AI governance channel that operates independently.

    Board-level visibility. AI risks rated as significant should be reported to the board through the normal reporting cycle, not only when an incident occurs.

    Governing Agentic AI: The Frontier That Requires Immediate Attention

    Agentic AI systems – those capable of autonomously executing multi-step tasks, interacting with external systems, and making sequential decisions without human intervention – represent a meaningful shift in governance requirements.

    Traditional AI governance assumes a human reviews outputs before action is taken. A model recommends; a human decides. Agentic systems break this assumption because an agentic AI that can browse the web, draft and send communications, execute transactions, modify data, or interact with third-party services is taking actions on behalf of the organisation without step-by-step human approval.

    The governance implications are significant.

    Scope creep is a material risk. Agentic systems operating within broad parameters may take actions that were not intended or anticipated. A system instructed to “manage our social media presence” could interpret that instruction in ways that create significant reputational or regulatory exposure.

    Audit trails may be incomplete. If an agentic system takes a harmful action, can your organisation reconstruct what it did, why, and what data it accessed? The absence of comprehensive logging is both a governance failure and a potential regulatory issue.

    Liability allocation is untested. When an agentic AI causes harm – through a contract it was not authorised to enter, information it disclosed, or a decision it made – who is liable? This question does not have settled legal answers in most jurisdictions, which means the organisation is carrying legal risk it cannot fully price.

    Human override mechanisms must be designed, not assumed. Boards should confirm that every agentic system in use has clear mechanisms for human intervention and override, defined limits on autonomous action, and monitoring that surfaces exceptions for human review.

    The governance standard for agentic AI should be more stringent than for traditional AI applications, not less. The autonomy that makes these systems valuable also makes them more capable of causing harm without human intervention.

    Responsible AI: Bias, Ethics, and Explainability as Governance Obligations

    Responsible AI is not a values statement. It is a governance approach with direct legal, financial, and reputational consequences.

    Bias All models have some degree of bias. The question for boards is whether that bias produces outcomes that are discriminatory, unfair, or harmful, and whether the organisation has systems to detect and correct it. Amazon’s recruitment AI is a well-documented example, but similar issues have arisen in credit scoring, healthcare triage, recidivism prediction, and insurance pricing.

    Explainability The EU AI Act, Australia’s privacy framework, and sector-specific regulations impose explanation rights on individuals subject to automated decisions. A credit refusal driven by an AI model that cannot explain its reasoning exposes the organisation to legal challenge and regulatory action.

    Boards should ask management two questions. First: for which AI applications can we explain outcomes to the individuals affected? Second: for applications where we cannot, have we assessed the regulatory and legal exposure, and is that exposure within our risk appetite?

    Ethical review Boards should confirm that ethical review is a standard gate in the AI development and procurement lifecycle, not an optional add-on.

    Preparing for When Things Go Wrong: AI Incident Response

    Most organisations have cyber incident response plans. Far fewer have AI incident response plans. This is a governance gap, because AI incidents have characteristics that make standard incident response procedures inadequate.

    AI incidents may be gradual rather than sudden. A model that begins producing biased outputs, a shadow AI tool that exfiltrates data incrementally, or an agentic system that slowly exceeds its intended scope may not trigger any of the monitoring thresholds designed to detect a discrete security event.

    AI incidents may be legally ambiguous. Whether an AI failure constitutes a data breach, a product defect, a regulatory violation, or a contractual non-performance will depend on the specific facts. Incident response procedures that trigger clear legal and regulatory notifications for cyber events may not have equivalent clarity for AI events.

    AI incidents may involve third parties. If the failure originates in a vendor’s model, who leads the response? What notification obligations apply? What contractual remedies are available?

    A functional AI incident response plan addresses:

    • Detection mechanisms: How does the organisation identify an AI incident? What monitoring is in place, and what thresholds trigger escalation?
    • Classification criteria: What distinguishes a significant AI incident from a routine performance issue?
    • Escalation paths: Who is notified, in what sequence, and within what timeframe?
    • Regulatory notification obligations: Which incidents require notification to regulators, customers, or third parties, and within what timeframes?
    • Containment and remediation: What is the process for taking a failing AI system offline, reversing harmful outputs where possible, and restoring safe operation?
    • Post-incident review: How does the organisation learn from AI incidents and prevent recurrence?

    The board should approve the AI incident response plan and receive reports of significant AI incidents. An incident that recurs after a formal post-incident review is not a technical failure but a governance failure because the system that produced the first incident is still in place.

    Third-Party AI Risk: The Audit Your Lawyers Wish You Had Done Earlier

    The AI tools most organisations use are predominantly purchased, not built. This means the organisation’s AI risk profile is substantially determined by the risk management practices of its vendors, practices the organisation has limited visibility unless it actively creates it.

    Third-party AI risk audits should be a standard requirement before AI vendor engagement and a regular obligation during it. The audit framework should address:

    Data handling. Where is your data stored and processed? Is data used to train or improve the vendor’s models? What happens to your data if the vendor relationship ends? These questions have direct privacy law implications and, in some sectors, such as healthcare, financial services, and government, may determine whether the vendor engagement is permissible at all.

    Model transparency. Can the vendor explain how their model produces outputs in your use case? What bias testing have they conducted, and what were the results? What is their process for identifying and addressing bias in production?

    Security posture. What certifications does the vendor hold? SOC 2 Type II and ISO 42001 are baseline indicators of process maturity, not guarantees of security, but their absence is a significant flag. What are the vendor’s vulnerability disclosure and patch management practices?

    Contractual protections. Does the contract include the right to audit? Does it allocate liability for harm caused by model failures or biased outputs? Does it specify notification obligations if the vendor experiences a data breach or model failure affecting your data? These provisions are far easier to negotiate before engagement than after an incident.

    Lifecycle commitments. What is the vendor’s commitment to model maintenance, performance monitoring, and eventual decommissioning? A vendor who provides no visibility into their model update process is a vendor whose risk profile you cannot manage.

    The audit programme should be proportionate to risk. High-risk or high-dependency vendors warrant more intensive scrutiny than low-risk peripheral tools. But the programme should be systematic and documented, not ad hoc.

    Lifecycle Governance: Risk Follows the AI From Design to Decommissioning

    AI risk is not static. A model that is safe and compliant at deployment may not remain so as the environment changes, as the model drifts, or as the organisation’s use case evolves. Lifecycle governance embeds risk management into every phase of an AI system’s existence.

    Plan and Design. Risk assessment at this stage identifies the risk profile of the proposed application before investment is committed. Ethical review, regulatory compliance assessment, and risk appetite alignment should all occur here, not after the system is built.

    Data Collection and Processing. Data quality, representativeness, and provenance determine model quality. Governance at this phase includes data provenance documentation, bias assessment of training data, and privacy compliance review.

    Model Building and Training. Bias testing, explainability assessment, and security review should occur before deployment. Models that cannot pass these tests should not proceed.

    Deployment and Use. Go-live governance includes user access controls, monitoring activation, and human oversight confirmation. The deployment decision should be a formal approval, not an informal go-ahead.

    Monitoring and Maintenance. Ongoing monitoring detects model drift, performance degradation, and emerging risks. The monitoring framework should specify thresholds that trigger review, the process for model retraining or replacement, and the board-level reporting that occurs when significant risks are identified.

    Decommissioning. Systems that are retired should be formally decommissioned, with data deletion or archival handled in accordance with retention obligations, and documentation retained for regulatory and legal purposes.

    The board should confirm that management has a documented lifecycle governance process and that it is consistently applied across all significant AI systems, not just those that are internally developed.

    Building AI Literacy in the Boardroom

    Boards cannot govern what they do not understand. This does not require every board member to become a technical AI expert. It does require sufficient collective literacy to ask good questions, evaluate management’s responses, and recognise when the board is being told what it wants to hear rather than what it needs to know.

    AI literacy at board level means understanding:

    • The difference between machine learning, generative AI, and agentic AI, and the different risk profiles they carry
    • How training data shapes model behaviour, and why historical data can embed historical biases
    • Why AI systems can produce confident-sounding outputs that are factually wrong
    • What “model drift” means and why it matters for ongoing governance
    • What the EU AI Act and relevant local frameworks require of the organisation
    • What “shadow AI” looks like in practice and why it is a governance problem, not just an IT problem

    Treat AI education as an ongoing obligation, not a one-time orientation. AI capabilities, risks, and regulatory frameworks are evolving rapidly. A board that was adequately informed twelve months ago may have significant knowledge gaps today.

    Conclusion: Governance That Keeps Pace With the Technology

    The organisations that will harness AI’s advantages while managing its risks are those where governance keeps pace with deployment. Where the board has set a clear risk appetite that management can operationalise. Where ownership is unambiguous and accountability is named. Where regulatory obligations are mapped and managed. Where ethical risks are assessed alongside financial ones. Where incident response is prepared, not improvised.

    None of this requires the board to become a technical body. It requires the board to exercise the same disciplined governance over AI risk that it exercises over financial risk, operational risk, and strategic risk.

    The question is whether your governance is structured to give the board the visibility, the accountability mechanisms, and the decision frameworks it needs to protect the organisation and create sustainable value from AI.

    If you are not confident the answer is yes, start with the risk appetite statement. Everything else in this article depends on it.

  • Navigating the AI Crisis: A Blueprint for Resilience

    Your organisation will face an AI incident. The question is whether you’ll have a plan when it happens, or be improvising under pressure.

    As AI systems embed themselves deeper into critical infrastructure and everyday operations, the failure modes multiply: hallucinated information presented as fact, algorithmic bias producing discriminatory outcomes, exposed API keys enabling abuse, and performance failures that erode customer trust. Traditional crisis management frameworks weren’t designed for any of these. Responding to an AI crisis demands something different – a comprehensive approach that combines technical containment, transparent communication, and proactive governance, built before the incident occurs.

    I. AI Crises Come in More Forms Than Most Organisations Plan For

    The most dangerous assumption in AI deployment is that crisis means catastrophic failure. It rarely does. It more often looks like a chatbot quietly failing half its users, a recommendation engine amplifying bias no one noticed, or a content filter missing outputs that should have been blocked.

    The DPD customer-service chatbot incident in 2024 illustrates the first category: overt, reputationally damaging failure. When adversarially prompted, the chatbot swore at customers and criticised the company it represented. What made the incident worse wasn’t the initial failure, it was the absence of any path out. The bot had no mechanism to escalate to a human agent, so it kept compounding the damage with each response. The system lacked two things that should be baseline requirements: robust content filtering and a clear handoff protocol to human intervention.

    The second category is subtler and arguably more dangerous. Reported failures from banking and airline chatbots in 2024 show systems that appeared operational while quietly failing the majority of users – one airline’s bot reportedly handled only a fraction of rebooking requests while the rest fell through the gaps. These incidents were initially classified as “implementation issues” rather than AI-induced service failures, which delayed both the response and any meaningful accountability.

    Both cases share a root cause: the organisations hadn’t defined what failure looked like before deployment, so they couldn’t recognise it when it arrived.

    II. Readiness: You Fall to Your Level of Preparation, Not Your Intentions

    The organisations that manage AI crises well didn’t get lucky. They prepared systematically. By the time an incident occurs, the decisions about how to respond have largely already been made – by whoever built the runbook, ran the tabletop exercises, and defined what failure thresholds would trigger action.

    Effective pre-incident readiness starts with defining “red-line” behaviours: outputs the system must never produce, enforced through hard technical blocks rather than guidelines alone. These are trip wires. Alongside them, organisations need measurable performance benchmarks, with predefined thresholds that trigger automatic rollback or system disabling. If you haven’t decided in advance what level of failure is unacceptable, you’ll be making that decision after the fact, under pressure, with incomplete information.

    Emerging best practice formalises this into AI-specific incident response runbooks, including documented protocols that designate an incident commander, define escalation paths, and specify the exact steps for containment the moment an incident is declared. The runbook exists so the team doesn’t have to think from scratch when it matters most.

    Tabletop exercises are the mechanism that makes these plans real. Simulating an AI incident before one occurs forces communication teams, technical staff, and leadership to work through the gaps together – who has authority to take a system offline, what language gets used publicly, how quickly can a rollback be executed. Robust logging and audit trails, built in before deployment, provide the evidence needed for both internal review and regulatory scrutiny when something goes wrong.

    III. First Response: Contain Specifically, Communicate Immediately

    The instinct during an AI incident is often to either minimise the response (“it’s just a minor issue”) or overreact by shutting everything down. Neither serves the organisation well. The goal in the first 24 hours is targeted containment: name the incident, assign an incident commander, and isolate the specific capability causing harm, not necessarily the entire system.

    xAI’s response to Grok producing antisemitic and extremist outputs in 2025 demonstrates this approach in practice. The company issued a public acknowledgement, deleted offensive posts, took Grok temporarily offline, and updated both system prompts and the relevant code paths. The response wasn’t perfect, but it was specific: they addressed the problem’s mechanism rather than issuing a vague statement and hoping it passed.

    Microsoft’s revocation of compromised API keys in 2025 to cut off abuse offers a parallel lesson from the security domain. Treating the API keys as a security incident – isolating the compromised credential, revoking access immediately, and publishing hardening guidance – illustrates the kind of precision that effective containment requires. The principle applies across AI failure types: know which lever to pull, and pull it fast.

    IV. Communication: Own the Failure, Show the Fix

    Generic crisis communication fails in AI incidents for a specific reason: the public increasingly understands that AI systems don’t deploy themselves. When an organisation says “our AI did this”, it raises an obvious question: who built, deployed, and monitored that AI? Deflection reads as an attempt to avoid accountability for decisions the organisation made.

    The cases of US lawyers sanctioned over AI-fabricated citations in 2023, and the improved practices that followed through to 2025-26, make the alternative clear. Courts acknowledged apologies and credited concrete remediation steps – “no AI output enters court filings without human verification” – as reasons to moderate penalties. What worked wasn’t contrition alone. It was demonstrating that the process had changed. Stakeholders and regulators need to see that the organisation understands what failed systemically, not just what failed in the moment.

    Effective crisis communication in this context is specific about what went wrong and why, uses plain language rather than technical deflection, and centres the experience of people affected. In high-risk scenarios involving sensitive outputs or vulnerable users, this isn’t optional. The response that rebuilds trust is the one that makes clear: we understand what happened, we accept responsibility for it, and here is what we have changed.

    V. Recovery: One Incident Is the Signal to Change the System

    An AI crisis that triggers only an internal post-mortem is a missed opportunity. The organisations that recover best treat each incident as a data point that feeds back into their governance, guardrails, and monitoring systems, not just a problem to close.

    This means establishing structured incident reporting practices, even when anonymised, and using them to update the systems that failed. The AI Incident Database, which documented over 80 AI incidents in the April–May 2025 period alone, represents a positive move toward standardised reporting and shared learning across the industry. The value isn’t just in cataloguing what went wrong, but in accelerating the collective ability to recognise emerging failure patterns before they become repeat incidents.

    The risk of localised learning is quite high. Legal AI hallucinations were reported in Australia in 2025 despite well-documented, high-profile US incidents in prior years. The profession-wide safeguards and verification requirements that should have followed the first incidents hadn’t been widely communicated or systematically implemented. One incident is the message to change the system. Two of the same type, in the same industry, means the learning didn’t transfer.

    Organisations have both a commercial and a broader obligation to share what they learn. Hoarding lessons from AI failures may feel protective in the short term, but it contributes to an environment where the same incidents keep happening to different organisations, in different jurisdictions, with the same preventable consequences.

    Conclusion: Build the Plan Before You Need It

    AI crisis management isn’t a reactive capability but a preparedness posture. Organisations that navigate AI incidents well have already defined their red-line behaviours, assigned their incident escalation paths, rehearsed their response, and built the logging infrastructure that makes containment and accountability possible.

    The “Resilience Loop” this framework describes – readiness, rapid response, transparent communication, and continuous learning – isn’t a theoretical model. Each element reflects a decision that needs to be made before an incident occurs. Waiting until something goes wrong to build the plan is the same as having no plan.

    The time to build your AI-specific incident response capability is now, while you have the space to think clearly. When the incident arrives, you won’t rise to the occasion, you’ll fall to your level of preparation.

    References

    [1] PR Daily. (2026, January 29). The AI incident response plan every comms team needs.

    [2] The AI Incident Database. (2025). Incident Report 2025 April–May.

    [3] blog.abv. (2025). Incident response for AI: Who’s on the hook and what to document in the first 24 hours?

    [4] Crisis Consultant. (2026, February 12). 10 Ways AI Will Cause Unprecedented Damage this Year.

    [5] Linkifico. (n.d.). Can one chatbot damage your business brand overnight? The DPD disaster.

    [6] LinkedIn. (n.d.). AI Chatbot Crisis: When Good Intentions Meet Poor.

  • Crisis Planning for the Age of AI

    Imagine waking to discover your company’s AI customer service chatbot has spent the night advising customers to break labour laws. Or learning that your predictive pricing algorithm has systematically overvalued millions of dollars worth of inventory, forcing you to sell at massive losses.

    This is the reality facing organisations that have integrated Artificial Intelligence into core business functions without adequate crisis preparation.

    The rapid adoption of AI has unlocked unprecedented efficiency and innovation. However, this reliance introduces a complex new category of risk that differs fundamentally from traditional operational failures. An AI crisis can stem from subtle algorithmic bias, unpredictable “hallucinations“, or systemic model drift, leading to financial catastrophe, regulatory penalties, and severe reputational damage.

    The organisations that survive these inevitable failures won’t be those with the most sophisticated algorithms – they’ll be those with the most robust governance frameworks.

    The Unique Nature of AI Failure

    AI failures demand a dedicated crisis approach because they often involve “black box” elements that make immediate public explanation nearly impossible. Unlike traditional system failures where you can point to a broken server or human error, AI failures emerge from complex algorithmic decisions that even their creators may not fully understand in real-time.

    The primary failure modes that necessitate specialised crisis planning include:

    Hallucination and Misinformation

    Generative AI models can confidently produce false or misleading information. When deployed in customer-facing roles, these “hallucinations” create immediate legal liability and public relations disasters.

    Air Canada discovered this when its chatbot provided a customer with incorrect bereavement fare information. The customer, Jake Moffatt, relied on the bot’s advice that he could retroactively claim discounted bereavement fares within 90 days of travel. When Air Canada denied his subsequent refund request, Moffatt took the airline to British Columbia’s Civil Resolution Tribunal (Moffatt v. Air Canada). The tribunal rejected Air Canada’s extraordinary defence that “the chatbot was a separate legal entity responsible for its own actions,” ruling instead that companies remain fully liable for all information provided through their AI systems. Air Canada was ordered to pay CAD $812.02 in damages and fees.

    Bias and Discrimination

    Models trained on skewed data can perpetuate and amplify societal biases, leading to discriminatory outcomes in hiring, lending, or law enforcement. Unlike human bias, algorithmic bias operates at scale and with apparent objectivity, making it particularly dangerous and legally vulnerable.

    The Apple Card controversy demonstrates how algorithmic bias allegations can create immediate reputational crises regardless of their ultimate validity. When tech entrepreneur David Heinemeier Hansson’s viral Twitter thread claimed gender discrimination in credit limits, Goldman Sachs faced intense public scrutiny and regulatory investigation. Though the eventual NY Department of Financial Services investigation found no fair lending violations, the company endured months of negative coverage and had to implement costly transparency measures. The lesson: social media can amplify bias allegations faster than organisations can investigate or respond, making proactive bias monitoring essential for reputation protection

    Systemic Financial Failure

    Over-reliance on predictive models for high-stakes decisions can lead to catastrophic losses when models fail to adapt to market shifts. Zillow’s algorithmic home-buying program demonstrates this risk perfectly. The company’s “Zestimate” algorithm, designed to predict housing prices, led Zillow Offers to purchase approximately 7,000 homes based on inflated valuations. When market conditions shifted and the algorithm couldn’t adapt, Zillow found itself unable to resell properties profitably. The result was devastating: over $500 million in losses, the complete shutdown of Zillow Offers in November 2021, and layoffs affecting 25% of the workforce.

    Model Drift

    Even thoroughly tested models can drift outside acceptable parameters as their operational environment changes subtly over time. This gradual degradation often goes unnoticed until it reaches crisis proportions, making early detection systems essential.

    Four Pillars of AI Crisis Preparedness

    The Framework: Four Pillars of AI Crisis Preparedness

    A robust AI crisis plan must extend beyond traditional communication strategies to encompass the technical and operational realities of AI systems. Build your crisis preparedness with a minimum of four interconnected pillars:

    Pillar 1: Continuous Monitoring and Evaluation Systems

    Establish real-time performance dashboards, drift detection mechanisms, and data quality checks that can identify model degradation before it escalates to public crisis. This includes tracking bias metrics, hallucination rates, and performance against baseline benchmarks.

    Crisis Planning Relevance: Early detection systems provide the window needed to implement containment measures and craft appropriate messaging before failures become public disasters.

    Pillar 2: Crisis Scenario Planning and Red-Team Exercises

    Conduct regular “red-teaming” exercises specifically designed around AI failure modes. Practice scenarios like deepfake attacks, major algorithmic errors, and bias-related discrimination claims. Train response teams to handle the unique aspects of AI crises, including technical complexity and rapid media escalation.

    Crisis Planning Relevance: AI failures unfold differently than traditional crises. Teams need specific experience with technical explanations, stakeholder communication about algorithmic decisions, and managing public confusion about AI capabilities.

    Pillar 3: Crisis Messaging and Transparent Communication

    Develop pre-approved response frameworks that can be rapidly customised for different AI failure scenarios. Establish clear chains of command that include technical experts who can provide accurate, understandable explanations of what went wrong and how it’s being fixed.

    Crisis Planning Relevance: AI crises often involve technical complexity that requires careful translation for public consumption. Prepared messaging prevents technical teams from inadvertently making commitments during crisis response that create further legal or operational challenges.

    Pillar 4: Human-in-the-Loop Override Protocols

    Define clear thresholds for when human oversight must be reintroduced and establish manual override procedures that can immediately stop errant AI systems. These protocols must be tested regularly and accessible to decision-makers outside of technical teams.

    Crisis Planning Relevance: Unlike traditional system failures, AI systems can continue operating and causing damage even after problems are identified. Immediate shutdown capabilities are essential for limiting exposure and demonstrating responsible action to stakeholders.

    Case Studies: Response Strategies That Work and That Fail

    The difference between organisations that recover from AI failures and those that suffer lasting damage often comes down to their immediate response strategy.

    Defensive Responses: Lessons in What Not to Do

    Air Canada’s Accountability Denial: When faced with its chatbot’s misinformation, Air Canada attempted to argue that the chatbot was a separate entity beyond the company’s control. This defensive approach prolonged the crisis, demonstrated poor understanding of legal liability, and ultimately failed when the tribunal firmly established that companies bear full responsibility for their AI systems’ actions.

    New York City’s Defensive Stance: NYC’s MyCity chatbot provided demonstrably false information about labour laws and housing regulations, telling business owners they could take workers’ tips and that landlords could discriminate based on income source. Despite widespread criticism and evidence of harmful misinformation, Mayor Eric Adams defended keeping the bot online, arguing that public testing was necessary for improvement. This approach demonstrated a fundamental misunderstanding of the reputational and legal risks involved in deploying untested AI systems.

    Proactive Responses: Building Trust Through Transparency

    OpenAI’s Safety-First Response: Following lawsuits alleging that ChatGPT contributed to suicide cases, OpenAI implemented immediate safety improvements rather than focusing primarily on legal defence. The company expanded access to crisis hotlines, redirected sensitive conversations to safer models, and added parental controls. While legal challenges continue, this proactive approach demonstrated clear prioritisation of user safety over defensive positioning.

    McDonald’s Decisive Action: When social media videos highlighted numerous failures in McDonald’s AI drive-through ordering system, the company quickly shut down the pilot program rather than defending the technology or attempting gradual fixes. This decisive response prevented further reputational damage and demonstrated that the company prioritised customer experience over technological ambitions.

    Moving Beyond Crisis to Resilience

    The organisations that will thrive in the AI era won’t be those that never experience failures – they’ll be those that transform failures into controlled, manageable incidents through superior preparation and response.

    This transformation requires acknowledging that AI governance is not merely a technical challenge but a comprehensive organisational capability spanning legal, operational, communication, and strategic functions. The most significant AI failures are rarely pure technical breakdowns; they’re failures of governance, oversight, and crisis management preparedness.

    By implementing robust monitoring systems, practicing realistic failure scenarios, preparing transparent communication strategies, and maintaining clear human oversight protocols, you can turn AI crises from existential threats into manageable business challenges.

    Your AI systems will fail. The question is whether you’ll be ready when they do. Start building your AI crisis framework today, because the next headline about AI failure could be about your organisation.

    Get the complete SECURE-AI Governance Roadmap now and be prepared for the crisis.

  • When AI Crises Strike: Your Response Determines Everything

    Twenty-eight percent of crises spread internationally within an hour. Nearly seventy percent escalate globally within twenty-four hours. In that narrow window, your organisation’s future gets decided.

    Apple’s credit card algorithm gave women lower credit limits than men. Users discovered it, posted screenshots, and within hours it became a congressional issue. Amazon’s hiring AI discriminated against women, revealed through leaked internal documents, not company disclosure. A healthtech firm’s AI pushed 483,000 patient records into unsecured workflows, discovered externally, reported months later.

    Each organisation learned the same brutal lesson: your crisis is public before you know it exists.

    The New Crisis Reality Makes Traditional Plans Obsolete

    Crisis detection now happens externally first. Your customers, not your monitoring systems, discover AI failures. Social media posts, forum discussions, and screenshot evidence surface problems before internal teams even know they exist. You’re defending against accusations before you understand what went wrong.

    Narrative velocity exceeds approval velocity. A single tweet, AI hallucination, or screenshot can create a dominant narrative within minutes. While your team schedules emergency meetings and drafts careful responses, the story is already trending globally. Traditional approval processes become impossible when you have minutes, not days, to respond.

    AI systems introduce continuous crisis risk. Unlike traditional systems that fail predictably, AI failures are probabilistic and distributed. Bias emerges from training data. Algorithms make decisions their creators never anticipated. Problems are often discovered by users conducting normal business, not QA teams running controlled tests.

    This fundamental shift makes traditional crisis management frameworks dangerous. They assume:

    • You’ll detect problems internally before they become public
    • You’ll have time to investigate and craft measured responses
    • Technical experts can explain what happened and why

    These assumptions are now false in environments where AI systems make decisions, users share everything instantly, and algorithms fail in ways nobody predicted.

    Organisations with defined crisis management plans experience 30% less reputational damage. But only if those plans actually work under modern conditions.

    Rapid Response Protocols Built for Today’s Crisis Reality

    We’ve developed crisis communication frameworks specifically designed for this environment.

    External Signal Detection Systems. Monitor social media, forums, customer communications, and technical channels for early warning signs of AI system problems. Catch issues in the first minutes of public discussion, not after they’ve become trending topics.

    Velocity-Matched Response Protocols. Pre-approved message frameworks and designated authority structures that enable responses within minutes, not hours. When narrative velocity exceeds approval velocity, only systematic preparation saves you.

    AI-Specific Crisis Frameworks. Specialised protocols for algorithmic bias, training data problems, and emergent AI behaviour. Address the unique challenges of explaining probabilistic failures to non-technical stakeholders while maintaining technical accuracy.

    What This Delivers

    Minutes, not hours. Detect and respond to AI-related crises while you still have narrative control, before external voices define the story.

    Consistent messaging. Pre-structured responses eliminate the contradictory statements that amplify crises and suggest organisational incompetence.

    Technical translation capability. Transform complex AI failures into clear stakeholder communication without losing accuracy or credibility.

    Regulatory protection. Demonstrate systematic crisis management competence that reduces regulatory scrutiny and compliance exposure.

    Why AI-Era Crises Demand Specialised Expertise

    Twenty-eight percent of crises spread internationally within an hour. When the crisis involves AI systems making unexpected decisions, that timeline compresses further. Screenshots travel faster than explanations.

    AI failures are discovered by users, not systems. Your monitoring dashboards won’t catch algorithmic bias until customers post evidence online. Your quality assurance processes won’t identify edge cases that occur probabilistically across millions of transactions. Your audit procedures won’t anticipate emergent behaviour that develops after deployment.

    Examples happen every day:

    • Credit algorithms that discriminate based on proxy variables hidden in data
    • Hiring systems that systematically exclude qualified candidates
    • Content moderation that fails catastrophically on edge cases
    • Recommendation engines that amplify harmful content
    • Customer service bots that provide discriminatory responses

    Each failure creates three crisis vectors: the technical failure itself, the delayed discovery response, and the inadequate explanation to stakeholders who don’t understand probabilistic systems.

    Traditional crisis management assumes failures follow predictable patterns with clear causation. AI systems fail probabilistically, often in ways their creators never anticipated, with causation that requires technical expertise to explain.

    Your AI Crisis Is Already Public Before You Know It Exists

    Every organisation deploying AI systems faces the same reality: your next crisis will be discovered externally, reported instantly, and trending globally before your internal teams even know there’s a problem.

    Screenshots of discriminatory outputs travel faster than technical explanations. User-generated evidence of AI failures spreads through social media while your team struggles to understand what happened. Narrative velocity exceeds approval velocity every time.

    Regulatory attention follows failed crisis response. Agencies notice organisations that can’t explain their AI systems’ decisions. They subject them to increased scrutiny across all operations. Poor crisis management becomes evidence of inadequate governance.

    Competitive advantage flows to organisations that respond systematically. While others struggle with explanation and damage control, prepared organisations maintain stakeholder trust and operational continuity.

    In an environment where AI systems make probabilistic decisions, users discover failures through normal business interactions, and evidence travels globally in minutes, amateur or non existent crisis management will make the problem worse.

    Our Rapid Response Protocols provide the systematic framework your organisation needs to survive external discovery, narrative velocity, and AI-specific failure modes.

    Your next AI crisis is already forming. The only question is whether you’ll be ready to respond when someone screenshots the evidence.

    Get the prebuilt crisis response plan as part of the SECURE-AI Roadmap now.

  • The Five Hazardous Attitudes in AI

    Lessons from the Cockpit

    The airline industry, a pioneer in safety protocols and risk management, offers invaluable lessons for the rapidly evolving field of Artificial Intelligence. At the heart of aviation safety lies a deep understanding of human factors, particularly the psychological predispositions that can compromise decision-making and lead to catastrophic outcomes. Pilots are rigorously trained to identify and mitigate what are known as the “five hazardous attitudes”: Anti-Authority, Impulsivity, Invulnerability, Macho, and Resignation.

    These attitudes, if left unchecked, can undermine even the most skilled aviator and degrade the safety in depth approaches. As AI systems become increasingly integrated into critical sectors, from healthcare to finance and transportation, the parallels to aviation’s safety challenges become strikingly clear. The development and deployment of AI are much more than technical endeavours; they are deeply human undertakings, shaped by the attitudes, biases, and blind spots of those who create and govern them.

    This article aims to draw a direct line between aviation’s hard-won wisdom and the emerging AI safety and ethics challenges. By examining analogous “hazardous attitudes” we can equip you:

    • Recognising the hazardous attitudes helps you make safer decisions
    • Gives you foresight necessary to navigate the complex landscape of AI development
    • Awareness and self-assessment reduce risks
    • Notice these attitudes in others and know the consequences they bring

    Understanding and actively mitigating these attitudes is prevents potential harms and fosters a culture of accountability, innovation, and sustainable value creation in the age of AI.

    The Five Hazardous Attitudes in AI: Identification and Impact

    A. Techno-Solutionism (“AI can fix everything”)

    “Techno-Solutionism” in AI is the common belief that artificial intelligence is a universal panacea, capable of solving all problems, often overlooking the intricate ethical, social, and practical complex real-world challenges. This attitude assumes AI can solve any problem and that technology alone is always the best answer. It dismisses the need for human oversight or understanding the problem deeply.

    Impact: This hazardous attitude can lead to several detrimental outcomes:

    • Creates over-reliance on AI, neglecting the crucial role of human judgment, intuition, and ethical reasoning in complex decision-making processes.
    • It can result in the misallocation of resources towards AI solutions for problems where they are not genuinely needed, or where simpler, non-AI alternatives would be more effective and efficient.
    • It can blind you to the potential for AI systems to exacerbate existing societal inequalities or introduce new forms of harm if deployed without careful consideration of their broader context and implications.
      The uncritical embrace of AI as a cure-all can thus lead to costly failures, ethical missteps, and a fundamental misunderstanding of both the technology’s capabilities and its limitations.

    B. “Move Fast, Break Things” (shipping before safeguards)

    The “Move Fast, Break Things” mantra, once a Silicon Valley ethos, translates into prioritising rapid deployment and innovation over thorough testing, robust safety protocols, and proper safety checks. This attitude often stems from competitive pressures or a desire to be first to market, leading to a rushed development cycle where teams view safeguards as impediments rather than essential components of responsible innovation.

    Impact: The consequences of this hazardous attitude are significant and far-reaching.

    • It can lead to the introduction of AI systems that are inherently biased, unreliable, or even unsafe, as insufficient time is allocated for rigorous testing and validation.
    • This haste can result in critical vulnerabilities, unexpected behaviours, and unintended negative impacts on users or society. – Organisations adopting this approach risk severe reputational damage, significant financial penalties from regulatory bodies, and a profound erosion of public trust in AI technologies.
    • A culture that condones breaking things in the pursuit of speed can stifle the development of robust ethical frameworks and accountability mechanisms, ultimately hindering the long-term, sustainable growth of the AI industry.

    C. Not My Problem / Rules Don’t Apply to Us

    “Not My Problem” attitude manifests as a dismissal of accountability for the downstream harms caused by AI systems. This often takes the form of developers or deployers claiming, “We only built the model, not how it’s used,” thereby abdicating responsibility for the real-world consequences of their creations. Hand-in-hand with this is the “Rules Don’t Apply to Us” mentality, where organisations or individuals dismiss regulations, ethical guidelines, or governance frameworks as mere “red tape” that hinders innovation or doesn’t pertain to their specific work.

    Impact: This dual hazardous attitude poses significant threats to responsible AI development and deployment.

    • It leads to a severe lack of accountability, allowing harmful biases embedded in AI systems to perpetuate and amplify societal inequalities without redress.
    • Organisations operating under this mindset are highly susceptible to legal and ethical liabilities, as they fail to anticipate and mitigate risks associated with their AI products.
    • This attitude undermines the collective effort to establish robust ethical standards and governance structures for the wider AI industry, eroding public trust and potentially inviting more stringent, less flexible regulation in the future.
      By disclaiming responsibility and ignoring established norms, these attitudes hinder the maturation of AI into a trustworthy and beneficial technology.

    D. Bigger = Better (prioritising scale, size, and benchmarks over safety, efficiency, or appropriateness.)

    In the AI landscape, the “Bigger = Better” attitude manifests as an almost singular obsession with scaling AI models and systems, prioritising performance metrics, model size, and performance measurements over practical utility, real-world safety, and appropriateness for the task at hand. This often involves the pursuit of ever-larger datasets, more complex architectures, and higher computational power, sometimes at the expense of efficiency, interpretability, or environmental sustainability.

    Impact: This hazardous attitude can lead to:

    • The development of overly complex and resource-intensive AI solutions that are not only inefficient but also difficult to audit, understand, and control.
    • It can divert attention and resources from simpler, more elegant, and often more effective solutions that might not boast impressive benchmark numbers but are better suited for specific applications.
    • Scale without adequate consideration for safety can lead to catastrophic failures, especially when these large, opaque models are deployed in critical domains.
    • The environmental impact of training and running increasingly massive AI models also becomes a significant concern, contradicting broader sustainability goals.
      Ultimately, this attitude risks creating a technological arms race where the true value and responsible application of AI are overshadowed by a relentless, and often misguided, pursuit of sheer size and processing power.

    E. Shiny Object Syndrome (Deploying AI for novelty or prestige, not real need or value)

    “Shiny Object Syndrome” describes the tendency to adopt AI technologies primarily for their novelty, prestige, or perceived competitive advantage, rather than based on a clear understanding of their real value, alignment with strategic business needs, or suitability for specific problems. This often involves jumping on the latest AI trend – be it a new AI design, a specific application, or a buzzword – without sufficient due diligence or a robust use-case analysis.

    Impact: This hazardous attitude can lead to

    • Significant wasted resources, both financial and human, as organisations invest in AI projects that lack a clear purpose or fail to deliver tangible value.
    • It can result in a series of failed pilot projects, leading to disillusionment with AI capabilities and a perception that the technology is overhyped.
    • Distracting from core business objectives and genuine problem-solving, Shiny Object Syndrome can hinder true innovation and prevent you from identifying and addressing their most pressing challenges effectively.
    • It fosters a culture of superficial adoption rather than deep, strategic integration of AI, ultimately undermining the potential for AI to create meaningful and sustainable impact.

    Mitigating Hazardous Attitudes in AI: A Leadership Role

    Just as aviation has developed robust systems and training to counteract hazardous attitudes, leaders developming AI must proactively implement strategies to foster a culture of responsibility, critical thinking, and ethical development. Mitigating these attitudes is more than compliance; it is about building resilient, trustworthy, and impactful AI initiatives.

    Fostering a Culture of Responsibility: At the core of responsible AI lies a culture that prioritises ethical considerations, accountability, and continuous learning. Leaders must actively champion an environment where open discussion about AI’s potential harms and biases is encouraged, not suppressed. This involves establishing clear lines of responsibility for AI system outcomes, from design to deployment and maintenance. Regular ethical training, workshops, and forums can help embed these values, ensuring that every team member understands their role in responsible AI development.

    Robust Governance and Oversight: Effective governance is crucial. This includes implementing clear policies and frameworks for AI development, deployment, and monitoring. Establishing independent AI ethics committees or review boards can provide an essential layer of oversight, ensuring that projects align with your values and societal expectations. Risk assessments should be integrated into every stage of the AI lifecycle, identifying potential harms and developing mitigation strategies before deployment. Plus, audit mechanisms, including explainability tools and regular performance reviews, are vital for maintaining transparency and accountability.

    Prioritising Safety and Ethics over Speed: While innovation often demands agility, the pursuit of speed must never compromise safety and ethical considerations. Leaders must establish rigorous testing, validation, and deployment protocols that include comprehensive bias detection, fairness assessments, and robustness checks. This may mean longer development cycles, but the long-term benefits of trustworthy AI far outweigh the short-term gains of rapid deployment. Emphasising a ‘safety-first’ mindset, similar to aviation’s approach, ensures that potential risks are thoroughly addressed before AI systems impact real-world scenarios.

    Promoting Critical Thinking and Realistic Expectations: Leaders must cultivate an environment where critical thinking about AI capabilities and limitations is encouraged. This involves moving beyond the hype and fostering a balanced understanding of what AI can and can’t do. Encouraging skepticism, questioning assumptions, and demanding evidence-based decision-making can counteract the ‘Techno-Solutionism’ attitude. Realistic expectations about AI’s development timelines, resource requirements, and potential challenges are essential for successful implementation.

    Investing in Education and Training: Equipping teams with the knowledge and skills for responsible AI development is critical. This goes beyond technical proficiency to include training in AI ethics, societal impact, and regulatory compliance. Cross-functional training can help bridge gaps between technical developers, ethicists, legal experts, and business leaders, fostering a holistic understanding of AI’s implications. Continuous learning programs ensure that teams remain updated on evolving best practices and emerging risks in the rapidly changing AI landscape.

    The aviation industry relies heavily on checklists to make it safe. We’ve put together a readiness checklist containing over 150 items to identify strengths, weaknesses and help you develop and deploy AI with ease.

    Navigating the Future of AI Responsibly

    The parallels between aviation’s hazardous attitudes and the emerging challenges in AI development serve as a powerful reminder: technology, no matter how advanced, is ultimately shaped by human decisions and attitudes. Just as pilots must constantly guard against psychological traps that can compromise safety, leaders using AI must proactively recognise and address the hazardous attitudes that can undermine the responsible and beneficial deployment of artificial intelligence.

    By fostering a culture of responsibility, implementing robust governance, prioritising safety and ethics over speed, promoting critical thinking, and investing in continuous education, you can navigate the future of AI with greater confidence and integrity. The goal is not to stifle innovation but to channel it responsibly, ensuring that AI serves humanity’s best interests. Embracing a safety-first, ethical approach to AI is a regulatory burden, but it is also a strategic imperative that will define the leaders and organisations that truly thrive in the AI era, building trust, driving sustainable value, and shaping a future where AI is a force for good.

    Use the Readiness Checklist to take the next step and make sure you’re not falling into one of these hazards.

  • 10 Key Questions to Ask About AI Governance and Risks

    AI isn’t just changing how your organisation operates, it’s redefining the very nature of business risk and opportunity. As a board member, your role isn’t to understand the algorithms, but to ensure your leadership team has thought through the fundamental questions that will determine whether AI becomes your competitive advantage or your Achilles’ heel.

    The following questions are structured under six essential categories that provide a comprehensive framework for AI governance:

    • Strategic Foundation (What’s our North Star?)
    • Risk Architecture (What could go wrong?)
    • Accountability Structure (Who’s responsible?)
    • Operational Readiness (How do we execute?)
    • Regulatory Compliance (What must we do?)
    • Stakeholder Trust (How do we build and maintain trust?)

    Strategic Foundation

    What is our organisation’s overall strategy for the responsible development, deployment, and use of AI? How is AI governance integrated into this strategy and our core values?

    AI governance involves tools, processes, and values that ensure your AI use remains legally compliant and ethically aligned. Effective AI governance translates ethical principles into operational practice while ensuring strategic alignment and value integration, ensuring AI use advances your organisation’s objectives while upholding its principles.

    Case Study: IBM’s Strategic AI Governance Framework

    IBM developed a company-wide AI Ethics Board in 2018, including leaders from legal, HR, product, research, and diversity teams. Their role is to translate high-level ethical principles into real decision-making about product design, client engagements, and hiring practices. One early success was when the board advised against selling facial recognition software to law enforcement agencies – despite clear commercial opportunities – because it conflicted with IBM’s values around racial equity and responsible AI use.

    Risk Architecture

    If we had to explain our AI decisions to angry customers or regulators tomorrow, would we be confident in our position?

    Transparency, explainability, and contestability are key AI ethics principles. Organisations must design technical and organisational structures to satisfy these expectations. Transparency about the role of AI and human involvement is particularly crucial where outcomes impact human rights.

    Case Study: The Dutch Tax Authority and Algorithmic Discrimination

    The Dutch tax authority used an algorithmic decision-making system to create risk profiles and identify child care benefits fraud. The system’s internal risk indicators falsely accused thousands of families—often those belonging to ethnic minorities or with lower incomes—of fraud. The consequences were devastating: over a thousand children were taken into foster care and many more families already struggling forced into poverty.

    Once the full scale of the scandal was exposed, the entire government resigned and the agency faced millions in fines. This catastrophic failure demonstrates how unexplainable AI systems can lead to institutional collapse when accountability is demanded.

    How confident are you that we’re not missing the risks that could blindside us, and what’s your process for staying ahead of threats we haven’t seen before?

    The rapid spread of AI, especially in high-risk areas, highlights the need to tackle risks and potential harms like bias and discrimination. Generative AI introduces new dimensions of risk, including hallucinations, misuse, lack of traceability, harmful output, and complexities in the value chain. Organisations need robust processes to determine the appropriate risk management based on their tolerance. This includes identifying emergent AI risks and building warning systems, stakeholder checks, and scenario modelling to avoid public failures.

    Case Study: CNET’s Generative AI Journalism Misstep

    In early 2023, technology news outlet CNET quietly began publishing articles written by a generative AI tool. While the goal was efficiency in producing simple explainer content at scale, the result exposed deep flaws in their AI risk management process. The articles contained factual errors, hallucinated information, and even plagiarised passages that went unnoticed until external parties raised concerns.

    While editorial staff had raised concerns about technical capability, ethics, reputation, and compliance, these warnings were not incorporated into the rollout. Compounding the mistake, CNET lacked a transparent disclosure policy, had insufficient human oversight, and did not anticipate how quickly public trust could erode.

    Accountability Structure

    How do we ensure accountability and shared responsibility for AI systems and their outcomes across the organisation, from design to operation and interactions with third parties?

    Accountable algorithm development and operation are key to sustainable AI use. Best practice emphasises shared responsibility among AI model creators, adapters, users, and application users. Defining clear roles and responsibilities for mapping, measuring, and managing AI risks is crucial. This includes specifying ownership for AI systems and algorithms throughout their lifecycle.

    Case Study: Amazon’s AI Hiring Tool and Hidden Bias

    In the mid-2010s, Amazon developed an internal AI hiring tool to automate screening of resumes for technical roles. The AI model, trained on resumes from the previous 10 years, learned to downgrade candidates who had attended women’s colleges or included words like “women’s” in their resume.

    While the issue was eventually discovered and the tool scrapped, the deeper failure lay in the lack of defined ownership and shared responsibility. Engineers built and trained the model, but there was no clear process for cross-functional review, designated roles for monitoring bias, or ethical oversight.

    AI systems often involve multiple stakeholders—from data scientists and product teams to business units, end users, and external partners. Unless accountability is explicitly shared and assigned from design to deployment, risks go unmanaged. Best practice requires a clear map of roles and responsibilities, including for testing, monitoring, and escalation.

    Operational Readiness

    How do we choose which AI solutions to investigate, and how do we know when we’re moving too fast or too slow compared to competitors?

    Organisations must evaluate potential AI use cases across multiple dimensions. For example, business impact, organisational readiness, and investment strategy. This assessment could consider downstream impacts like hallucinations and the need for appropriate guardrails.

    Case Study: Zillow’s AI Pricing Model Collapse

    In 2021, Zillow abruptly shut down its high-profile “iBuying” business—an AI designed to identify undervalued homes, make competitive offers, and resell at a profit. The result was disastrous: a loss of over $500 million, a 25% workforce reduction, and serious reputational damage.

    What went wrong wasn’t the idea but the lack of robust evaluation. Zillow didn’t adequately account for model drift and uncertainty, nor did it put the necessary guardrails and scenario planning in place. The initiative outpaced the company’s operational capabilities and exposed a strategic blind spot: the failure to assess AI maturity against business risk.

    How do we know when an AI system is no longer serving us, and who makes the call to shut it down?

    AI systems need to be governed over their entire life cycles. The NIST AI Risk Management Framework emphasises applying its functions iteratively throughout the AI lifecycle, including safely phasing out AI systems at the end of their useful life.

    Case Study: Twitter’s Image Cropping Algorithm Decommission

    In 2020, Twitter faced backlash when users discovered that its AI-driven image cropping algorithm consistently favoured white faces over black ones in photo previews. Twitter conducted an internal audit and confirmed that, although the algorithm was not explicitly biased by design, its output showed a preference for lighter skin tones.

    Rather than trying to tweak a flawed system, Twitter made the strategic decision to decommission the algorithm entirely and shift to showing full images by default, giving users control over image framing. The decision was not just technical, but ethical and reputational, aligning with user expectations and values.

    What would happen if our data foundation proved inadequate, and how would we know before it becomes a crisis?

    Data is the crucial foundation for AI systems and algorithm development. Ensuring data are sourced, used, and monitored in alignment with organisational values is an essential operational governance component. Data quality analyses must check for representativeness, and datasets must be inclusive, diverse, and representative to avoid bias.

    Case Study: Apple Card’s Gender Bias Investigation

    In 2019, Apple and Goldman Sachs launched the Apple Card, which used an AI algorithm to determine credit limits. Users began reporting that women were being granted significantly lower credit limits than men, even when their financial profiles were equal or better.

    The companies claimed the algorithm did not intentionally use gender as an input, but the training data reflected historical credit practices, which were themselves biased. In effect, the algorithm learned and reproduced past inequities due to unrepresentative and skewed data foundations. The incident triggered an investigation by the New York Department of Financial Services and drew attention from regulators globally.

    Neither Apple nor Goldman had adequate pre-launch data auditing processes to detect this issue. This shows that flawed or unrepresentative data doesn’t need to be malicious to cause harm—it just needs to be undetected. Operational governance must include ongoing validation of data quality and alignment with organisational values, especially when those data drive automated decisions with human impact.

    Regulatory Compliance

    How are we ensuring compliance with current and emerging AI regulations and legal requirements (e.g., the EU AI Act), and how are we tracking changes in the regulatory landscape?

    Binding regulations for AI systems and users are still in development. Organisations must map relevant regulations and understand and manage the fast-moving regulatory landscape. This includes regulatory horizon scanning, jurisdictional mapping, and a compliance monitoring system embedded into operational processes.

    Case Study: Clearview AI and Global Regulatory Backlash

    Clearview AI scraped billions of images from public websites like Facebook, LinkedIn, and Instagram to build a powerful facial recognition database. The company operated under the assumption that publicly available images could be freely used for commercial AI training.

    In 2020, however, regulators in the EU, UK, Canada, and Australia found Clearview in breach of privacy laws. Regulators ruled that biometric data like facial data constitutes personal data and can’t be collected without proper user consent. This case demonstrates how regulatory interpretations can shift rapidly in the AI space, catching unprepared companies off guard.

    Stakeholder Trust

    Who could AI hurt if we get this wrong, and how are we protecting them while still moving boldly?

    AI governance has a multi-stakeholder nature. Organisations need to consider their AI systems’ impacts on various stakeholder groups and engage in algorithmic impact assessment. Structured public feedback methods like focus groups, user studies, and AI red-teaming can help evaluate system performance and identify risks before they manifest.

    Case Study: YouTube’s Recommendation Algorithm and the Radicalisation Spiral

    Over the last decade, YouTube’s AI-powered recommendation engine has been a cornerstone of the platform’s success. But as early as 2017, researchers, journalists, and users began to notice the algorithm often recommended increasingly extreme or polarising content. Several academic studies confirmed that users could be nudged down rabbit holes toward misinformation, radical ideologies, and harmful narratives. This wasn’t intentional, but resulted from the system’s optimisation for watch time.

    The missing element was algorithmic impact assessment. YouTube didn’t systematically test how its algorithm affected different stakeholder groups, nor did it invite public oversight or feedback. There was no formal red-teaming or participatory evaluation mechanism to identify potential harms before they affected millions of users.

    What is our plan for developing the necessary workforce skills, capabilities, and organisational structures to support responsible AI transformation and address the human impact of AI adoption?

    Responsible transformation involves strategic coordination across an organisation’s governance, operations, talent, and communications. Leaders need to plan and implement talent transformation, ensuring staff have access to tools and training. This includes upskilling, reskilling, or hiring employees with appropriate generative AI skills.

    Case Study: DBS Bank AI Talent Transformation at Scale

    DBS Bank recognised early that AI adoption without internal capability building would create bottlenecks and risk misuse. They launched a multi-year organisation-wide AI and data literacy initiative to prepare their workforce. This included reskilling thousands of employees, running AI ethics and explainability training, creating cross-functional ‘data squads’, and establishing a Responsible AI Council.

    By investing in human capability alongside technological capability, DBS created an environment where AI could be deployed responsibly and effectively, with the right checks and balances in place.

    The Bottom Line

    The failures highlighted in this article, from $500 million losses to institutional resignations, were rarely caused by bad code but by a vacuum of leadership and a lack of clear accountability. For a board, the greatest AI risk is not the hallucination of a chatbot, but the blind spots in the boardroom of the potential impacts the hallucination could cause. As you integrate AI into your core operations, these six categories of inquiry must become a permanent fixture of your agenda.