AI isn’t just changing how your organisation operates, it’s redefining the very nature of business risk and opportunity. As a board member, your role isn’t to understand the algorithms, but to ensure your leadership team has thought through the fundamental questions that will determine whether AI becomes your competitive advantage or your Achilles’ heel.
The following questions are structured under six essential categories that provide a comprehensive framework for AI governance:
- Strategic Foundation (What’s our North Star?)
- Risk Architecture (What could go wrong?)
- Accountability Structure (Who’s responsible?)
- Operational Readiness (How do we execute?)
- Regulatory Compliance (What must we do?)
- Stakeholder Trust (How do we build and maintain trust?)
Strategic Foundation
What is our organisation’s overall strategy for the responsible development, deployment, and use of AI? How is AI governance integrated into this strategy and our core values?
AI governance involves tools, processes, and values that ensure your AI use remains legally compliant and ethically aligned. Effective AI governance translates ethical principles into operational practice while ensuring strategic alignment and value integration, ensuring AI use advances your organisation’s objectives while upholding its principles.
Case Study: IBM’s Strategic AI Governance Framework
IBM developed a company-wide AI Ethics Board in 2018, including leaders from legal, HR, product, research, and diversity teams. Their role is to translate high-level ethical principles into real decision-making about product design, client engagements, and hiring practices. One early success was when the board advised against selling facial recognition software to law enforcement agencies – despite clear commercial opportunities – because it conflicted with IBM’s values around racial equity and responsible AI use.
Risk Architecture
If we had to explain our AI decisions to angry customers or regulators tomorrow, would we be confident in our position?
Transparency, explainability, and contestability are key AI ethics principles. Organisations must design technical and organisational structures to satisfy these expectations. Transparency about the role of AI and human involvement is particularly crucial where outcomes impact human rights.
Case Study: The Dutch Tax Authority and Algorithmic Discrimination
The Dutch tax authority used an algorithmic decision-making system to create risk profiles and identify child care benefits fraud. The system’s internal risk indicators falsely accused thousands of families—often those belonging to ethnic minorities or with lower incomes—of fraud. The consequences were devastating: over a thousand children were taken into foster care and many more families already struggling forced into poverty.
Once the full scale of the scandal was exposed, the entire government resigned and the agency faced millions in fines. This catastrophic failure demonstrates how unexplainable AI systems can lead to institutional collapse when accountability is demanded.
How confident are you that we’re not missing the risks that could blindside us, and what’s your process for staying ahead of threats we haven’t seen before?
The rapid spread of AI, especially in high-risk areas, highlights the need to tackle risks and potential harms like bias and discrimination. Generative AI introduces new dimensions of risk, including hallucinations, misuse, lack of traceability, harmful output, and complexities in the value chain. Organisations need robust processes to determine the appropriate risk management based on their tolerance. This includes identifying emergent AI risks and building warning systems, stakeholder checks, and scenario modelling to avoid public failures.
Case Study: CNET’s Generative AI Journalism Misstep
In early 2023, technology news outlet CNET quietly began publishing articles written by a generative AI tool. While the goal was efficiency in producing simple explainer content at scale, the result exposed deep flaws in their AI risk management process. The articles contained factual errors, hallucinated information, and even plagiarised passages that went unnoticed until external parties raised concerns.
While editorial staff had raised concerns about technical capability, ethics, reputation, and compliance, these warnings were not incorporated into the rollout. Compounding the mistake, CNET lacked a transparent disclosure policy, had insufficient human oversight, and did not anticipate how quickly public trust could erode.
Accountability Structure
How do we ensure accountability and shared responsibility for AI systems and their outcomes across the organisation, from design to operation and interactions with third parties?
Accountable algorithm development and operation are key to sustainable AI use. Best practice emphasises shared responsibility among AI model creators, adapters, users, and application users. Defining clear roles and responsibilities for mapping, measuring, and managing AI risks is crucial. This includes specifying ownership for AI systems and algorithms throughout their lifecycle.
Case Study: Amazon’s AI Hiring Tool and Hidden Bias
In the mid-2010s, Amazon developed an internal AI hiring tool to automate screening of resumes for technical roles. The AI model, trained on resumes from the previous 10 years, learned to downgrade candidates who had attended women’s colleges or included words like “women’s” in their resume.
While the issue was eventually discovered and the tool scrapped, the deeper failure lay in the lack of defined ownership and shared responsibility. Engineers built and trained the model, but there was no clear process for cross-functional review, designated roles for monitoring bias, or ethical oversight.
AI systems often involve multiple stakeholders—from data scientists and product teams to business units, end users, and external partners. Unless accountability is explicitly shared and assigned from design to deployment, risks go unmanaged. Best practice requires a clear map of roles and responsibilities, including for testing, monitoring, and escalation.
Operational Readiness
How do we choose which AI solutions to investigate, and how do we know when we’re moving too fast or too slow compared to competitors?
Organisations must evaluate potential AI use cases across multiple dimensions. For example, business impact, organisational readiness, and investment strategy. This assessment could consider downstream impacts like hallucinations and the need for appropriate guardrails.
Case Study: Zillow’s AI Pricing Model Collapse
In 2021, Zillow abruptly shut down its high-profile “iBuying” business—an AI designed to identify undervalued homes, make competitive offers, and resell at a profit. The result was disastrous: a loss of over $500 million, a 25% workforce reduction, and serious reputational damage.
What went wrong wasn’t the idea but the lack of robust evaluation. Zillow didn’t adequately account for model drift and uncertainty, nor did it put the necessary guardrails and scenario planning in place. The initiative outpaced the company’s operational capabilities and exposed a strategic blind spot: the failure to assess AI maturity against business risk.
How do we know when an AI system is no longer serving us, and who makes the call to shut it down?
AI systems need to be governed over their entire life cycles. The NIST AI Risk Management Framework emphasises applying its functions iteratively throughout the AI lifecycle, including safely phasing out AI systems at the end of their useful life.
Case Study: Twitter’s Image Cropping Algorithm Decommission
In 2020, Twitter faced backlash when users discovered that its AI-driven image cropping algorithm consistently favoured white faces over black ones in photo previews. Twitter conducted an internal audit and confirmed that, although the algorithm was not explicitly biased by design, its output showed a preference for lighter skin tones.
Rather than trying to tweak a flawed system, Twitter made the strategic decision to decommission the algorithm entirely and shift to showing full images by default, giving users control over image framing. The decision was not just technical, but ethical and reputational, aligning with user expectations and values.
What would happen if our data foundation proved inadequate, and how would we know before it becomes a crisis?
Data is the crucial foundation for AI systems and algorithm development. Ensuring data are sourced, used, and monitored in alignment with organisational values is an essential operational governance component. Data quality analyses must check for representativeness, and datasets must be inclusive, diverse, and representative to avoid bias.
Case Study: Apple Card’s Gender Bias Investigation
In 2019, Apple and Goldman Sachs launched the Apple Card, which used an AI algorithm to determine credit limits. Users began reporting that women were being granted significantly lower credit limits than men, even when their financial profiles were equal or better.
The companies claimed the algorithm did not intentionally use gender as an input, but the training data reflected historical credit practices, which were themselves biased. In effect, the algorithm learned and reproduced past inequities due to unrepresentative and skewed data foundations. The incident triggered an investigation by the New York Department of Financial Services and drew attention from regulators globally.
Neither Apple nor Goldman had adequate pre-launch data auditing processes to detect this issue. This shows that flawed or unrepresentative data doesn’t need to be malicious to cause harm—it just needs to be undetected. Operational governance must include ongoing validation of data quality and alignment with organisational values, especially when those data drive automated decisions with human impact.
Regulatory Compliance
How are we ensuring compliance with current and emerging AI regulations and legal requirements (e.g., the EU AI Act), and how are we tracking changes in the regulatory landscape?
Binding regulations for AI systems and users are still in development. Organisations must map relevant regulations and understand and manage the fast-moving regulatory landscape. This includes regulatory horizon scanning, jurisdictional mapping, and a compliance monitoring system embedded into operational processes.
Case Study: Clearview AI and Global Regulatory Backlash
Clearview AI scraped billions of images from public websites like Facebook, LinkedIn, and Instagram to build a powerful facial recognition database. The company operated under the assumption that publicly available images could be freely used for commercial AI training.
In 2020, however, regulators in the EU, UK, Canada, and Australia found Clearview in breach of privacy laws. Regulators ruled that biometric data like facial data constitutes personal data and can’t be collected without proper user consent. This case demonstrates how regulatory interpretations can shift rapidly in the AI space, catching unprepared companies off guard.
Stakeholder Trust
Who could AI hurt if we get this wrong, and how are we protecting them while still moving boldly?
AI governance has a multi-stakeholder nature. Organisations need to consider their AI systems’ impacts on various stakeholder groups and engage in algorithmic impact assessment. Structured public feedback methods like focus groups, user studies, and AI red-teaming can help evaluate system performance and identify risks before they manifest.
Case Study: YouTube’s Recommendation Algorithm and the Radicalisation Spiral
Over the last decade, YouTube’s AI-powered recommendation engine has been a cornerstone of the platform’s success. But as early as 2017, researchers, journalists, and users began to notice the algorithm often recommended increasingly extreme or polarising content. Several academic studies confirmed that users could be nudged down rabbit holes toward misinformation, radical ideologies, and harmful narratives. This wasn’t intentional, but resulted from the system’s optimisation for watch time.
The missing element was algorithmic impact assessment. YouTube didn’t systematically test how its algorithm affected different stakeholder groups, nor did it invite public oversight or feedback. There was no formal red-teaming or participatory evaluation mechanism to identify potential harms before they affected millions of users.
What is our plan for developing the necessary workforce skills, capabilities, and organisational structures to support responsible AI transformation and address the human impact of AI adoption?
Responsible transformation involves strategic coordination across an organisation’s governance, operations, talent, and communications. Leaders need to plan and implement talent transformation, ensuring staff have access to tools and training. This includes upskilling, reskilling, or hiring employees with appropriate generative AI skills.
Case Study: DBS Bank AI Talent Transformation at Scale
DBS Bank recognised early that AI adoption without internal capability building would create bottlenecks and risk misuse. They launched a multi-year organisation-wide AI and data literacy initiative to prepare their workforce. This included reskilling thousands of employees, running AI ethics and explainability training, creating cross-functional ‘data squads’, and establishing a Responsible AI Council.
By investing in human capability alongside technological capability, DBS created an environment where AI could be deployed responsibly and effectively, with the right checks and balances in place.
The Bottom Line
The failures highlighted in this article, from $500 million losses to institutional resignations, were rarely caused by bad code but by a vacuum of leadership and a lack of clear accountability. For a board, the greatest AI risk is not the hallucination of a chatbot, but the blind spots in the boardroom of the potential impacts the hallucination could cause. As you integrate AI into your core operations, these six categories of inquiry must become a permanent fixture of your agenda.