Something happens the moment your organisation settles on a word for what AI is. A design philosophy clicks into place and governance questions answer themselves. Oversight feels obviously necessary, or it feels obviously unnecessary. Most of the time, no one in the room notices this happening.
The words are not a label but a mental frame that imports an entire domain of human experience, complete with its own logic about agency, control, and responsibility. The linguist George Lakoff calls this a conceptual metaphor: we don’t merely describe experience through metaphor, we think through it. The frame structures what feels like common sense, and what feels like common sense rarely gets examined.
When your organisation calls AI a tool, you are not describing a deployment approach alone. You are importing the complete conceptual logic of tool use, and that logic will quietly answer a hundred design and governance questions before anyone thinks to ask them explicitly.
Tool: The Human Is the Only Agent in the Room
A tool has no agency. A hammer doesn’t decide where it lands, a calculator doesn’t choose what to calculate. When a hammer hits the wrong nail, no one convenes a review of the hammer. The logic flows in one direction: the user holds all intention and bears all responsibility.
Under the tool metaphor, this feels like common sense. The design imperative is a high-control interface that keeps the human firmly in command, and oversight infrastructure feels redundant in the same way you would never audit your word processor. If the AI produces wrong output, the frame says it’s a user problem: either the instructions were wrong, or the output wasn’t checked carefully enough.
The metaphor works well when it accurately describes what the system does. Grammar checkers, data analysis dashboards, and coding assistants genuinely behave like sophisticated tools. The user directs and the tool responds.
The metaphor breaks when it’s applied to systems that exercise something resembling judgement. AI that screens job applications, assesses loan risk, or makes triage recommendations is not behaving like a hammer. It generates outputs the user didn’t specify, based on patterns the user didn’t choose. When those outputs are wrong, the tool metaphor offers no mechanism for catching errors and no conceptual language for asking why. The frame has already answered that question: user error.
Peer: The Machine Gets a Voice
The colleague or co-pilot metaphor imports a different logic entirely. Colleagues have agency, offer opinions you didn’t ask for, and can be wrong while remaining entirely confident. You expect them to explain their reasoning, and if they can’t, you trust the recommendation less.
This is a more honest metaphor for how AI behaves in many deployments. Fraud detection systems that flag anomalies for human review, content systems that propose and revise, diagnostic tools that suggest rather than decide: these are genuine collaborations where both parties contribute and neither is simply executing the other’s instructions.
The peer metaphor makes explainability feel natural because of course you want the AI to show its work, of course a human reviews before acting. Shared responsibility follows from how we think about working alongside colleagues: you don’t fully outsource your judgement to someone else, no matter how capable they appear.
What this metaphor hides is an asymmetry of confidence. A human colleague who doesn’t know something usually knows they don’t know, whereas AI can be wrong with complete conviction and no visible hesitation. The peer frame can lead people to extend more trust than the relationship warrants, precisely because the metaphor makes trust feel appropriate.
Authority: The System Decides, Humans Comply
When AI replaces a human process entirely, a different metaphor tends to take hold, often without being named: the system as authority, processing, deciding, and acting while humans monitor the results.
The authority metaphor imports from the domain of institutional rules and procedures, where rules are correct by definition and the system knows best. Challenging the output feels like questioning the process itself, which feels obstructive rather than responsible. When the system flags something, people act on the flag. When it doesn’t, people don’t look further.
This is not inherently dangerous, for example automated invoice processing and robotic process automation handle large volumes of low-stakes work effectively, and the authority metaphor fits those applications well.
The problem arrives when it’s applied to high-stakes decisions and the system’s reliability is treated as given rather than tested. Automated hiring decisions, credit scoring, and content moderation all operate under this logic. The system makes the call, and human judgement enters only at the edges. The conceptual frame makes this feel like efficiency. The practical consequence is that errors encoded into the system become very hard to see, and harder still to challenge, because the authority metaphor has already told everyone that challenging the system is not their role.
The Frame You Don’t Examine Is the One That Governs You
Any of these frames can be appropriate when it accurately describes what the system does: the tool approach for systems the user genuinely directs, the peer model for genuine collaboration, and the authority model for high-volume, low-stakes processes with clear oversight in place.
Failures accumulate when the metaphor doesn’t match the reality, and no one has named the mismatch. Fpr example. an authority-level system governed by tool-level assumptions, a peer-level AI trusted like a colleague long before it has earned that trust, or a replacement system with no escalation path because, conceptually, there is nothing to escalate from.
These are not governance failures in the conventional sense so much as failures of conceptual clarity: the organisation built what it thought it was building, solving the problems it thought it was solving, but hadn’t examined the underlying assumptions.
Naming the Metaphor Is the First Act of Governance
Before asking what guardrails your AI needs, ask what your organisation believes it is at the level of operating assumption, not at the level of documentation.
The questions below are designed to surface the operating metaphor. What they reveal is not always what the answer says: often it is what the answer assumes, or what the question itself appears to disturb.
Who is responsible when the AI gets it wrong?
The tool frame answers quickly: the person who used it. The peer frame identifies whoever reviewed the output before it was actioned. The authority frame produces something else: confusion, a redirect to IT or the vendor, or a long pause followed by “that hasn’t really come up”.
The reaction worth noting is impatience. “Obviously the user” said with certainty is itself diagnostic when the system is making decisions the user never specified.
What happens when someone disagrees with an AI output?
This question separates the existence of a mechanism from the existence of permission. In a tool frame, no formal process is needed: the user simply doesn’t act on the output. With a peer frame, disagreement has a path: escalation, review, documentation, and move on. In an authority frame, the question often produces a category error. Disagreement is routed to a technical team rather than a domain expert, because the operating assumption is that the output is a system output, not a decision subject to challenge.
The tell is when the question appears to confuse process with permission. “Anyone can disagree” is not a mechanism.
How would you know if the AI started getting things wrong?
The tool frame assumes the user would notice immediately. The peer frame has monitoring, defined thresholds, and review cycles. The authority frame tends to produce the longest pause of any question on this list, followed by “we’d get complaints” or “the vendor monitors that”.
“We’d get complaints” means the error detection mechanism is your customers.
Can the AI explain why it produced that output, and does anyone ask it to?
The first half of this question is technical and the second half is cultural. Systems that are capable of explanation but where no one has ever requested one reveal the operating assumption as clearly as any governance document. In a tool frame, explanation feels unnecessary or obvious. With a peer frame, it is routine and integrated into the process. In an authority frame, the capability may exist in a dashboard somewhere that no one opens. If the honest answer is “I’m not sure anyone has”, that is the operating metaphor speaking.
What’s the reason for that answer? Is the standard followup question to answers, following the ‘Five whys’ format.
The point is not which metaphor is correct in the abstract, but whether the one in use was chosen deliberately rather than inherited by default and made explicit to everyone. The frame you pick will make certain things feel like common sense. Make sure those are the things you want to feel natural.
References
[1] Lakoff, G., & Johnson, M. (1980). Metaphors We Live By. University of Chicago Press.