It’s October 2026 and somewhere in your inbox there’s a proposal for a ‘tool or platform’ with ‘blah blah AI’ in the name. It might be a quoting tool, a CRM add-on, a marketing dashboard, or even an agent that answers the phone after 6pm, etc. You’ve probably asked the obvious question, ‘is it any good’, and they’ve said something like ‘it’s the future, it’s a proprietary AI solution, 100% success rate, yada yada yada’. That’s the problem I want to write about here, and how to find the right tools that work for YOUR business, not just the ones that made it through the spam filter!
Last month the Department for Science, Innovation and Technology put its AI Risk Management Toolkit on gov.uk. It’s guidance rather than law, so nobody outside Whitehall is bound by it, and it’s the government writing down the questions ‘it thinks’ it should ask itself before it conceives, designs, buys or runs anything with AI inside it. A business should ask those questions of itself first, and then put them to each and every vendor at the table.
The translator is usually the vendor
The difficulty most businesses have right now is that the person translating this product or tool or software is also the person selling it to you. If you’re putting OpenAI into the business, the consultant you’ve been given is duty bound to OpenAI, and is probably an engineer from their frontline enterprise team. Same with Google and Gemini, and with Anthropic and a Claude enterprise consultant. None of this makes any of what they say a lie, it’s just the wrong person to ask for impartial advice on what YOUR business needs. I’ve watched this pattern for years with Adobe: engineers and sales teams overselling their product line beyond what’s needed, right, or accurate, or effective for the business in front of them.
Which is why most of my initial engagements start with where your data lives, how it moves around the business, and how consolidating and categorising it gets you a more streamlined stack later, rather than with any product, AI tool or service. I have no affiliation with any LLM vendor, and no discounts or rates for implementing one system over another. Ask that of anyone translating for you before you ask anything of the product: who’s this person paid by?
A document written for someone else
The reader who says “that’s a Whitehall document, it’s got nothing to do with me” is half right. The toolkit’s first line says it supports anyone involved in designing, operating, procuring or delivering products that use AI, which sounds like you. The guidance underneath was created, in its own words, to assist departments. It says “your department should”. It asks departments to share their risk logs with DSIT’s central team. And it assumes a risk team led by an AI governance officer, with data, security and legal people round the table.
You’ve got thirty people and a sole marketing manager. So really you’re borrowing a document that wasn’t written for you, but what’s worth borrowing is that many of its questions are already pointed at suppliers: what third-party data, software or infrastructure is in the solution and where the data is stored, what the contract or licence terms are and how they’re being complied with, and whether the dispute route with the supplier is, in the document’s own phrase, “a way to pass risk onto the third party”. But instead of trawling through the whole thing, I’ve consolidated all of this into what I would ask a potential supplier.
The nine core questions, for a thirty-person company
The document’s nine categories, in its own names, are financial; legal and regulatory compliance; appropriate transparency and explainability; fairness; accountability and governance; contestability and redress; technical robustness; security; and environment and people. Each carries a page of questions, and the list isn’t exhaustive. Asked across a table in a business with no data team, they come out like this, and I’ve said where I’d fold them.
Financial. What does this cost once the pilot’s over? The document asks about the cost to scale after the pilot and to fix and maintain the thing. The per-seat price is the small number. The real one is the compute cost, the ‘hours a month’ someone spends checking its output, plus what a wrong output costs you the first time a customer sees it. ASOS found that out last week, when a push notification it didn’t write reached its customers’ phones through a third-party platform it uses to talk to them.
Legal and regulatory. Whose data goes in, whose data is already in there, where does the data live and pass to, and what does the contract say about all of it? Does the vendor (or related suppliers) ‘train’ on what you feed it, where does it sit, can you get it out, and what happens to ‘it’ if you leave. If the answer to any of that is “it’s in the terms”, I’d ask them to show you the line in the clause - and discuss the frequency of the updates to the terms (and the exit clause associated with any changes). Embody your inner ‘procurement team’ here.
Transparency and explainability. Just two questions here. Can anyone in my business explain why it produced that? And do my customers ‘need to know’ they’re dealing with a machine? The second one is about to turn inside out, and I’ll come back to it shortly.
Fairness. Does it treat some people worse (bias control), and would I know? If it’s scoring leads, screening CVs, setting prices or deciding who gets a call back, it’s making decisions about real people, and the protected-characteristics questions apply to you as much as to a department (the Public Sector Equality Duty won’t reach you, the Equality Act will). I’d fold environment and people, the document’s safety and well-being category, into this one, because the person it could hurt is usually the person fairness is protecting. Care, health, finance or vulnerable customers: unfold it. The answer you’re looking for is ‘bias and weighting’ testing…and the regularity of this alignment - with verifiable and accessible results.
Accountability and governance. Who owns this tool or platform? The document asks whether that person has the authority, the resource and the budget. In a small firm it’s whether they have the time. I’d fold this into the first conversation, because if nobody owns the process and the output, the other eight questions are pointless.
Contestability and redress. This is the one I’d bet nobody in a thirty-person company has asked. When the results are wrong about a customer, how do we argue with it, and who do we argue with? And underneath that, for the supplier version: when the vendor’s model gets it wrong, does your contract let you pass the liability and fine-tuning to the vendor, or does it stay with you? And don’t make this a ‘you’ problem, i.e. you shouldn’t need to code round it, or reflect on it in a different way.
Technical robustness and security. I’d take these two together. What happens when it breaks, what happens when the model behind it changes (which will happen ALL THE TIME), and what happens when someone attacks it? The document asks about contingency plans for failures in any third-party components, which for you means: your tool is a wrapper around someone else’s model, and when that model is swapped, deprecated or repriced, what does your vendor do, and what do you do?
Watermarks
There’s a lot of noise about the transparency question right now. There’s negativity about watermarks in Claude’s output and about the metadata hiding inside AI-generated images, and the reading today is that the mark is there to catch you and your business out: like a shortcut, a cheat, or evidence you didn’t do the work. I disagree with that summation. In a few years the watermark will be irrelevant. It will become table stakes in everything. The expectation will become that you HAVE used AI to check, refine, render and replicate the information in whatever forms the job needed, and if you didn’t, that’s what makes the reader question the validity of what’s in it. It’s the new citation and source data points (a domain authority ranking if you will).
So, the risk you’re managing today is being seen to have used it, and the risk in two years is being seen not to have bothered. Worth knowing before you sign anything that promises to keep the AI integration hidden. Authenticity and human-only will 100% be a category in commercial enterprise. But is it a sub-category in 2 years?
Ask yourself these questions first, too…
The nine are a rubric, and a rubric works on whoever you point it at really.
I was thinking earlier today about a board-level “what if machine”: something that sits alongside a consultant and helps draw information out of the supply line, the manufacturing line and the operations team, so you can look at a business you’re refining or buying on the basis of context. The problem it’s for: if I walk into a business whose pipeline I don’t know, I’ve got two routes. I can sit in the business long enough to be able to contradict the head of operations when he says “it works this way”. Or I can go looking for the hidden context in why something did or didn’t work at the time.
If we don’t know the context of the operational attempt, how can we validate the reasoning, or worse, suggest a solution that’s robust enough to work with agentic support? For example: two of their staff were off that day, and the output dropped because at the same time they tried to do something else. There are any number of those triggering factors that were not actively attributed to the fail of the test at the time. So my “what if machine” asks these questions as live iterative reinforcements, like a rolling rubric, to find out whether a limit is a ‘subjective risk’ or an actual physical operational control limitation - then cross-checking this against time and technological factors.
These nine questions are doing this job for AI, if you point them at yourself before the vendor turns up. Some of the restrictions in a business’s AI policies are genuinely subjective: someone said “we can’t put customer data anywhere near it” eighteen months ago and it stuck. Some said that customers won’t trust AI avatars, someone suggested the word hallucination!
These are the same questions you’d ask about a new employee really. If they are not trained in the way YOU work - how are they expected to be effective for YOUR business? And the days of 6 fingers and 20% hallucination risks are long gone. Some of these are still real: the contract with your biggest customer has a clause about subprocessors, and no vendor deck or AI risk analysis is going to move it, but knowing where those lines and limitations are, and when THEY move is worth understanding.
What it looks like in writing
On paper, the nine come out as a short list for whoever’s selling to you: an explainability statement (what it does with our data, and why it produced that); a named owner on their side; a contestability route (how a customer disputes a decision, and who picks it up); third-party provenance (which models, whose data, under what licence, and how outputs are marked); a robustness and security summary that says what happens when the model behind the tool changes; and a monitoring cadence in their own words. And if nobody in the building can ask those questions, borrow someone who can, as long as whoever’s selling it isn’t paying them.
