Assessing AI vendors: the questions standard due diligence misses

A standard vendor security questionnaire will tell you whether an AI vendor encrypts data at rest. It will not tell you whether your data trains their model, what happens when they swap the underlying model, or who is accountable when the output is wrong. Those are the questions that matter, and most due diligence processes have not caught up.

Start by classifying the AI relationship

The risk profile differs sharply:

• AI-native product - the AI is the service. Highest scrutiny.
• AI feature in an existing tool - often enabled by default, frequently missed entirely in review.
• AI in the vendor's back office - they use AI to process your data internally, which you may only discover by asking.
• Model provider - you consume a model API directly.

The second category is where most organisations have unassessed exposure: a tool reviewed two years ago that has since added AI processing.

The AI-specific questions

Data handling

• Is our data used to train or fine-tune models - yours or a third party's? Get it in writing, not from marketing pages.
• What is retained, for how long, and can we require zero retention?
• Where does inference occur geographically?
• Is our data isolated from other customers' at every stage, including any human review process?
• Is there human review of prompts or outputs? This surprises people - some services include human quality review, which means staff read your data.

The model supply chain

• Which underlying models are used, and are they first-party or licensed?
• Who are the model providers - these are fourth parties in your chain and belong in your subprocessor register.
• What notice do you give before changing the underlying model? A model swap can change behaviour materially without any change to the product.
• Can we pin a model version, or opt out of automatic upgrades?

Output quality and accountability

• How is accuracy measured, and what happens when output is wrong?
• Is output traceable to source data, or generated from model knowledge?
• What contractual liability attaches to erroneous output? Many AI terms disclaim it entirely - know that before it matters.
• Are there guardrails against harmful or confidential-data-revealing outputs?

Governance

• Do they hold or pursue ISO 42001, and what is its scope?
• Have they assessed their systems against applicable AI regulation such as the EU AI Act?
• Is there an AI incident process distinct from the security incident process?
• Can they evidence bias or robustness testing where their AI makes decisions about people?

Consider whether the vendor's AI makes or materially influences decisions about individuals - hiring, credit, access, pricing. If so, your obligations rise sharply and the assessment should be proportionate, regardless of how small the vendor is.

Contract terms worth insisting on

• Explicit prohibition on training with your data, unless deliberately agreed.
• Data location commitments covering inference, logs and backups.
• Notification before material model or provider changes.
• Subprocessor disclosure that names model providers.
• Deletion on termination, including anything derived from your data.
• Audit or evidence rights proportionate to the risk.

Key takeaways

• AI features added to existing tools are the most common unassessed exposure.
• Ask about training use, retention, human review and model provenance explicitly.
• Model providers are fourth parties - record them as subprocessors.
• Check liability for erroneous output before you rely on it.