AI Vendor Evaluation Checklist for Small Teams
Before you give an AI vendor access to your data, even indirectly via prompts, run through this checklist. It takes under 30 minutes and catches the issues that cause problems later.
A structured checklist for evaluating AI vendors before sign-up: data handling terms, training opt-out, DPA availability, security certifications, subprocessors, and exit rights. Run it in 30 minutes; store the result.
TL;DR: A structured checklist for evaluating AI vendors before sign-up: data handling terms, training opt-out, DPA availability, security certifications, subprocessors, and exit rights. Run it in 30 minutes; store the result.
Start here (5 minutes)
- Prefer a faster "pass/fail + score" version? Use AI Vendor Due Diligence in 30 Minutes.
- If you have not defined data rules yet, start with AI Policy Starter Kit so vendor questions map to policy.
- For ongoing oversight, adopt the Lightweight AI Governance Operating Rhythm.
1. Data handling
- Where is data processed? Confirm the data region (EU, US, etc.) matches your obligations.
- Is data used to train models? Get a written answer. Many consumer tiers say yes by default.
- Can you opt out of training? If yes, is it account-level or requires a paid tier?
- How long is data retained? Prompts, outputs, and conversation history.
- Do they have a Data Processing Agreement (DPA)? Required for GDPR. Request it before signing.
2. Security
- SOC 2 Type II or ISO 27001? Ask for the report or check their trust page.
- SSO / SAML support? Centralised auth matters as the team grows.
- Audit logs available? Can you see who used the tool and when?
- Subprocessors disclosed? They should list third-party services that touch your data.
3. Compliance & legal
- GDPR / CCPA compliant? Don't assume, verify via their privacy policy or DPA.
- Industry-specific requirements met? HIPAA, PCI, SOC 2 if relevant to your sector.
- Liability clause reasonable? Some AI vendors disclaim all liability for outputs.
- IP ownership clear? Who owns content you generate using their tool?
4. Operational risk
- Pricing model stable? Free tiers disappear. Understand the paid path early.
- API or export available? Can you get your data out if you need to switch?
- SLA / uptime commitment? If the tool is business-critical, you need a commitment.
- Support channel and response time? For enterprise/team plans, not just docs.
5. Exit and lock-in
- Can you export all data on cancellation?
- Cancellation notice period? Month-to-month vs annual lock-in.
- What happens to your data after cancellation? Deletion timeline should be documented.
How to Run the Evaluation in 30 Minutes
The checklist works best as a structured conversation with the vendor's sales or solutions engineer, not as a questionnaire sent by email. Vendors respond faster and more completely when the questions come in a call. Run it in this order:
Minutes 1-10: Data handling (Section 1). These are your highest-stakes questions. Ask for direct answers on training and retention. If the vendor hedges, escalate to written confirmation before you continue.
Minutes 11-18: Security and compliance (Sections 2 and 3). For SOC 2, ask when the last audit was and whether the report covers AI-specific controls. Many SOC 2 reports predate the vendor's AI features and do not cover those systems.
Minutes 19-25: Operational risk and exit (Sections 4 and 5). These matter most if the tool becomes business-critical. Ask about the export format specifically, "you can export your data" and "you can export your data in a usable format within 30 days" are different answers.
Minutes 26-30: Document the results. Capture the vendor's answers immediately after the call. A record of what the vendor said, and when, is useful if terms change later.
Red Flags That Should Stop the Evaluation
Some vendor answers are immediate disqualifiers for most use cases:
"We use your data to improve our models by default." If the tool processes work data and there is no opt-out or the opt-out requires contacting sales and waiting days, stop the evaluation. Find a vendor with explicit no-training commitments.
"We don't offer a DPA, our terms of service cover it." A Terms of Service is a unilateral contract the vendor can change. A DPA is a bilateral agreement. For GDPR-regulated data, a TOS-only position is not adequate. Walk away unless you are using the tool exclusively for public data.
"We can't tell you who our subprocessors are for competitive reasons." This is not a legitimate position for a vendor handling business data. GDPR Article 28 requires processor-level transparency. A vendor that refuses to disclose subprocessors is either not GDPR-aware or is hiding something worth hiding.
"Your data is deleted when you close your account." This is not a retention policy, it is a deletion trigger. Ask how long after account closure data is actually deleted, whether backups retain it, and whether audit logs are included in the deletion. "Deleted on account closure" that actually means "deleted within 90 days from backup rotation" is a meaningful difference.
How to Use This Checklist for Ongoing Vendor Management
The checklist is not one-time. AI vendor terms change frequently, more than any other software category. Build a lightweight annual review into your vendor management process:
Annual re-verification: Re-run Sections 1 and 3 annually for any vendor that touches sensitive data. Specifically check: has the training opt-out status changed? Has the DPA been updated? Are there new subprocessors that were not disclosed at onboarding?
Event-triggered review: Immediately re-evaluate when a vendor announces: a new AI feature that processes your data, a pricing change that moves you to a different tier, a merger or acquisition, or a security incident. All four events can change the compliance posture you evaluated at onboarding.
Tier change alert: Many vendors offer training opt-out only on paid tiers. If a team member uses a free tier account for work, the checklist answers you gathered for the paid tier may not apply. Track which tier each team member is on and flag downgrades.
Document each annual review in the same place you store the original vendor evaluation. Auditors reviewing your AI governance programme will ask whether your vendor assessments are current, a timestamp on the evaluation record answers that question immediately.
Scoring and decision
Run this with your IT or security contact. Any red (data trains models, no DPA, no export) should trigger a conversation before sign-off, not after.
| Finding | Risk level | Action before signing |
|---|---|---|
| Data used for model training by default, no opt-out | Red, deal-breaker for most regulated use cases | Do not proceed; find a vendor with explicit training opt-out or no-training guarantee |
| No DPA available | Red for any GDPR or CCPA-scoped use | Do not send EU or California personal data; require a DPA before use |
| No SOC 2 Type II (or equivalent) | Orange, acceptable for low-risk tools; not for sensitive data | Accept only for productivity tools; require SOC 2 for anything touching customer data |
| Liability fully disclaimed in contract | Orange, acceptable if vendor is low-risk; concerning for high-stakes outputs | Negotiate a reasonable liability cap; document the gap if it cannot be closed |
| No data export on cancellation | Orange, lock-in risk | Confirm export format and process before signing; build in 90-day notice period |
| No subprocessor list | Orange, supply chain opacity | Request list; if vendor refuses, treat as Red for any regulated data |
| Pricing free tier only, no paid path | Yellow, sustainability risk | Confirm paid path exists; free tiers disappear |
A vendor that cannot answer these questions in writing is a risk. Document the answers alongside the contract.
What AI Vendor Evaluation Looks Like in Practice
Here is what the 30-minute evaluation typically looks like for a real team:
The scenario: A 12-person SaaS team wants to add an AI writing assistant for their customer success team. The tool will draft email responses to customer queries. Some queries include account details and subscription information, Confidential data under their classification policy.
What the team actually does: They schedule a 30-minute call with the vendor's sales engineer. They lead with Section 1: the vendor confirms data is processed in the EU, used to improve models by default on the Team plan, opt-out is available on Business plan only (at 3× the cost), retention is 30 days for prompts. The team's GDPR obligation means they request a DPA, the vendor sends a standard one within 24 hours.
The judgment call: Model training opt-out requires upgrading to Business. The team calculates the cost difference, documents the risk of using Team plan without opt-out for Confidential data, and decides to start on Business plan with training disabled.
What gets recorded: Vendor name, plan selected, training opt-out status, DPA location (shared drive), data classification covered (Confidential), and the date of the evaluation. The record takes five minutes to write and sits alongside the signed DPA.
This is what governance looks like for most small teams, not a formal procurement board, but a documented conversation with a clear outcome. The checklist makes the conversation systematic rather than ad hoc, and the record makes the decision auditable.
Red flags that should pause a vendor decision
Some findings during evaluation should not be treated as risks to weigh and accept. They should stop the evaluation until resolved:
- Vendor cannot produce a signed DPA on request, or the DPA does not cover your data types
- Training opt-out requires a custom enterprise contract you cannot negotiate at your tier
- Vendor's AI model documentation does not disclose what data was used to train it (relevant for California AB 2013 compliance if selling in California)
- No sub-processor list, or sub-processors include jurisdictions that conflict with your regulatory requirements
- Security certifications (SOC 2, ISO 27001) are expired or not applicable to the product you are evaluating
If a vendor you want to use has one of these red flags, go to their sales team with a specific ask before proceeding. Many vendors will provide custom DPA terms or opt-out confirmations for business customers even when those options are not advertised on their website.
References
- National Institute of Standards and Technology, AI Risk Management Framework (AI RMF 1.0)
- ENISA, AI Cybersecurity Guidance
- European Parliament and Council, EU AI Act
- OECD, OECD AI Principles
Related Reading
- Meta Posed as Teenagers to Attack Rival AI Chatbots: What It Means for Vendor Trust
- AI Companies Built Empires by Scraping the Internet: The Vendor Risk Behind the Backlash
- AI project intake checklist: 6-phase workflow for approving new AI tools
- AI vendor due diligence checklist
- AI vendor due diligence in 30 minutes
- AI vendor evaluation prompts
- GDPR-compliant AI assistants comparison, 6 major AI assistants scored on DPA, training opt-out, EU data residency, SOC 2
- VC AI governance due diligence checklist, 18 questions investors ask about AI vendor governance
- AI regulatory readiness scorecard, 25-question compliance maturity scoring for software teams
- AI risk decisioning governance checklist, for vendors used in financial or credit automated decisions
- California SB 942 AI transparency compliance, vendor disclosure questions for California-facing tools
- AI governance guide for small teams
- AI regulation deadline calendar 2026, every jurisdiction
- EU AI Act GPAI compliance checklist, August 2 deadline
- GenAI vendor risk assessment framework 2026
