GenAI Vendor Risk Assessment Framework: FS AI RMF and SOC 2

TL;DR: GenAI Vendor Risk Assessment: A Framework for 2026 (Treasury FS AI RMF + SOC 2), a practical compliance guide for enterprise and HR teams in 2026.

Most procurement teams still treat AI vendor risk the same way they treat SaaS vendor risk: send a 40-question security questionnaire, collect a SOC 2 certificate, review the DPA, and file everything in a shared drive. That approach satisfied auditors in 2022. It does not satisfy them now.

Treasury released the Financial Services AI Risk Management Framework (FS AI RMF) on February 19, 2026. Its third-party control objectives ask institutions to build AI-specific checks into vendor due diligence and to run periodic vendor assessments that examine "AI system performance, data quality, algorithmic bias, security controls," and alignment with their AI principles. A security questionnaire and a SOC 2 report do not cover most of that list.

This creates a practical problem for any team evaluating ChatGPT Enterprise, Claude, Azure OpenAI, Gemini for Workspace, or any other GenAI tool. The vendors have policies. They have SOC 2 reports. They have privacy documentation that runs to dozens of pages. None of that documentation tells you what you actually need to know before you sign a contract and start sending company data through the system.

This guide gives you a six-dimension assessment model mapped to the FS AI RMF third-party control objectives, a comparison table across major vendors, and a 30-minute shortcut for teams that cannot run a full assessment on every tool.

The questionnaire trap

Security questionnaires were designed for a different problem. When your vendor is a payroll processor or a CRM, the key questions are about access controls, encryption, backup procedures, and incident response. Those are meaningful controls for systems that store and transmit data. They are insufficient for systems that generate content, make predictions, or support decisions.

A GenAI system introduces risks that a traditional security questionnaire does not address. The model may produce confident, incorrect outputs at a rate that is acceptable for low-stakes use cases but dangerous for regulated ones. The model may have systematic biases in outputs affecting particular demographic groups in ways that create liability under employment discrimination law or fair lending requirements. The model may be vulnerable to prompt injection attacks that allow a malicious input to override system instructions. A model update released without notice may change the system's behavior in ways that violate your AI acceptable use policy.

None of these failure modes appear in a questionnaire that asks "Do you encrypt data at rest?" or "What is your vulnerability disclosure policy?" The controls are orthogonal to the risk.

The FS AI RMF points at this gap. Its control objectives on third parties go beyond documentation: GV-6.1.3 emphasises "participating in joint testing with third-party entities," and MG-3.1.2 asks for periodic assessments of the vendor system's performance, data quality, algorithmic bias and security controls, with findings documented.

What the Treasury FS AI RMF asks of AI vendor oversight

The FS AI RMF was written for financial institutions and built by the Cyber Risk Institute with more than 100 of them. It follows the NIST AI RMF functions (Govern, Map, Measure, Manage) and sets out 230 control objectives, which an institution filters by its AI adoption stage. It is guidance, not a regulation, and it is not aimed at vendors directly. Vendors meet it through their financial services customers' due diligence, so expect its themes in questionnaires from banks and insurers.

The third-party objectives most relevant to a GenAI vendor review are these.

Due diligence that covers AI. GV-6.1.1 and GV-6.1.4 ask institutions to build AI-specific criteria into procurement, due diligence and contracting, including data provenance, secondary data use and vendor data practices.

Testing with the vendor. GV-6.1.3 emphasises joint testing with third parties to evaluate AI technologies and validate their effectiveness. The framework does not prescribe who must run the tests; testing on your own use case and data, not only the vendor's published benchmarks, is our recommendation.

Periodic assessments, including bias. MG-3.1.2 asks for periodic vendor assessments that examine performance, data quality, algorithmic bias and security controls, and MS-2.11.1 covers fairness and bias evaluation of AI systems generally.

Fourth parties, incidents and exit. GV-6.1.6 asks vendors to disclose their own third-party relationships and sets termination guidelines where risks are significant, GV-6.1.10 adapts incident protocols to third-party AI, and GV-6.2.1 asks for contingency plans for high-risk third-party AI systems.

The framework defines hallucinations, prompt injection and data poisoning in its glossary but sets no hallucination rate and no security test list. Setting an acceptable error rate per use case, and testing for prompt injection and data poisoning, is our recommendation, built on those definitions.

The framework also sets no reassessment tiers. The cadence below is our suggestion.

The 6-dimension assessment model

Running a sound vendor assessment does not require a dedicated AI risk team. It requires asking the right questions in the right order and knowing which answers are acceptable. The following six dimensions map to the FS AI RMF third-party control objectives and cover the risks that standard security questionnaires miss.

Dimension 1: Model transparency

The first question is whether the vendor can tell you what the system is. Not a marketing description. A technical account of what the model does, what data it was trained on, and what its known limitations are.

Ask whether the vendor publishes a model card. A model card is a standardized document that describes a model's intended use cases, performance on evaluation benchmarks, known limitations, and demographic analysis of training data. Google, Anthropic, and Meta publish model cards. Some vendors do not.

Ask what the vendor discloses about training data. You do not need to know every dataset (that is legitimately proprietary), but you should know the general categories: web crawl data, licensed datasets, synthetic data, domain-specific corpora. If the vendor responds to this question with "proprietary architecture" or "we cannot disclose that," treat it as a red flag. The absence of training data disclosure makes bias auditing impossible and makes it impossible to assess whether your data category is similar to what the model has seen before.

Ask whether known limitations are documented. Every model has them. A vendor that cannot tell you what its system gets wrong has either not tested it or is not willing to share the results.

Dimension 2: Data handling and privacy

For most companies, data handling is the highest-stakes dimension because it determines whether using the tool creates GDPR, CCPA, or HIPAA liability.

The DPA must specify the legal basis for any processing under GDPR Article 6. If the DPA does not name a legal basis, if it just says "we process data to provide the service," it is insufficient. Ask specifically whether the vendor relies on contractual necessity (Article 6(1)(b)), legitimate interest (Article 6(1)(f)), or another basis, and whether that basis is documented in the DPA itself.

Training data use is the question that most teams ask too late. Find out whether your data is used for model training by default and what the opt-out process is. Several major vendors default to using customer data for training improvement unless the customer takes explicit action, such as signing an enterprise agreement or toggling an API setting. This is not always disclosed prominently. The privacy-first AI API no training GDPR CCPA guide covers which major APIs offer training-off by default versus by request.

EU data residency is a growing compliance requirement, particularly for companies subject to Schrems II or national data localization requirements. Check whether EU data residency is the default configuration for your account or an additional paid tier. For several major vendors, EU residency requires an enterprise contract at a significant price premium over the standard API.

Subprocessor transparency matters because data does not stay within the vendor's own infrastructure. Ask for the current subprocessor list and, more importantly, ask how and how quickly the vendor notifies customers of subprocessor changes. The GDPR requires vendors to give customers sufficient notice of new subprocessors to allow them to object before the change takes effect. "Sufficient notice" is not defined by the regulation, but 30 days is a reasonable minimum and 5 days is a compliance-grade requirement in most enterprise DPAs.

Dimension 3: Security posture

SOC 2 Type II certification tells you that an independent auditor examined the vendor's controls over a defined period and found them to be operating effectively. It does not tell you that the controls cover AI-specific risks.

The Trust Services Criteria most relevant to AI are CC6 (Logical and Physical Access Controls), CC7 (System Operations: monitoring and incident response), and CC9 (Risk Mitigation: vendor and third-party management). Ask the vendor specifically whether its SOC 2 audit scope includes AI system controls such as model change management, output monitoring, and adversarial testing procedures. Many vendors' SOC 2 reports cover standard SaaS infrastructure but do not extend to the AI pipeline.

Prompt injection testing is a specific security evaluation that most enterprise software vendors do not conduct because traditional software does not have this attack surface. A prompt injection attack uses a malicious input to override a system's instructions. For example, embedding instructions in a document that a user asks the AI to summarize can cause the AI to ignore its system prompt and follow the injected instructions instead. Ask whether the vendor conducts red-team adversarial testing specifically for prompt injection and whether the results are available for review.

API key management and incident response SLAs round out the security picture. What is the vendor's detection-to-notification timeline for security incidents? What counts as an incident for AI-specific failures such as hallucination at scale, bias detection in production, or a confirmed prompt injection breach? These definitions matter because a vendor whose incident response plan does not include AI-specific failure modes will not notify you when those failures occur.

Dimension 4: Hallucination and accuracy measurement

Hallucination is the technical term for a model producing confident, incorrect output. It is the most common failure mode for production GenAI deployments and the one most likely to create liability in regulated use cases.

Ask what benchmarks the vendor uses to evaluate accuracy. General-purpose benchmarks like MMLU (Massive Multitask Language Understanding) and HellaSwag measure broad capability. TruthfulQA specifically measures how often a model gives truthful answers to questions where common misconceptions are widespread. These are useful starting points but insufficient for domain-specific assessment.

For regulated use cases such as producing legal analysis, generating financial summaries, or supporting medical documentation, you need to know the vendor's hallucination rate on domain-specific test sets that resemble your actual use case. This data is rarely published in marketing materials. Ask for it directly in the vendor questionnaire. If the vendor cannot provide domain-specific accuracy data, ask what benchmarks they do use and what their published scores are.

Determine the acceptable hallucination rate for your use case before you send the question. A tool used for internal brainstorming can tolerate a higher error rate than a tool used to generate client-facing compliance summaries. Document your threshold. If the vendor cannot demonstrate that its system meets that threshold for your domain, the appropriate response is either not to deploy the tool or to implement human review gates that reduce the impact of errors. The AI vendor evaluation checklist includes a section for documenting these thresholds.

A vendor who cannot answer this question, who has not measured hallucination rates and cannot tell you what floor to expect, cannot manage the risk they are creating for your organization. Treat the absence of any measurement data as a disqualifying gap for Tier 1 use cases.

Dimension 5: Bias and fairness

Bias in GenAI systems is not a hypothetical concern. Multiple peer-reviewed studies published between 2023 and 2026 have documented systematic differences in outputs based on names, demographic descriptions, and linguistic patterns associated with particular groups. These differences can create liability under employment discrimination law, fair lending requirements, and the EU AI Act's prohibited practices provisions.

Ask whether the vendor has conducted disparate impact analysis on model outputs, not on training data, but on actual outputs across demographic groups. Training data analysis is a precursor; output analysis is the evidence. If the vendor's bias documentation covers only training data composition, the audit has not addressed the question that matters for liability.

For use cases involving hiring decisions, lending, healthcare, or insurance, a third-party bias audit is not just best practice. It is increasingly a legal requirement. New York City Local Law 144 requires employers using automated employment decision tools to obtain annual independent bias audits and publish summaries of results. For vendors positioning their tools for these use cases, ask for the third-party audit results directly.

Ask what demographic groups were included in the bias evaluation. A bias audit that only covers race and gender and excludes age, disability status, and national origin provides incomplete coverage for employment discrimination purposes.

Dimension 6: Contractual remedies

Contracts for GenAI tools often have liability caps that bear no relationship to the harm potential of the system. A tool used to generate customer communications, support underwriting decisions, or produce regulatory filings can generate errors that cause harm well in excess of a standard SaaS liability cap of 12 months' fees.

Before the contract is signed, establish what remedies are available when the system fails. What is the vendor's SLA for model degradation, meaning a measurable decline in output quality between model versions? Several major vendors have changed model behavior significantly between versions without formally announcing a model change.

What happens if a model update changes the system's behavior in ways that violate your AI acceptable use policy or produce outputs that are inconsistent with your compliance requirements? Ask specifically whether the vendor offers a rollback mechanism (the ability to revert to a prior model version) or whether updates are applied universally without customer control. For Tier 1 use cases, the absence of rollback capability is a contractual gap that should be addressed before deployment.

Review the liability cap relative to harm potential. Seek a carve-out from the standard liability cap for AI output failures that result in regulatory penalties, third-party claims, or discrimination liability. Most vendors will resist this, but documenting the request and the vendor's response is itself valuable for your internal risk record.

The agentic AI vendor contract clauses 2026 and AI vendor contract redline template 2026 guides cover specific contract language for these provisions.

Vendor comparison: major GenAI platforms

The following table reflects publicly available information as of June 2026. "Partial" indicates that a feature exists but with meaningful limitations, such as EU residency available only under enterprise contracts, or training opt-out requiring explicit configuration rather than being off by default.

A few observations worth noting. Azure OpenAI scores well on data residency because Azure's existing regional infrastructure allows customers to select EU regions for all processing, including model inference. Mistral is EU-headquartered and hosts data in Europe by default, which is a meaningful differentiation for companies with EU data residency requirements, but its SOC 2 program is newer than the other vendors on this list. Anthropic's Claude API does not use customer data for training by default on the API (confirmed in its usage policy), but EU data residency requires a specific contract configuration rather than being the default for API customers.

The 5-question assessment that takes 30 minutes

Running a full six-dimension assessment is the right approach for high-risk vendors (customer-facing, regulated decisions, operationally critical). For initial screening, deciding which vendors go to full assessment and which are excluded early, these five questions surface the most critical gaps in under 30 minutes.

Q1: Does your SOC 2 Type II include AI-specific controls, or does the audit scope cover only standard SaaS infrastructure?

A vendor with a SOC 2 that excludes the AI pipeline has audited its data center, not its model. This does not disqualify the vendor, but it means you need to obtain evidence about model-specific controls through a different path.

Q2: What is your documented opt-out process for training data use, and is training opt-out the default configuration for API customers or enterprise customers?

If the answer is that opt-out requires a specific contract amendment and is not the default, ask how many of their current customers are actually opted out. The answer tells you whether the opt-out is a real operational control or a paper policy.

Q3: Can you provide a list of all current subprocessors with a contractual commitment to notify us within five business days of any addition or change?

The list itself matters. The notification commitment matters more. Many vendors bury subprocessor change notification in a 30-day email list subscription. Five days is the threshold that gives your legal and security teams time to review before the change takes effect.

Q4: What is your incident detection-to-notification SLA, and what constitutes an incident for AI-specific failure modes, including hallucination at scale, bias detection in production outputs, and confirmed prompt injection breaches?

Most vendors have a standard incident response SLA covering data breaches and system outages. Fewer have documented incident definitions that include AI-specific failures. If the vendor's incident response policy does not define hallucination at scale as a reportable event, you will not be notified when it happens.

Q5: If your model is updated and the update changes behavior in ways that affect my use case, what is my recourse?

This question distinguishes vendors with real model governance from vendors who update models as they choose and treat customer complaints after the fact. The answer you want is some combination of advance notice, staging environment access, and a rollback mechanism. The answer that should concern you is "updates are applied automatically and we cannot revert individual customer environments."

Reassessment cadence

Vendor assessment is not a one-time activity. GenAI models update more frequently than traditional software, and behavior changes between versions can affect accuracy, bias properties, and safety in ways that are not always announced. A model that passed your initial assessment in January may behave differently in June after a silent update.

We suggest a tiered cadence; the FS AI RMF asks for periodic assessments without fixing an interval. For vendors whose AI is customer-facing, supports regulated decisions, or is operationally critical, review quarterly. That does not mean a full assessment every quarter. It means requesting a SOC 2 bridge letter covering the period since the last full report, reviewing any model update announcements from the vendor, verifying that incident notification SLAs were met in the prior period, and re-running a spot test of key outputs on a defined test set you maintain internally.

Lower-risk vendors (internal tools, productivity use cases, limited data exposure) can be reassessed annually. Annual reassessment should include a full SOC 2 review, a check on subprocessor changes, and a review of any regulatory or legal developments involving the vendor.

Build the reassessment calendar into your vendor contracts. Include a right to audit clause that allows you to request updated SOC 2 documentation and testing results at each reassessment interval. Most enterprise vendors will accept this language; resistance to a right to audit clause is itself a signal about the vendor's confidence in its controls.

For teams building out their vendor management program, the AI vendor due diligence checklist 2026 and embedded AI governance third-party tools guides provide supporting documentation templates. For companies that have already signed contracts and need to address gaps after the fact, the AI vendor contract redline template 2026 covers how to negotiate amendments.

The GDPR AI enforcement picture is also shaping vendor risk. The GDPR AI fines enforcement cases article documents the cases from 2025 and early 2026 where companies faced penalties for AI systems that lacked adequate vendor oversight, including cases where the company's own governance was adequate but the vendor's was not, and the company bore the liability anyway.

The regulatory direction is toward more accountability, not less. The Treasury FS AI RMF is a detailed US statement of what AI vendor oversight looks like in financial services. Treating it as a financial services-only concern misreads where procurement standards are heading.

Correction (October 22, 2026): An earlier version said the FS AI RMF has a "Section 4.2" separating intent documentation from evidence testing, sorts vendors into Tier 1 and Tier 2 with quarterly and annual reassessment, requires independent testing, hallucination rate thresholds and specific AI security tests, and has been referenced by NIST. We found none of these in the framework or Treasury's release. The framework follows the NIST AI RMF functions with 230 control objectives; the section now quotes its third-party objectives, and the cadence and testing advice are marked as our recommendations (Treasury, Cyber Risk Institute).

Related Reading

• Ambriz v. Google: AI Call Recording Consent Checklist

• USA Today sues OpenAI: $250M claim over ChatGPT news summaries

• MAS AI risk guidelines for vendors selling to Singapore financial institutions

• AI Models 2x More Likely to Refuse Criticizing Repressive Regimes

• OpenAI Wants Stricter AI Rules on Itself: What That Means for You

• Illinois SB 315 Signed: The Toughest AI Safety Law Yet

• China's AI Companion Law Takes Effect July 15: What It Requires

• AI insurance exclusion checklist 2026

• AI vendor due diligence checklist 2026

• AI vendor evaluation checklist

• Agentic AI vendor contract clauses 2026

• AI vendor contract redline template 2026

• Embedded AI governance third-party tools

• Privacy-first AI API no training GDPR CCPA

• Flock Safety: when your AI vendor's default settings share your data with federal agencies

• Anthropic's $1.5B Copyright Settlement: Training Data Has a Price Now