AI vendor intake and review in hospitals

AI vendor intake and review is the process a hospital uses to evaluate a third-party AI tool before it is piloted or deployed. It typically covers the intended use and risk tier, privacy and security, evidence of validation and bias testing, integration and monitoring plans, and contract terms, and ends with an approval, conditions, or a decline.

Why intake matters

Most AI in hospitals comes from vendors, and the hospital remains accountable for how it is used. The NIST AI Risk Management Framework's GOVERN 6.1 calls for "policies and procedures ... that address AI risks associated with third-party entities," and GOVERN 6.2 for "contingency processes ... to handle failures or incidents in third-party data or AI systems deemed to be high-risk."

The Joint Commission and CHAI guidance on the Responsible Use of AI in Healthcare says that during procurement, organizations "should request information from developers/vendors on how AI tools were tested and validated for their intended use, whether they are willing to tune and/or validate a sample that is representative of the deployment context, and how relevant biases were evaluated." It adds that "monitoring responsibility should be discussed as part of third-party procurement and contracting."

What a typical review covers

• Use case and risk tier. What the tool does, who uses it, how close it is to clinical decisions, and whether it can act on its own.
• Regulatory status. Whether the tool is an FDA-regulated device, and whether it is a patient care decision support tool under 45 CFR 92.210, which triggers identification and mitigation duties for tools using protected characteristics.
• Transparency. Model documentation, intended and out-of-scope uses, training data description, and, for predictive tools in certified EHRs, the HTI-1 source attributes.
• Validation and bias. Performance evidence on populations similar to the hospital's, subgroup results, and willingness to support local validation.
• Privacy and security. Whether PHI is involved; a business associate agreement under HIPAA (45 CFR 164.308(b)); the minimum necessary standard; data retention; and whether hospital data may be used to train vendor models. HIPAA also requires an accurate and thorough risk analysis of risks to ePHI (164.308(a)(1)(ii)(A)).
• For agents, permissions. Which systems the agent will connect to, which actions it can take, whose credentials it uses, and whether the hospital can revoke access without the vendor.
• Monitoring and exit. Who monitors performance after go-live, how updates are communicated, how incidents are reported, and how the tool is turned off and data returned at the end.

Contract terms to consider

The RUAIH guidance suggests data use agreement terms such as clearly defined permitted uses, data minimization, prohibition of re-identification, third-party security obligations, and audit rights. Hospitals may also seek notice of material model changes, cooperation with local validation, and access to logs.

How Skovos handles this

Skovos picks up where intake ends for AI agents. Approved scopes from the review become the agent's registered permissions, so the agent can only do what was approved, and the audit trail shows whether it stayed within them.

Frequently asked questions

Who should run AI vendor intake?

Usually the AI governance committee or a working group under it, drawing on privacy, security, legal, clinical, and procurement reviewers.

How long should AI intake take?

It should scale with risk. A risk-tiered process lets low-risk administrative tools move quickly while clinical or autonomous tools get deeper review.

Does intake replace post-deployment monitoring?

No. Intake reviews the vendor's evidence before use. Monitoring checks how the tool performs in the hospital's own setting over time.

Related reading

• Enterprise AI procurement for health systems
• How health systems evaluate enterprise AI platforms
• AI agent registry for hospitals

Sources

• NIST, AI Risk Management Framework (AI RMF 1.0), GOVERN 6.1 and 6.2: https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf
• Joint Commission and CHAI, Guidance on the Responsible Use of AI in Healthcare: https://digitalassets.jointcommission.org/api/public/content/dcfcf4f1a0cc45cdb526b3cb034c68c2
• 45 CFR 164.308, HIPAA administrative safeguards: https://www.law.cornell.edu/cfr/text/45/164.308
• 45 CFR 164.502(b), minimum necessary: https://www.law.cornell.edu/cfr/text/45/164.502
• 45 CFR 92.210, Section 1557 patient care decision support tools: https://www.law.cornell.edu/cfr/text/45/92.210