AI vendor intake for hospitals: make the evidence change the decision

What should AI vendor intake produce?

AI vendor intake is a hospital's review of a proposed third-party AI workflow before a pilot or deployment. Its useful output is a scoped decision: proceed within stated boundaries, hold for evidence or remediation, or decline. Record who decides, what they relied on, and what would reopen the review.

A questionnaire response is not a verified control. A completed contract is not evidence that a workflow performs acceptably. Keep commercial review, evidence verification, and deployment authorization distinguishable, even when the same working group coordinates them.

The NIST AI RMF Core, GOVERN 6 and MANAGE 3, addresses third-party AI risk. This article offers a practical review method, not a regulatory checklist.

Agree the scope before requesting documents

Describe the intended users, department, task, product version, connected systems, data categories, and allowed actions. Name excluded uses. If a tool moves from drafting to sending or writing, reopen the relevant review rather than carrying the earlier approval forward.

Ask what could happen if the output is wrong, unavailable, or acted on without review. Use that discussion to choose reviewers and review depth under your hospital's policy. The workflow owner defines acceptable use; evaluation staff examine performance; privacy, security, and integration teams investigate their respective boundaries. Procurement coordinates supplier responses, while the authorized hospital decision-maker resolves exceptions.

Ask privacy and legal reviewers which obligations and contract terms apply to this particular use. This guide does not determine regulatory status or whether a specific agreement is legally sufficient.

Request evidence that can be checked

• Use and performance: What was tested, on which version, in what setting, and against which comparator? Request the protocol, limitations, relevant subgroup findings, and consequential error examples through approved channels. Agree local acceptance criteria before evaluating results.
• Data handling: Where does each data category travel? Request processing locations, downstream providers, retention and deletion arrangements, and training or secondary-use terms. Have privacy and security compare documentation with the proposed configuration.
• Actions and oversight: Which account can read, draft, write, send, or execute? Have the integration owner test prohibited actions as well as successful ones. Ask users to demonstrate the actual approval point.
• Failure and change: Who receives incidents and update notices? What happens to queued work during a pause? Identify fallback, restart authority, monitoring ownership, and exit arrangements before deployment.

The CHAI Applied Model Card offers a disclosure structure for intended use, performance, and risks; its resource page labels the materials a draft release. A completed card is information to assess, not local validation or an endorsement.

Joint Commission and CHAI guidance, section 4, recommends seeking intended-use validation evidence during procurement and discussing monitoring responsibilities. Use those discussions to request relevant evidence, not to infer certification.

Worked example: a pause button that leaves queued work

Consider a fictional hospital reviewing a work-order assistant from an unnamed fictional supplier. Workflow FAC-017 drafts non-clinical maintenance tickets. The sandbox contains only synthetic requests. Production use, emergency dispatch, and clinical equipment decisions are outside this example.

The supplier says pausing prevents further ticket creation. The hospital writes its criterion first, then tests new and queued actions separately. In this fictional test, a queued draft appears after the interface reports paused. The reviewer records a failed criterion rather than translating the supplier's answer into approved.

Assign the supplier liaison as action_owner_role, a proposed action_due_date of 2026-10-06, and a decision review on 2026-10-07. These are fictional scheduling choices, not recommended deadlines. A new version needs new evidence; changing the response cell cannot close the failed test.

Separate receipt, verification, and authorization

Use one tracker row per evidence request. Keep evidence_date distinct from verification_date, and record evidence_version so reviewers can see what changed. Write a rationale in applicability_and_reason when a request does not apply. An unanswered request remains an open_gap, not an implicit pass.

1. Received: the supplier provided material, but the assigned verifier has not assessed it.
2. Verified for stated scope: record the method, result, limitations, and configuration tested. This is not approval of unrelated uses.
3. Gap: specify remediation, owner, due date, and the decision affected.
4. Authorized decision: the hospital decision-maker records proceed, hold, or decline and any enforceable conditions.

For FAC-017, business discussions can continue while production remains on hold. Closing this pause gap would not close a separate permission gap or establish that drafts are accurate. Keep each decision traceable to its own evidence.

Carry the conditions into operations

After authorization, link the evidence tracker to the workflow inventory. Assign monitoring and change-review owners, next_review_date, and earlier triggers such as expanded access or a changed model. Recheck supplier claims when the deployed scope changes.

An inventory or evidence CSV cannot enforce those conditions. With Skovos, evaluate supported agents and configured controls against the actual integration: demonstrate allowed and denied actions, pause behavior, and available evidence. Do not assume it governs unconnected tools or replaces clinical evaluation, security review, or legal advice.

Before the next supplier meeting, choose one unresolved decision and ask: Which evidence would change our answer, who will verify it, and what happens if it is unavailable? That is a more useful next step than requesting another generic presentation.

Keep working