AI Agent Inventory for Health Systems: How to Find Every Agent Already Running
Ask a health system CIO how many AI systems are running in production and the honest answer is usually a range. Ask the compliance officer and the range gets wider. Ask the department directors and it gets wider still, because each of them knows about tools the others do not. The gap between the highest and lowest answer is the governance problem in one number.
An AI agent inventory closes that gap. It is the first control in any serious AI governance program and the one most organizations rush past, because it looks like administrative work. It is not. Everything that follows, policies, enforcement, audit, remediation, is only as complete as the list it runs against. This article covers where agents hide, what to record about each one, why discovery beats surveys, and what to do with the inventory once it exists.
Where the AI you did not register is hiding
Very little of the AI running in a health system arrived through an "AI procurement." Most of it came in sideways. The places to look:
- Vendor products that added a model. The EHR's documentation assistant, the revenue-cycle platform's coding suggestions, the patient-engagement tool's message drafting, the imaging viewer's triage flag. Each shipped in a release note, not a contract, and each is an AI system with its own data flows and failure modes.
- Departmental tools. A scheduling optimizer the OR team licensed. A transcription service two clinics adopted. A chatbot on the foundation's donor site. Bought on a card, renewed automatically, never reviewed.
- In-house builds. A data science team's readmission model in a scheduled job. An analyst's script that calls a model API to summarize referrals. An internal copilot a developer wired to the ticketing system over a weekend.
- Agents with tools. The newest class: software that does not just answer but acts. It has a model key, a set of tools (calendar, email, a records API, a database), and permission to use them. Some were built on purpose; some are vendor features that quietly gained write access in an update.
- Direct model-provider access. Individual API keys to OpenAI, Anthropic, Google, or Azure, issued to a person or a project, used from wherever. The provider bill is often the first place the inventory shows up.
The common thread is that the health system's traditional control points, procurement and change management, saw none of it. Which means the inventory cannot be built from those records. It has to be built from where the AI actually runs.
What to record for each system
A name on a list is not an inventory. The governance profile is what makes the entry useful, and it is what every downstream control reads. For each system, record:
- Identity. Name, vendor (or "internal"), the team that owns it, and a named accountable person.
- Purpose and use case. What it is for, in one sentence, and which category it falls into: clinical decision support, documentation, revenue cycle, patient communication, operations, research. Purpose is what policies are written against.
- Data classes. PHI, PII, financial, de-identified, or none. Be honest; a system that "only summarizes" the chart handles PHI.
- Model and provider. Which model, hosted where, by whom. For vendor products this is often unknown, and "unknown" is a finding worth recording.
- What it can do. Read-only, or does it write? To which systems? Does it send anything to a patient? Does it influence a clinical decision? The answers determine risk tier.
- Deployment status. Proposed, pilot, production, retired. A surprising amount of "pilot" AI has been in production for a year.
- Vendor terms. Is there a business associate agreement that covers this specific AI function, including the model provider behind the vendor? Signed, pending, missing, or not required.
- Review cadence. When the owner last attested that the profile is accurate, and when they must next.
Every empty field is a governance finding. That is a feature: the inventory tells you what you do not know, in priority order, before anything goes wrong.
Discovery beats surveys
The instinct is to send a form. Department leads list their AI tools, someone compiles a spreadsheet, the committee reviews it. Three months later the spreadsheet is a historical document.
Surveys fail for structural reasons. People report what they remember and what they think counts as AI. They do not report the feature that turned on in a vendor update, the key a contractor used, or the agent a colleague built. And nothing updates the survey when things change, which in AI is weekly.
Discovery works from the other end. Connect the places AI actually runs and let the inventory populate itself:
- Model providers and gateways. An organization account with a model provider lists every key and every project. Routing model traffic through a governed gateway surfaces every caller, every model, and every tool call, continuously.
- Connected applications. Integrations into the EHR, the email system, calendars, and data platforms show which agents have been granted access to what.
- Agent heartbeats. Agents built on a governed runtime report their own environment: model, tools, connectors, version. That is the difference between "the agent said it could send email" and "the agent has a send-email tool configured right now."
Discovery does not replace the human step; it feeds it. When a new agent appears, the owner is asked to complete its profile. When a profile is stale, the owner is asked to re-attest. The system does the finding and the nagging; people do the judgment.
Risk-rank before you review
The first pass of discovery usually produces more entries than anyone expected, and the committee cannot review them all at once. Rank them. A simple, defensible ordering:
- Takes actions in a system of record or sends anything to a patient.
- Handles PHI and has no confirmed BAA covering the AI function.
- Influences a clinical decision.
- Handles PHI with a confirmed BAA.
- Everything else.
Work the list top down. Tier one gets a completed profile and an enforced policy this month. Tier five gets a profile when its owner gets to it. The inventory shows leadership the whole picture and lets them accept, explicitly, the risk of what is not yet reviewed, which is a far better position than not knowing.
From inventory to enforcement
An inventory that stops at the list is a compliance artifact. An inventory that feeds enforcement is a governance program. The transition looks like this:
- Policies are written against profile fields. "Systems that handle PHI may not call a model provider without a BAA." "No agent may write to the EHR unless its profile says clinical write is approved." The policy engine reads the inventory to know which rule applies to which system.
- Drift is detected against the profile. The profile says read-only; the agent's environment now includes a delete tool. That is a configuration violation, raised automatically, before the tool is ever used.
- Findings roll up to a compliance score. Unregistered agents, missing owners, expired attestations, uncovered PHI: each is a control with a severity, and the board report is generated from the same data, not assembled by hand the week before the meeting.
How Skovos does it
Skovos builds the inventory by discovery: connect your model providers, gateways, and applications, and every agent and vendor system appears in one register with its observed environment. Each entry carries the governance profile described above, and every gap is a finding with a severity. Policies are written in plain language against those profiles, enforced on every model call and tool action, and drift between what a profile claims and what an agent can actually do is flagged before it matters. See how the inventory works, or start a free trial and register your first system in ten minutes.
Frequently asked questions
What counts as an AI agent in a health system?
Any software that uses a model to make or influence a decision, or to take an action, in a clinical, operational, or patient-facing workflow. That includes standalone agents your teams built, AI features inside vendor products, and integrations that call a model provider directly. If it reads PHI or writes to a system of record, it belongs in the inventory.
How is an AI inventory different from an IT asset inventory?
An IT asset inventory tracks systems. An AI inventory tracks what those systems are allowed to decide and do, what data they use to do it, which model and vendor sit behind them, and who is accountable. The governance profile is the point; the list of names is just the index.
Can we build the inventory with a survey?
You can start one, but a survey captures what people remember on the day they answer it. Discovery from the systems agents actually use (model providers, gateways, connected applications) captures what is running, and keeps capturing it as things change.
How often should the inventory be reviewed?
Continuously for discovery, and on a fixed cadence for attestation. A practical pattern: automated discovery daily, owner re-attestation quarterly for systems that touch PHI or take actions, annually for everything else.
What do we do with the inventory once we have it?
Register a governance profile for each entry, assign an owner, record the BAA status, then write policies against the inventory and enforce them. An inventory that does not feed enforcement is a spreadsheet.