What is shadow AI in hospitals?

Shadow AI in hospitals is any use of artificial intelligence that happens outside the organization's approved governance process. Examples include staff pasting patient details into a consumer chatbot, a department buying an AI tool without review, or a vendor switching on an AI feature inside existing software without telling the hospital.

Where shadow AI comes from

Shadow AI is rarely malicious. It usually grows from three sources:

• Individual use. Clinicians and staff try public generative AI tools to draft letters, summarize notes, or answer questions, often because approved alternatives do not exist yet.
• Departmental purchases. A team signs up for a software subscription with a credit card or through a small contract that never reaches IT, privacy, or the AI committee.
• Embedded features. Existing vendors add AI capabilities to products the hospital already licenses. The contract may predate the feature, so nobody reviewed it as an AI tool.

AI agents add a fourth source: an approved agent can be granted new tools or connections over time, drifting into uses nobody reviewed.

Why it matters

The concern is not that staff are experimenting. It is that the hospital cannot meet its obligations for tools it does not know about.

• HIPAA. Under 45 CFR 164.308(b)(1), a covered entity may let a business associate handle protected health information only after obtaining "satisfactory assurances" that it will safeguard the information, documented in a business associate agreement. Entering PHI into a tool with no such agreement can be an impermissible disclosure. The minimum necessary standard in 164.502(b) also applies.
• Section 1557. 45 CFR 92.210(b) gives covered entities an "ongoing duty to make reasonable efforts to identify uses of patient care decision support tools" that use protected characteristics as inputs. Unknown tools cannot be identified or mitigated.
• Patient safety and quality. Joint Commission and CHAI guidance expects local validation and ongoing monitoring of AI tools. Shadow tools receive neither.

How hospitals reduce it

Bans alone can push use further out of sight. Approaches that tend to work better combine visibility with a fast path to approval:

1. Offer approved options. Provide sanctioned tools, with business associate agreements in place, for common tasks like drafting and summarizing.
2. Make intake quick. A short intake form and a risk-tiered review let low-risk tools move in days rather than months.
3. Discover what is already in use. Review network and expense data, software contracts, and vendor release notes for AI features.
4. Keep one inventory. The NIST AI Risk Management Framework (GOVERN 1.6) calls for mechanisms to inventory AI systems. Everything found goes into one registry with an owner.
5. Require identity to act. When agents must present a registered identity before they can reach clinical or business systems, unregistered agents are blocked by default.

How Skovos handles this

Skovos addresses the agent side of shadow AI. It keeps a registry of record with an accountable owner for each agent, and agents checked through Skovos act only within their registered scopes. Skovos does not observe systems it is not connected to, so discovery work remains necessary.

Frequently asked questions

Is using ChatGPT at work shadow AI in a hospital?

It is if the use has not been approved through the hospital's governance and privacy process, especially if any patient information is involved and no business associate agreement is in place.

How common is shadow AI in healthcare?

Reliable, peer-reviewed prevalence figures for hospitals are limited, so treat survey claims with care. Any hospital with broad internet access should assume some unsanctioned use exists.

Should hospitals block all public AI tools?

Blocking can reduce risk, but without approved alternatives it can shift use to personal devices. Pairing restrictions with sanctioned tools addresses the underlying demand.

Related reading

• AI agent inventory for health systems
• HIPAA-compliant AI deployment for health systems
• AI agent registry for hospitals

Sources

• 45 CFR 164.308, HIPAA administrative safeguards, including (b)(1) business associate contracts: https://www.law.cornell.edu/cfr/text/45/164.308
• 45 CFR 164.502(b), HIPAA minimum necessary standard: https://www.law.cornell.edu/cfr/text/45/164.502
• 45 CFR 92.210, Section 1557 patient care decision support tools: https://www.law.cornell.edu/cfr/text/45/92.210
• NIST, AI Risk Management Framework (AI RMF 1.0), GOVERN 1.6: https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf
• Joint Commission and CHAI, Guidance on the Responsible Use of AI in Healthcare: https://digitalassets.jointcommission.org/api/public/content/dcfcf4f1a0cc45cdb526b3cb034c68c2