Photo: John M. Lund Photography/Getty Images
Unauthorized – or shadow – AI tools and chatbots are widely used across U.S. hospitals and health systems, including for direct patient care, according to a survey from Wolters Kluwer Health.
The prevalence raises concerns about patient safety, data privacy and regulatory compliance, as the AI tools are being used without clear governance or oversight.
Forty percent of survey respondents have encountered an unauthorized AI tool in their organizations, and nearly 20% have used them, according to the national survey of more than 500 healthcare professionals and administrators.
Scott Simeone, SVP and chief information officer at Tufts Medicine said by statement: “GenAI is showing high potential for creating value in healthcare but scaling it depends less on the technology and more on the maturity of organizational governance. While progress has been made in governance, there is still work to be done to evolve tools for control and monitoring, so they scale for clinical contexts. As clinical use grows, health systems need enterprise‑grade controls, transparency, and literacy – so clinicians and patients understand when AI is supporting decisions, how it works, and where human judgment remains essential.”
The top drivers for using shadow AI are faster workflows, better functionality and lack of approved tools.
The findings from the survey showed:
- Administrators are three times more likely to be actively involved in healthcare AI policy development than providers (30% versus 9%), suggesting clinicians are likely using AI without clear guidance.
- More than half of healthcare professionals frequently use AI tools or rely on AI tools for their work. Nearly 90% believe that AI will significantly improve healthcare within the next five years.
- Patient safety ranks as the top AI risk for both providers (25%) and administrators (26%), followed by concerns around inaccurate outputs, privacy and data breaches.
- Health data security is also a worry: Nearly a quarter (23%) of healthcare professionals express concern about privacy and security risks associated with AI in healthcare, highlighting fears of healthcare data breaches, unauthorized access and the need for robust protection measures.
“Doctors and administrators are choosing AI tools for speed and workflow optimization, and when approved options aren’t available, they may be taking risks,” said Yaw Fellin, senior vice president and general manager of Clinical Decision Support and Provider Solutions at Wolters Kluwer Health. “Shadow AI isn’t just a technical issue; it’s a governance issue that may raise patient safety concerns. Leaders must act now to close the policy gap around AI use, develop clear compliance guidelines, and ensure that only validated, secure, enterprise-ready AI tools are used in clinical care.”
Email the writer: SMorse@himss.org