A Calm, Practical Checklist for Quality Pros to Start Mapping the AI Already at Work

You don’t need a robot nurse rolling down the hallway to know artificial intelligence has entered the room.

AI in healthcare isn’t just some “someday” innovation or C-suite talking point. It’s already here—embedded in the workflows, dashboards, and tech tools we touch every day. But here’s the catch: most of it is working quietly, behind the scenes.

And if you don’t know where it’s hiding… how can you evaluate it?

As quality professionals, we’re wired to improve processes, reduce risk, and keep patient care grounded in safety and equity. But that mission becomes harder if we don’t even know that machine learning is silently influencing decisions—sometimes good ones, sometimes… well, not so good.

In the past, we trusted computer-generated reports without much hesitation—nobody questions a validated rule-based fall risk alert. But AI is different. It’s not following fixed rules—it’s making predictions based on patterns in data we didn’t design. So when we’re handed an AI-generated alert list, are we validating the list itself... or just assuming it’s accurate?

That distinction matters. If we haven’t reassured ourselves—and occasionally confirmed—that those outputs are accurate… how can we improve what they’re building in our care?

Where AI Might Be Hiding in Plain Sight

You might not see “AI” printed on the label, but here are a few places it’s often tucked away:

• A smart tool that predicts patient risk scores? Likely trained on historical data.
• That sepsis alert that fires based on pattern recognition? Yep, probably AI.
• Radiology reads pre-populated with suggested findings? AI is likely your ghostwriter.
• Chatbots handling patient questions in the portal? Hello, large language model.

Some of these tools are helpful. Some may carry risks. Most just need someone to ask the right questions.

 

The AI Inventory Checklist

Does your quality team have a list of all the AI integrations in your health system? 
Without a clear inventory of where AI is operating and what it’s doing, how can we create oversight processes to ensure it’s accurate, unbiased, and truly improving outcomes?

To help you get started, here is a one-page AI Inventory Checklist you can use with your team. It’s designed for quality pros, clinical leaders, or anyone who may be missing this foundational piece—the one that ensures your AI oversight process isn’t just present, but purposeful.

AI USE AREA	QUESTIONS TO ASK
Clinical Decision Support	Is this tool generating or suggesting clinical advice based on data?
Risk Scoring / Predictive Analytics	Is this score generated by a machine-learning model? What data trained it?
Clinical Documentation Tool	Does this tool auto-summarize or insert language based on AI predictions?
Imaging and Diagnostics	Are any radiology or imaging reads augmented by AI? Which ones?
Patient Communications	Is there a bot responding to portal messages or scheduling requests?
Administrative Automation	Are prior auths or insurance checks automated using AI?
Operational Management	Do we use systems that predict patient flow, length of stay, or scheduling?
Billing and Coding Tools	Are any tools auto-assigning billing codes or flagging claims using AI?
Vendor-Supplied Algorithms	Do any of our vendors include AI in their platforms—even if it’s not labeled?
Third-Party Apps or Devices	Do any connected tools send alerts or recommendations directly to staff?

Here’s how to use it:

1. Ask around – Talk with IT, clinical ops, nurse managers, or your EHR rep.
2. Start a list – Wherever you see AI functionality, document it.
3. Ask the questions – What kind of data trained it? Who validates it? How often is it reviewed for bias?  
4. Make a plan - How will you monitor it? How will issues be reported? What outcomes or quality measures will you track?

This isn’t about halting progress. It’s about shining a light so we can steer it wisely and be proud of it.

The Question That Grounds Us

You don’t need a high-tech IT degree to be a quality leader in the AI-era.
You just need curiosity, a little courage, and the instinct to ask,
“Is this truly improving our care?”

And if the answer isn’t clear—then maybe that’s where the work begins.

I’m excited about the role AI is playing in transforming healthcare and unlocking extraordinary patient care.  Together, let’s keep exploring its impact and creating practical tools that help us ask better questions and shape better systems.

Let’s build this thing together.

 

 

Author’s Note:

This post was developed in collaboration with my ChatGPT AI assistant, Quincy—who helps me shape the swirl of insights, questions, and late-night epiphanies into clear, actionable content. The strategy and direction are mine; the wording, a tag-team back and forth effort polishing the rhythm and flow.  Welcome to the new way of working—human-led, AI-supported.