Turning what you now know into a point-of-care habit
You know an LLM predicts tokens (L1), samples from uncertainty (L2), and that retrieval vs. reasoning are different strategies with different failure modes (L3). This lesson converts all of that into a small set of habits you can use Monday.
The goal isn't to use these tools more, or less. It's to use them where their strengths fit and your verification covers their weaknesses.
Sort tasks by one question: is the model working from information you gave it, or recalling specifics from memory? The first is safe; the second needs verification.
Turn your op note into a discharge summary; simplify a consent explanation. It's working from your text.
Mechanisms, definitions, widely-taught principles. Densely represented in training; usually reliable.
Differential lists, talk outlines, first drafts. You supply the judgment; the model supplies breadth.
Condense a paper you pasted in. Grounded in the source, not memory.
References, doses, p-values, trial figures from memory. Highest fabrication risk -- verify every one.
Letting the output decide management. It can't examine the patient and reflects the average case.
Last month's trial or guideline, asked from memory. It guesses unless it retrieves.
PHI into a non-approved tool is a privacy breach, full stop. Covered next.
You don't need to verify everything equally. You need to verify the checkable specifics and prompt in ways that reduce error.
Stop asking "Is this answer right?" and start asking "What would make this answer wrong, and have I checked that?" For a citation: does it exist? For a recommendation: does it fit this patient? For a recent claim: is it past the cutoff? The failure mode is usually predictable from the task.
Everything above is about accuracy. This is about not breaking the law or your patients' trust.
Do not paste protected health information into a consumer AI tool. Public ChatGPT, Claude, Gemini, and most consumer apps are not HIPAA-compliant by default. Names, MRNs, dates, and identifiable details entered there may be retained, used for training, or exposed. De-identification is harder than it looks -- a rare diagnosis plus a date can re-identify a patient.
"What's the workup for an acutely ischemic limb?" carries no patient data. Asking about mechanisms, definitions, and general management is fine.
Drafting a note from a real chart, summarizing a real patient's course -- only inside an institutionally-sanctioned, BAA-covered environment. Many systems now offer these; use them, not the consumer app.
Feeding clinical datasets to any AI for a study needs IRB review and data-governance sign-off -- even de-identified. Loop in compliance before the data moves.
Simple rule of thumb: if you'd hesitate to post it on a public forum, don't paste it into a consumer AI tool. When in doubt, strip the identifiers or use your institution's sanctioned system.
For each real-world use, decide whether to proceed, verify, or stop -- and why.
Module 2 - Lesson 4 complete · You finished the LLM module
You can now explain, to a skeptical colleague, how these tools actually work, why they fail, and how to use them without getting burned. That's the whole goal of this module.