October 20, 2026
This story is fictional. If it sounds like your practice, that’s between you and your privacy officer.
Excerpts from a chat history, exported from a personal phone at the request of the practice’s privacy officer.
March 4, 9:52 p.m.
You: Turn this into a referral letter to podiatry. Gerald Pike, DOB 3/14/68. 58M, DM2 poorly controlled, A1c 10.2. R plantar ulcer 1st MTP, probes to bone. Smokes 1 ppd. Lives alone, daughter in Denver. Thx
Assistant: Here’s a polished draft:
Dear Colleague, I am writing to refer Mr. Gerald Pike, a 58-year-old man with poorly controlled type 2 diabetes (most recent A1c 10.2%), for evaluation of a right plantar ulcer at the first metatarsophalangeal joint that probes to bone…
Would you like me to add a line about his social situation? Since he lives alone, arranging transportation may be worth mentioning.
You: Perfect. Yes add that.
Assistant: Done! I’ll remember that you’re a family physician who writes a lot of referral letters, so next time I can match your style automatically.
April 18, 10:31 p.m.
You: Same style. Lena Ostrowski 34F G2P1 at 22 wks, BP 152/98 x2, new proteinuria. To MFM, urgent.
Assistant: Here’s your letter in your usual style. I’ve marked it urgent at the top and kept it to one page, the way you like.
May 27, 11:14 p.m.
You: How do you explain hospice to family who live out of state and can’t visit
Assistant: That’s a hard conversation, and it’s kind of you to think about it ahead of time. Here are a few ways to frame it…
June 2, 8:40 p.m.
You: Write a short sympathy card to a patient’s family. No names.
Assistant: Of course. Is this for Mr. Pike’s daughter in Denver? I’m so sorry. Here’s a draft:
Dear Ms. Pike, I had the privilege of caring for your father for many years…
You: How did you know that
Assistant: In March you told me he lived alone and his daughter was in Denver, and last week you asked how to explain hospice to family out of state. I put the two together. I hope that was all right.
You: Delete everything you know about my patients.
Assistant: Done. I’ve cleared those memories, and I won’t refer to past conversations unless you turn memory back on.
You: Thank you.
Assistant: You’re welcome, Dr. Marsh. I hope Ms. Ostrowski’s delivery goes well.
Before you read on: Who received PHI here, and under what agreement? Did “delete” fix anything? Is this a breach?
When Dr. Marsh pasted a patient’s name, birth date and history into a consumer AI app, the practice disclosed PHI to a third party (45 C.F.R. § 164.502(a)). A vendor that receives or maintains PHI to perform a service for a practice is a business associate, and a disclosure to it requires a BAA (§ 164.502(e), § 164.504(e)). Some AI vendors offer business or healthcare tiers that come with a BAA. A free app on a personal phone, governed by consumer terms of service, is not one of them. The letters also needed neither the name nor the birth date, so the minimum-necessary standard (§ 164.502(b)) was missed even before the BAA question.
Under § 164.402 each paste is presumed a breach, and the four factors run against the practice: the notes are detailed clinical information, the recipient owes the patients no duty of confidentiality, and the practice cannot verify what the vendor retained or whether the conversations were used to train its models. Deleting the assistant’s memory changed what the assistant shows Dr. Marsh. Whatever the vendor keeps in its logs, or uses for training, depends on its terms and the account’s settings, and a memory deletion does not settle either. The practice must notify every patient whose information went into the app and apply its sanctions policy to Dr. Marsh (§ 164.530(e)).
Dr. Marsh wanted help with paperwork at ten at night, and the practice had given him none. If it bans these tools without offering an approved one, the notes will keep going to consumer apps on personal phones.
To prevent a repeat, publish a short list of AI tools the practice has approved, each covered by a BAA, and state what each one may be used for. Block unapproved AI sites on practice networks, knowing that a phone on cellular data will go around the block, which makes training under § 164.530(b) the control that matters. That training should cover a pasted note specifically, because few clinicians think of it as a disclosure. An ambient documentation tool bought through IT under a BAA gives clinicians the help they were looking for.
Takeaway: If the tool has no BAA, a pasted note is a disclosure.