Can You Use AI in Healthcare Without Breaking HIPAA? A Practical Guide

The short answer is yes — clinics use AI every day without breaking HIPAA. But there's a version of "using AI" that absolutely breaks it, and I see it constantly: a staff member pastes patient details into a consumer chatbot to draft a letter or summarize a note.

That one habit is a reportable breach. Building HIPAA compliant AI workflows isn't complicated, but it requires knowing where the lines are. This is engineering guidance, not legal advice — talk to a healthcare attorney before you ship anything.

What PHI actually is

PHI — Protected Health Information — is health information tied to an identifier. The medical content alone isn't PHI; it's the link to a person that matters. There are 18 identifiers under HIPAA, including the obvious ones (name, SSN, medical record number) and ones people forget: email addresses, phone numbers, full-face photos, IP addresses, and any date more precise than a year when tied to an individual.

"John Smith, DOB 03/14/1962, has stage 2 hypertension" is PHI. "A 64-year-old patient has stage 2 hypertension" generally is not. That distinction drives every strategy below.

The BAA: the contract that makes AI legal

If a vendor handles PHI on your behalf, HIPAA requires a Business Associate Agreement (BAA) — a contract where the vendor accepts liability for protecting that data. No BAA, no PHI. It doesn't matter how good the model is.

The good news: major AI vendors sign them.

• OpenAI signs BAAs for API customers (not the consumer ChatGPT app — the API, and the enterprise tiers).
• Anthropic offers BAAs on enterprise plans.
• Microsoft Azure OpenAI and Google Cloud Vertex AI sign BAAs under their cloud agreements.
• AWS covers Bedrock under its healthcare BAA.

The pattern: the enterprise/API tier gets a BAA; the consumer product never does. Always confirm the specific service and region are covered — a BAA with a vendor doesn't automatically cover every product they ship.

De-identification: make the problem disappear

The cheapest compliance strategy is to never send PHI at all. Strip identifiers before data hits the model:

• Remove or mask names, dates, MRNs, and contact details at the pipeline level — before the API call, not in the prompt hoping the model ignores them.
• Tools like Microsoft Presidio or a custom regex-plus-NLP layer handle most of this.
• For many tasks — drafting patient-education content, summarizing guidelines, building templates — de-identified data works just fine.

De-identified data falls outside HIPAA entirely. When in doubt, de-identify first.

Zero-retention API tiers

Even with a BAA, minimize what vendors keep. Enterprise AI APIs now offer zero data retention configurations: your inputs aren't stored, aren't logged beyond transient processing, and aren't used for training. Azure OpenAI and Anthropic's enterprise tier both offer this, as do platforms like OpenRouter on specific routes.

Two rules: never use an API tier that trains on your data for anything remotely clinical, and get the retention terms in writing, not from a marketing page.

Self-hosted models: the nuclear option

When no cloud vendor will do — ultra-sensitive data, institutional policy, or a client who just wants maximum control — run the model yourself. Open-weight models (Llama, Mistral, Qwen families) deployed on your own GPU server or a private VPC mean no PHI ever leaves your infrastructure. No BAA needed for the model, because there's no business associate — just you.

The trade-offs are real: you own the ops burden, the hardware cost, and a quality gap versus frontier cloud models for some tasks. For summarization, drafting, and internal Q&A, a good 70B-class open model is more than enough. I've deployed these for healthcare clients where "the data doesn't leave our building" was a hard requirement.

Practical rules of thumb for a clinic

1. Never paste patient details into consumer chatbots. Not ChatGPT free, not any personal account. Ever.
2. Get a BAA before any PHI touches a vendor — and verify the specific service is covered.
3. Use API/enterprise tiers with training opt-out and zero retention.
4. De-identify by default, even when you have a BAA. Less exposure is always better.
5. Log access. Know who sent what data where. If you can't audit it, you can't defend it.
6. Train your staff. The biggest HIPAA risk with AI isn't the technology — it's a well-meaning employee with a deadline.

HIPAA compliant AI is less about exotic technology and more about boring diligence: the right contract, the right tier, and a pipeline that treats patient data as radioactive by default. Get those three right and AI healthcare compliance stops being scary — it becomes a checklist.

This post is engineering guidance based on how I build these systems, not legal advice. Consult a healthcare attorney for your specific situation.