Physician-in-the-loop: where AI belongs in clinical workflow (and where it doesn't)

"Physician-in-the-loop" is the phrase of the moment in clinical AI. It appears in investor decks, marketing copy, and regulatory filings — sometimes accurately, and sometimes as a softening verb applied to products that would prefer the physician be no more than a rubber stamp at the end of an automated pipeline.

I want to make the case for the stricter version of the term: the version where the physician is the sole interpreter of clinical data, the only entity allowed to assign meaning to a pattern, and the trigger for every output that could affect patient care. An architectural commitment, not a tagline.

The seductive claim

The seductive claim in clinical AI goes something like this: our model reads the scan and tells the clinician which cases are urgent. Or: our model reviews the patient's intake and flags the likely diagnoses. Or: our model scores each patient's risk and routes them to the appropriate care pathway.

Each of these is an interpretation. Each performs a function that, if performed by a human, would require a medical licence. Each, in the regulatory frameworks of Canada, the United States, and the European Union, is likely to constitute a medical device — Software as a Medical Device (SaMD).

SaMD is not a forbidden category. Many valuable products live inside it. But it is a regulated category with specific obligations: classification review, evidence of clinical and analytical validity, post-market surveillance, labeling, adverse event reporting, and — depending on the jurisdiction and risk class — prospective clinical studies. These are the cost of entry.

The mistake I see most often is not the choice to build a SaMD product. The mistake is assuming you are not building one because the interpretation is dressed up as "decision support" or "triage assistance" or "flagging." Regulators are largely unimpressed by the packaging.

The line

The line between a structured data tool and a medical device is not as fuzzy as it is sometimes made out to be. A few rough tests:

Does the product produce an output that a clinician acts on without independently re-deriving the reasoning? Does it assign clinical meaning to a pattern — this looks like condition X, this patient is high risk, this case should be seen urgently? Does it recommend a specific clinical action, even indirectly, by ordering or ranking options? Is the clinician's role limited to confirming an output the system produced?

If the honest answer to any of these is yes, you are building a medical device. Physician-in-the-loop, in that architecture, is marketing.

What physician-in-the-loop actually requires

The stronger version of the principle has three properties.

The physician is the origin of the clinical reasoning, not the validator of it. The system does not pre-select a diagnosis and wait for approval. The physician examines the structured data, arrives at an impression, and enters it. The system's output is a consequence of that entry, not a prompt for it.

The system's outputs are triggered by explicit physician action. A clinical note is not generated in the background on the hope that the physician will edit it. It is generated when — and only when — the physician has confirmed the diagnosis. Until that moment, there is nothing to edit, because there is nothing to say.

The system does not fill gaps the physician left blank. If the patient did not record a field, the system does not impute a value. If the data is incomplete, the output reflects that incompleteness. The system does not speculate on the physician's behalf.

These are not cosmetic. They make the difference between a tool the physician uses and a tool that is using the physician.

What a non-interpreting AI can do well

Giving up interpretation does not mean giving up value. It means relocating the value to a different place in the workflow.

A non-interpreting system — rule-based logic and deterministic summaries — can do a great deal of clinically useful work without crossing into SaMD territory. It can capture patient-reported data in a structured, enforceable format before the visit. It can present validated questionnaires by demographics and chief complaint, using explicit non-diagnostic rules the clinician can audit. It can calculate validated instrument scores using their published, deterministic algorithms, and present the numeric result and published severity band without asserting what that band means for the specific patient. It can assemble a formatted clinical note, once the physician has confirmed the diagnosis, by transcribing the physician's decision into a well-structured document. It can present patient data in a form designed for the clinician's reasoning — visit-over-visit comparisons, time-indexed diary data, completeness indicators — without interpreting any of it.

None of this requires interpretation. All of it delivers real time savings and real improvements in data quality. And all of it is architecturally stable: there is no model in the interpretive sense, so there is no model drift to manage. There is a schema, a set of rules, and a set of views.

The unglamorous truth is that most of the time savings in a modern clinic come from this kind of work — capture, selection, assembly — rather than from anything that looks like clinical cognition. The cognition is what the physician is paid for. The friction around it is what the software can remove.

What we give up, honestly

A non-interpreting, rule-based, deterministic system cannot produce the kind of output that does well in an investor demo. It does not read the scan. It does not flag the urgent case. It does not predict the outcome. The demo is quieter.

What it produces instead is a dashboard the clinician actually uses, every week, for years, without being retrained. A note format the clinician stops noticing because it gets out of the way. A data structure that gets more valuable every quarter, not less.

The loud version of clinical AI is a race. The quiet version is a compounding asset. We have placed our bet on the quiet version, and the physician-in-the-loop principle — in its stricter reading — is the architecture that makes the bet coherent.

The one sentence

The physician's action is the trigger.

Nothing in the system that could affect patient care happens before the physician explicitly causes it to happen. The software's role is to make that action faster, cleaner, better-informed, and better-documented. The software's role is not to perform the action on the physician's behalf and wait for a signature.

That is what "physician-in-the-loop" means to me. It is not a description of the user interface. It is a description of who is doing the medicine.

---

Dr. Christian Diab is the Medical Director and co-founder of CuraDia, a Québec medtech company building Uro-OS — the clinical operating system for urology and pelvic medicine.