How are LLMs used in dermatology?

Most writing about artificial intelligence in dermatology is really about computer vision. That is understandable: skin disease is visible, dermatology generates enormous quantities of labelled images, and image classification was the first medical task where machine learning produced results that impressed clinicians. Large language models are a different technology addressing different problems, and conflating the two has made the field harder to think about clearly.

This is an attempt to separate them.

Vision models and language models solve different problems#

A convolutional network trained on dermoscopic images learns a mapping from pixels to a label. Given a photograph of a lesion, it returns a probability distribution over diagnoses. That is a narrow, well-posed task, and it is the task that most “AI beats dermatologists” headlines refer to.

A language model does not see the lesion. It works with text: the referral letter, the history, the clinic note, the literature. If you ask a text-only model to diagnose from a description, you are asking it to do something a dermatologist would also refuse to do, and it will usually answer anyway. That failure mode - fluent, confident, and unanchored - is the central practical problem.

The interesting recent development is that the distinction is dissolving. Multimodal models accept images and text in the same context window, so the boundary between “the vision system” and “the language system” is no longer architectural. It is still useful conceptually, because the failure modes remain different.

Where language models are actually being used#

In practice, the deployments that survive contact with a clinic are unglamorous:

• Documentation. Ambient scribes that listen to a consultation and draft the note. This is the single most widely adopted clinical application of language models, and it is not a diagnostic tool at all.
• Correspondence. Drafting GP letters from structured clinic data, which in dermatology is high-volume and highly templated.
• Structured extraction. Turning decades of free-text histopathology reports and clinic letters into queryable fields. Retrospective research in dermatology is often bottlenecked by exactly this.
• Triage support. Reading a referral and flagging urgency, rather than making a diagnosis.
• Patient-facing explanation. Rewriting information at an appropriate reading level, which is genuinely hard to do well by hand at scale.

Notice that none of these is diagnosis. They are the paperwork that surrounds diagnosis, which is where most clinical time actually goes.

A concrete example#

Structured extraction is the case where the value is easiest to demonstrate. A histopathology report is prose written for another clinician; a research dataset needs fields. Historically that conversion was a research fellow with a spreadsheet.

from anthropic import Anthropic

SCHEMA = {
    "type": "object",
    "properties": {
        "diagnosis":        {"type": "string"},
        "breslow_mm":       {"type": ["number", "null"]},
        "ulceration":       {"type": ["boolean", "null"]},
        "margins_clear":    {"type": ["boolean", "null"]},
    },
    "required": ["diagnosis"],
}

client = Anthropic()
response = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=1024,
    tools=[{
        "name": "record_report",
        "description": "Record the structured findings of a histopathology report.",
        "input_schema": SCHEMA,
    }],
    tool_choice={"type": "tool", "name": "record_report"},
    messages=[{"role": "user", "content": report_text}],
)

fields = response.content[0].input

Forcing the model to answer through a schema rather than in prose is what makes this usable. The output is either valid against the schema or it is not, which gives you something to validate. It does not stop the model being wrong - it stops it being wrong in an unparseable way.

The remaining problem is that a field extracted incorrectly looks exactly like a field extracted correctly. Any pipeline like this needs a human-reviewed sample, and the review burden is the real cost.

What the different tasks demand#

The pattern is that there is no row where the answer is yes. That is not an argument against using these systems - it is an argument about where to put the human, and about being honest that “AI-assisted” means someone is still reading the output.

The useful question is not whether the model is as good as a dermatologist. It is whether the model plus a dermatologist is better than the dermatologist alone, and at what cost in review time.

Regulation is the binding constraint#

In the UK, software that informs diagnosis or treatment of an individual patient is generally a medical device, and falls under MHRA regulation. A scribe that drafts a note the clinician then edits and signs is usually positioned outside that boundary; a system that outputs a triage priority is usually inside it. Vendors are acutely aware of exactly where the line sits, which is a large part of why the deployed products cluster around documentation.

Anyone building in this space should get regulatory advice early, because the classification determines the entire development pathway, not just the paperwork at the end.

What I think happens next#

The near-term value is in removing administrative load, not in replacing diagnostic judgement. That is a less exciting claim than the headlines support, but it is where the deployments that actually persist are concentrated.

The longer-term question is what multimodal models do to the triage of skin lesions in primary care, where the real bottleneck in skin cancer diagnosis sits. That is a genuinely open problem, and unlike the documentation work, it is squarely inside the regulated space.

Dr Magnus Lynch is founder at Epistat Therapeutics. He is a practising dermatologist and Mohs surgeon at Guy's Hospital, and Honorary Reader at King's College London.