LLM skin cancer test - data and code
Dr Magnus Lynch, October 2026. Article: https://www.drmagnuslynch.com/can-llms-diagnose-skin-cancer

cases.csv       The 200 test lesions: case number, class (MEL, BCC, SCC, ACK, NEV, SEK), whether the
                lesion is safe to discharge, the full histological diagnosis, and the ISIC ids of the
                clinical and dermoscopic images. The images are not included; they are in MILK10k
                (https://doi.org/10.34970/648456, MILK study team, CC BY-NC).
prompts.txt     The three prompts, exactly as sent. Clinical image first, dermoscopic image second. The
                third prompt was sent with the clinical image only.
results/        One JSON line per case per run:
                  *-v2.jsonl               clinician prompt (diagnosis, confidence, discharge decision)
                  *-patient.jsonl          patient prompt (free-text reply)
                  *-patient-graded.jsonl   the patient reply as coded by GPT-6.1 Sol, blind to the diagnosis
                  *-patient-clinical.jsonl and *-patient-clinical-graded.jsonl
                                           the same, for the patient prompt sent with the clinical image only
scripts/        make_selection.py and make_selection_200.py drew the cases; classify.py ran the models;
                grade_patient.py coded the patient replies; analyse_clinician.py and analyse_patient.py
                scored them. They expect the layout used on the author's machine and are provided as a record.

Models: GPT-6 Astra (OpenAI API) and Claude Fable 5.1 (via OpenRouter). Default settings, no system
prompt, no cap on reply length, one run per case.
