Deciphering clinical abbreviations with a privacy protecting machine learning system

The paper addresses the challenge that clinical notes contain numerous abbreviations and shorthand that hinder comprehension by patients and even clinicians from other specialties. With regulatory mandates increasing patient access to notes, improving understandability is important. Prior studies show patients misunderstand many abbreviations, and misinterpretations can cause harm. Simply avoiding abbreviations adds burden and does not help with existing notes. The research question is whether a privacy-preserving machine learning system can contextually detect and expand a broad set of clinical abbreviations in free text at expert-level accuracy, without using sensitive clinical training data. The authors propose reframing abbreviation deciphering as a text-to-text translation problem and explore training solely on public web data with reverse substitution, then transferring to clinical notes.

Abbreviation detection and expansion have been studied via heuristic string matching and various machine learning methods including Naive Bayes, SVMs, profile-based approaches, hyperdimensional computing, CNNs, LSTMs, transformer encoders (e.g., clinicalBERT), latent meaning cells, and decoder-based transformers. Many prior systems build one expansion model per abbreviation and often assume prior detection of the abbreviation span, limiting scalability. Heuristic detection (e.g., string matching) fails for tokens overlapping with common English words (e.g., “us” as ultrasound vs pronoun). Large-scale creation of labeled data has relied on de-identified clinical notes with reverse substitution, raising privacy concerns and generalization issues. Prior state-of-the-art systems separate detection from disambiguation, and expansion models are often abbreviation-specific. The authors identify three key challenges: absence of a corpus of paired abbreviated and expanded clinical text, reliance on de-identified clinical data with privacy risks and logistical hurdles (e.g., need for federated learning infrastructure), and complexity from multi-model pipelines. They position their work to address these gaps with public web-based data generation, a single end-to-end model that unifies detection and expansion, and inference techniques to bridge web-to-clinical domain shift.

The task is formulated as translation: input snippets may or may not contain abbreviations; the model outputs the same snippet with all abbreviations expanded. A single encoder-decoder transformer (T5 family) performs detection and expansion jointly via self-attention. Data generation (WSRS): The authors curate an abbreviation-expansion dictionary (3,758 unique abbreviations; 5,794 pairs) combining multiple sources (Vanderbilt, sign-out notes, BIDMC, Wikipedia, plus manual additions) with prioritized manual quality review (>550 edits). They apply reverse substitution to public health-related web pages from a medically filtered web crawl (med-crawl, MC), creating MC-WSRS: snippets (1–3 sentences) where long forms in the dictionary are probabilistically replaced by abbreviations. To address frequency imbalance and include negative (no-abbreviation) examples, they design Web-Scale Reverse Substitution (WSRS), a distributed algorithm that down-samples frequent pairs, up-samples rare ones using shard-level counters and power-law keep probabilities, substitutes long forms with 95% probability when kept, filters very short outputs, and retains a small proportion of non-substituted snippets via a separate keep probability. Snippets are grouped by rarest substitution and capped at N=1000 per group. Hyperparameters: a_rs=1.0, a_nors=1.5, resulting datasets ~5M (MC-WSRS) and ~3M (C4-WSRS) examples. Model: T5 variants (small 60M, large 770M, 11B, and 80B). Pre-training: masked language modeling on web corpora (MT5 vocabulary with 250k wordpieces, multilingual, byte fallback). Fine-tuning: input prefixed with “expand abbreviations:” and contains abbreviated snippet; target is original unmodified snippet. Optimization: Adafactor (decay 0.8), LR 1e-3 with 1000-step warmup, batch size 64, dropout 0.1, 200k steps. Inference: Beam size 2. Three procedures: (1) Standard (single pass); (2) Iterative (feed output back until stable); (3) Elicitive inference (iterative with beam search; continue feeding back top beam until stable and evaluate second beam; if second-beam log-likelihood above threshold, continue with it; stop when top beam stable and second-beam below threshold). Threshold tuned via grid search on held-out synthetic data; optimal ranges reported for smaller models; 80B used elicitive inference. Evaluation datasets: (1) 302 synthetic clinician-authored snippets with ground-truth expansions; (2) CASI (University of Minnesota): after filtering, 21,514 anonymized snippets covering 64 abbreviations and 123 pairs (single labeled target per snippet; various corrections/exclusions applied); (3) MIMIC-III-derived test set: 6,544 snippets from 59,652 discharge notes with reverse-substituted labels (3,872 unique pairs; 16,872 labeled abbreviations); (4) i2b2-2014-derived test set: 2,888 snippets (1,913 unique pairs; 5,087 labeled abbreviations). Additionally, native (non-synthetic) abbreviations sampled from MIMIC-III and i2b2-2014 were physician-graded for expansion accuracy. Metrics and alignment: They align input-output sequences using a token-level Needleman-Wunsch variant with custom scores (match/mismatch/indel/SFC/EG = 0/1/1/0.5/0.6) to attribute expansions to abbreviations. Metrics: Detection Recall (DR), Detection Precision (DP), Expansion Accuracy (EA; clinically equivalent accepted), and Total Accuracy (TA=DR*EA). Confidence intervals via 400 bootstrap samples. Human evaluation: 30 synthetic snippets (≥3 abbreviations each) translated by laypeople (with/without Google), medical students, and attending internists; instructions: expand only if reasonably confident; lay-with-Google allowed 2-minute searches.

• Model scaling and inference: On synthetic snippets, elicitive inference substantially boosts performance; with elicitive inference, larger models achieve higher total accuracy, with T5 80B reaching ~97.0% total accuracy. Detection precision on synthetic is 99.3%.
• External clinical datasets (T5 80B + elicitive inference): • Synthetic: DR 99.1%, EA 97.9%, TA 97.0%. • CASI: DR 96.8%, EA 95.1%, TA 92.1%. • MIMIC-III (RS-derived): DR 99.5%, EA 96.1%, TA 95.7%. • i2b2-2014 (RS-derived): DR 99.7%, EA 96.7%, TA 96.5%. Ambiguous vs unambiguous: detection recall comparable; expansion accuracy >99% for unambiguous across datasets. Performance independent of abbreviation rarity (MIMIC-III stratification).
• Native abbreviations (not synthetically injected): physician review showed EA 95.5% (147/154) on MIMIC-III and 97.4% (150/154) on i2b2-2014.
• Abbreviations that are also English words: Of 180 such abbreviations, 44 appeared in 550 instances in synthetic data; 6.9% (38/550) were true abbreviations. The model correctly left English-word usages unexpanded 99.2% (508/512); all 38 intended abbreviations were correctly expanded.
• Human vs model comparison (30-sample synthetic subset): • Laypeople (no Google): DR 34.9%, EA 82.8%, TA 28.6%. • Laypeople with Google: DR 82.9%, EA 90.0%, TA 74.5%. • Medical students: DR 91.3%, EA 97.1%, TA 88.7%. • Attending physicians: DR 92.4%, EA 96.0%, TA 88.7%. • T5 80B + elicitive: DR 98.4%, DP 100%, EA 99.2%, TA 97.6%.
• Data quality: Models fine-tuned on C4-WSRS (public) show slightly lower EA and, in CASI, lower DR compared to MC-WSRS; elicitive inference mitigates DR gaps. Medical enrichment of web crawls improves coverage and performance. Overall, the single end-to-end model trained solely on public web data attains state-of-the-art accuracy across multiple clinical datasets, matching or exceeding physician performance, with high precision and robust handling of ambiguous cases and English-word overlaps.

The findings demonstrate that comprehensive abbreviation deciphering can be cast as translation and achieved by a single end-to-end model trained without sensitive clinical data. Using WSRS on public medical web text creates large-scale supervised signals while protecting privacy. Elicitive inference effectively overcomes domain shift from web to clinical notes, enabling high recall and accuracy without additional clinical fine-tuning. Compared with prior work, the system unifies detection and expansion, scales to thousands of abbreviations (ambiguous and unambiguous), and attains superior detection recall and expansion accuracy on external datasets. It also handles context-sensitive cases where abbreviations overlap with common English words and where the same abbreviation recurs with different senses within a snippet. Clinically, this capability can bridge comprehension gaps for patients and support clinicians by standardizing language. The approach reduces pipeline complexity (single model/objective) and shows that fine-tuning large seq2seq models with carefully constructed synthetic supervision can provide strong constraints on outputs, maintaining near-perfect precision. Remaining gaps relate to computational cost of elicitive inference, generalization to broader language models, human factors variability, and ensuring expanded terms are understandable to diverse audiences.

The study introduces a privacy-protecting, end-to-end translation system that detects and expands thousands of clinical abbreviations using only public web data for training. Through WSRS and elicitive inference, the T5-based model achieves state-of-the-art performance across multiple clinical datasets and matches or exceeds physician accuracy. This work shows that large-scale synthetic supervision can yield expert-level clinical NLP without accessing sensitive notes. Future work includes improving single-pass detection to reduce inference cost, evaluating and comparing alternative large language model architectures (e.g., decoder-only LLMs), expanding resources for rare diseases, addressing potential biases and insensitive terminology, and solving the “last mile” for patient comprehension by coupling expansion with simplification and patient-centered explanations.

Key limitations include: (1) elicitive inference increases computational cost due to multiple inference rounds; (2) no direct comparisons were made to other high-performing LLM architectures (e.g., decoder-only models); (3) human benchmark performance may vary with literacy, specialty, and other factors, limiting generalizability of the human comparison; (4) generative outputs carry risks, though observed precision was near 100% and expansions adhered to the curated dictionary; (5) even correctly expanded clinical terms may remain difficult for patients to understand; (6) the clinical impact of expansion errors, especially for rare diseases, is unknown due to lack of suitable datasets; (7) some abbreviations and clinical language may encode bias or insensitive terminology, requiring further fairness-oriented mitigation.