You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

This model and associated code are released under the CC-BY-NC-ND 4.0 license and may only be used for non-commercial, academic research purposes with proper attribution. Any commercial use, sale, or other monetization of the MEDIATOR model and its derivatives, which include models trained on outputs from the MEDIATOR model or datasets created from the MEDIATOR model, is prohibited and requires prior approval. Please note that the primary email used to sign up for your Hugging Face account must match your institutional email to receive approval. By downloading the model, you attest that all information (affiliation, research use) is correct and up-to-date. Downloading the model requires prior registration on Hugging Face and agreeing to the terms of use. By downloading this model, you agree not to distribute, publish or reproduce a copy of the model. If another user within your organization wishes to use the MEDIATOR model, they must register as an individual user and agree to comply with the terms of use. Users may not attempt to re-identify any patient data used to develop the underlying model. If you are a commercial entity, please contact the corresponding author.

Tong Ding*, Caiwei Tian*, Dongmin Bang, Ming Y. Lu, Sophia J. Wagner,
Andrew Zhang, Rowland Pettit, Long Phi Le, Faisal Mahmood

_{*Equal contribution · Mass General Brigham AI Institute · Harvard University · Massachusetts Institute of Technology}

Paper · Code · Quickstart · Citation

MEDIATOR maps every clinical concept, whether a diagnosis, medication, lab, procedure, local identifier, or free-text mention, to one of 100,000 discrete interlingua codes in a shared vector-quantized codebook, independent of any specific ontology (ICD, SNOMED CT, RxNorm, LOINC, …). Concepts that mean the same thing land on the same code, so EHR harmonization becomes open-vocabulary tokenization without manual mappings or a fixed vocabulary.

Ontology-agnostic. Any concept string can be coded, whether or not it appears in a particular ontology, so downstream models transfer across hospitals and coding systems. On eICU, closed-vocabulary tokenizers trained on MIMIC-III can use only 13% of events; MEDIATOR uses all of them.
A ready-made vocabulary for sequence models. 100,000 codes with 512-d code vectors can be dropped into a BEHRT-style EHR Transformer as the input embedding vocabulary.
Text-encoder based. Codes are assigned through a frozen Qwen3-Embedding-8B encoder, with no manual mappings and no string matching.

Note: This release focuses on using the trained MEDIATOR checkpoint.

Model details


Developed by	Mass General Brigham AI Institute
Model type	Vector-quantized interlingua: frozen Qwen3-Embedding-8B text encoder with a trained MLP adapter (4096 → 1024 → 512) and a 100,000 × 512 cosine-assignment codebook
Parameters	126.6M in the checkpoint: codebook 102.5M (incl. EMA buffers), reconstruction decoder 18.9M, adapter 4.7M, contrast heads 0.5M. The inference path uses 55.9M (adapter + codebook).
Input	Clinical concept strings, embedded with Qwen3-Embedding-8B into 4096-d vectors; or precomputed Qwen3-Embedding-8B embeddings
Output	A discrete code id in [0, 100000) per concept; 512-d code vectors
Training	100 epochs on UMLS synonym groups with contrastive, soft-assignment agreement, reconstruction, PPMI-graph and description-teacher objectives. Only the adapter, codebook and heads are trained; the text encoder stays frozen. See `training_config.yaml`.
Base model	Qwen/Qwen3-Embedding-8B
Paper	Discrete Neural Interlingua for Ontology-Agnostic EHR Modeling, NeurIPS 2026
Repository	github.com/mahmoodlab/MEDIATOR
License	CC-BY-NC-ND 4.0

From clinical concepts to interlingua codes

MEDIATOR is trained on UMLS synonym groups: strings that name the same concept are pulled onto the same code by an agreement loss, while contrastive objectives and medical knowledge teachers (enriched concept descriptions, code co-occurrence, and relation graphs) keep clinically related but distinct concepts distinguishable. At inference, a concept string is embedded by the frozen text encoder, mapped to the interlingua space by the adapter, and assigned a code by hard cosine-argmax against the codebook.

MEDIATOR animation: clinical strings from different ontologies and free text are mapped to shared interlingua codes; a frozen text encoder, a projection and a 100,000-code VQ codebook, trained with contrastive equivalence, assignment agreement and medical knowledge signals

_{Any clinical string lands on a shared interlingua code. Synonyms merge; medical knowledge arranges distinct concepts.}

Used as the token interface of an EHR Transformer, MEDIATOR improves mean AUPRC over the strongest prior tokenizer by 4.0 points on 20 MIMIC-III tasks and by 8.6 points in zero-shot transfer to 14 eICU tasks. See the paper for the full results.

Requesting access

Access requires agreeing to the terms of use in the form on this page, with the primary email of your Hugging Face account matching your institutional email. Requests from accounts whose primary email is personal (@gmail/@hotmail/@qq) are denied. To fix this, (1) add your official institutional email to your Hugging Face account and verify it, and (2) set it as your primary email. Other common reasons for denial are mistakes in the form, for example a full name with abbreviations, an affiliation that is not spelled out, an insufficient description of the research use, or an unrecognized email domain.

Installation & access

The model is gated. Request access with the form on this page (see Requesting access), then install the inference code from the GitHub repository and log in:

git clone https://github.com/mahmoodlab/MEDIATOR.git && cd MEDIATOR
pip install -r requirements.txt
hf auth login

Loading the model

import torch
from mediator import MediatorVQ

model = MediatorVQ.from_pretrained("MahmoodLab/MEDIATOR", device="cuda")   # or a local directory

The model has two inference components:

Component	Role
Adapter	MLP 4096 → 1024 → 512 (GELU + LayerNorm). Maps a text-encoder embedding into the interlingua space.
Codebook	100,000 × 512 vectors. Hard cosine-argmax assignment; the winning code id is the concept's interlingua code.

Important: embeddings must follow the Qwen3-Embedding-8B instruct recipe (task instruction, lowercased input, left padding, last-token pooling, L2 normalization) used at training time. mediator.embeddings.QwenEmbedder implements it; ad-hoc pooling or prompts will not map to same meaningful codes.

Quantizing clinical terms

from mediator.embeddings import QwenEmbedder

embedder = QwenEmbedder(model_name="Qwen/Qwen3-Embedding-8B", device="cuda")

terms = [
    "Type 2 Diabetes Mellitus",
    "Diabetes Mellitus, Non-Insulin-Dependent",   # synonym -> same code
    "Myocardial Infarction",
    "Heart Attack",                                # synonym -> same code
    "Breast Cancer",
]

with torch.inference_mode():
    embeddings = embedder.embed(terms)                 # (5, 4096) float32
    code_ids = model.quantize(embeddings.to("cuda"))   # (5,) interlingua code ids

"Type 2 Diabetes Mellitus" and "Diabetes Mellitus, Non-Insulin-Dependent" receive the same code; so do "Myocardial Infarction" and "Heart Attack".

Using precomputed Qwen3-Embedding-8B embeddings

If you already have Qwen3-Embedding-8B embeddings (4096-d, L2-normalized, computed with the instruct recipe), skip the embedder:

with torch.inference_mode():
    code_ids = model.quantize(embeddings.to("cuda"))   # (B, 4096) -> (B,) code ids
    codes = model.code_embeddings                       # (100000, 512) codebook vectors

Preparing a hospital vocabulary for a downstream model

To train a downstream model over interlingua codes (as done in the paper), map your vocabulary to code ids and export the codebook vectors with prepare_vocab.py:

python prepare_vocab.py \
    --ckpt MahmoodLab/MEDIATOR \
    --input_csv your_vocab.csv \
    --term_col description --label_col token_id \
    --output_dir out_dir

File	Contents
`out_dir/codebook_emb.jsonl`	One 512-d vector per code, 1-indexed, one JSON object per line. Use it to initialize the downstream model's token embedding layer.
`out_dir/with_vq_indices.csv`	Your vocabulary plus a `vq_indices` column with the assigned code id per term.

Intended use and limitations

Intended use: non-commercial academic research on EHR modeling, such as converting hospital vocabularies to a shared discrete vocabulary, training BEHRT-style sequence models over interlingua codes, and measuring cross-ontology concept equivalence.
Out of scope: clinical diagnosis or any other clinical decision-making, and commercial use.
Coverage: codes are learned from UMLS; strings far outside clinical language may map unreliably. MEDIATOR is trained on English sources only.
No context: each concept description is tokenized on its own, so ambiguous abbreviations with several expansions cannot be disambiguated.
Partial synonym agreement: not every UMLS synonym group collapses onto a single code; agreement is high but not perfect (see the paper for the quantitative analysis).

License

The model weights are released under CC-BY-NC-ND 4.0 for non-commercial academic research only. The model is not validated for clinical use.

Citation

If you use MEDIATOR in your research, please cite:

@misc{ding2026discreteneuralinterlinguaontologyagnostic,
      title={Discrete Neural Interlingua for Ontology-Agnostic EHR Modeling}, 
      author={Tong Ding and Caiwei Tian and Dongmin Bang and Ming Y. Lu and Sophia J. Wagner and Andrew Zhang and Rowland Pettit and Long Phi Le and Faisal Mahmood},
      year={2026},
      eprint={2610.27047},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2610.27047}, 
}

Questions

Please open a GitHub issue for questions about using MEDIATOR.

Downloads last month: 5

Model tree for MahmoodLab/MEDIATOR

Base model

Qwen/Qwen3-8B-Base

Finetuned

Qwen/Qwen3-Embedding-8B

Finetuned

(48)

this model

Paper for MahmoodLab/MEDIATOR

Discrete Neural Interlingua for Ontology-Agnostic EHR Modeling

Paper • 2610.27047 • Published 11 days ago