Request access to The QUEST models

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

The QUEST models is released under CC BY-NC-ND 4.0 and may only be used for non-commercial research with proper attribution. Any commercial use, sale or other monetisation of it, or of models or datasets derived from it, is prohibited and requires prior approval. The primary email of your Hugging Face account must match your institutional email: requests from free providers (@gmail, @outlook, @hotmail, @qq, @163, @foxmail and the like) are denied. By downloading you attest that all information given below is correct and up-to-date, and you agree not to distribute, publish, redistribute or reproduce a copy in whole or in part. If another user in your organisation wishes to use it, they must register as an individual user and agree to these terms. Commercial entities should contact the corresponding author.

QUEST

QUEry-based virtual STaining: multiplex immunofluorescence predicted from H&E, with dynamic output panel. The model reads a frozen encoding of the H&E plus a semantic embedding of each marker name.

Quick start

import matplotlib.pyplot as plt
from quest import QuestGenerator

he = plt.imread("assets/example_he.png")[..., :3]  # (224, 224, 3)
mif = QuestGenerator("quest-semantic")([he], ["PanCK", "CD68", "DAPI"])

Tutorials

We prepared tutorials from virtual staining to downstream applications.

Tutorial	What for
📦 00_data	what to download, where it goes, and the format of each cohort
🔬 01_predict_mif	predict a panel from one H&E patch, and score it against the real MIF
🧫 02_cell_typing	cell types from a virtual stained expression
📐 03_distance_entropy	cell-cell distance and neighbourhood entropy from based on cell typing
🗺️ 04_cn_annotation	cell states and cellular neighbourhoods discovered from H&E alone
🔎 05_retrieval	search a stained archive with an H&E query
📈 06_patient_aggregation	patches to a patient survival prediction, and which patch contribution analyses

Set the two constants at the top of tutorials/_common.py, get the data with Tutorial 0, then explore them step-by-step.

Weights

repo
yandrewl/QUEST	three QUEST models, cell typer, the Eva marker table
yandrewl/Eva	`Eva_model.ckpt`, the MIF foundation encoder the retrieval benchmark embeds in
MahmoodLab/UNI2-h	the frozen H&E encoder

model
`quest-semantic`	marker queries from a semantic embedding of the marker's name
`quest-id`	the same architecture with a fixed learned per-marker table instead
`quest-eva`	a masked autoencoder that inpaints the marker channels of a partly observed stack

Check quest.zoo for detailed description of model architectures and configurations.

Data quick start

huggingface-cli login first.

import shutil
from huggingface_hub import snapshot_download, hf_hub_download

snapshot_download("yandrewl/QUEST", local_dir="checkpoints",             # weights, 1.5 GB
                  allow_patterns=["*.ckpt", "*.npz"])

snapshot_download("yandrewl/QUEST-tutorial-data", repo_type="dataset",   # cohorts, 1.7 GB
                  local_dir="data",
                  allow_patterns=["crc-metu/*",      # 73 MB   tutorial 6
                                  "bog-86337/*",     # 36 MB   tutorial 4
                                  "stanford-pc/*"])  # 1.6 GB  tutorials 1, 5

shutil.copy(hf_hub_download("yandrewl/Eva", "Eva_model.ckpt"),           # tutorial 5
            "checkpoints/Eva_model.ckpt")

from questkit import cohort                                             # tutorials 2, 3
cohort.fetch_pathocell(["reg016_B", "reg032_B"])                        # ~350 MB / region

QUEST_DATA points elsewhere if you keep the cohorts outside the repo. Tutorial 0 has the file formats.

License

Code, weights and staged cohorts are released under CC BY-NC-ND 4.0. See LICENSE.

Downloads last month: 5