You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

This model and associated code are released under the CC-BY-NC-ND 4.0 license and may only be used for non-commercial, academic research purposes with proper attribution.
Any commercial use, sale, or other monetization of the CytoSyn model and its derivatives, which include models trained on outputs from the CytoSyn model or datasets created from the CytoSyn model, is prohibited and requires prior approval.
By downloading the model, you attest that all information (affiliation, research use) is correct and up-to-date. Downloading the model requires prior registration on Hugging Face and agreeing to the terms of use. By downloading this model, you agree not to distribute, publish or reproduce a copy of the model. If another user within your organization wishes to use the CytoSyn model, they must register as an individual user and agree to comply with the terms of use. Users may not attempt to re-identify the deidentified data used to develop the underlying model.
This model is provided “as-is” without warranties of any kind, express or implied. This model has not been reviewed, certified, or approved by any regulatory body, including but not limited to the FDA (U.S.), EMA (Europe), MHRA (UK), or other medical device authorities. Any application of this model in healthcare or biomedical settings must comply with relevant regulatory requirements and undergo independent validation. Users assume full responsibility for how they use this model and any resulting consequences. The authors, contributors, and distributors disclaim any liability for damages, direct or indirect, resulting from model use. Users are responsible for ensuring compliance with data protection regulations (e.g., GDPR, HIPAA) when using it in research that involves patient data.

CytoSyn: a Foundation Diffusion Model for Histopathology

CytoSyn is a REPA-E [1] diffusion model trained on ~40M tiles, extracted from ~10k TCGA Diagnostic slides, achieving high-quality histopathology image generation at 224×224 resolution. Tech report.

Figure 1: A sample of tiles sampled unconditionally with CytoSyn

The model consists of the following components:

VAE: SD-VAE f8d4 [2],
Latent Diffusion Transformer: SiT-XL/2 [3],
Conditioning: H0-mini [4], a ViT-B/14 distilled from H-optimus-O [5], by Owkin & Bioptimus,
Sampling scheme: Euler-Maruyama SDE sampler [3].

Load REPA-E Model

from diffusers import DiffusionPipeline
import torch

device = torch.device('cuda')

pipeline = DiffusionPipeline.from_pretrained(
    "Owkin-Bioptimus/CytoSyn",
    custom_pipeline="Owkin-Bioptimus/CytoSyn",
    trust_remote_code=True,
    revision="v1" # default
)

pipeline.to(device)

New: you can load the improved CytoSyn-v2 by specifying "v2" for the revision parameter.

Load H0-mini for Conditioning

⚠️ CytoSyn uses H0-mini [CLS]-token for conditional generation. It must be loaded externally and the features must be extracted beforehand.

Note: You can get access to H0-mini on HuggingFace here.

import timm
from timm.data import resolve_data_config
from timm.data.transforms_factory import create_transform

# Load h0_mini encoder
h0_mini = timm.create_model(
    "hf-hub:bioptimus/H0-mini",
    pretrained=True,
    mlp_layer=timm.layers.SwiGLUPacked,
    act_layer=torch.nn.SiLU,
)
h0_mini = h0_mini.to(device)
h0_mini.eval()

# Get preprocessing transform
transform = create_transform(**resolve_data_config(h0_mini.pretrained_cfg, model=h0_mini))

Unconditional Generation

Generate histopathology images without conditioning:

# Generate 4 samples
output = pipeline(
    num_images_per_prompt=4,
    num_inference_steps=250,
    guidance_scale=1.0,  # No guidance for unconditional
)

images = output["images"]

# Save images
for i, img in enumerate(images):
    img.save(f"sample_{i}.png")

Conditional Generation

Generate images conditioned on reference histopathology images:

from PIL import Image

# Load and preprocess conditioning image
conditioning_image = Image.open("reference.png") # 224x224 image
img_tensor = transform(conditioning_image).unsqueeze(0).to(device)

# Extract h0_mini features (CLS token)
with torch.inference_mode():
    h0_mini_embeds = h0_mini(img_tensor)[:, 0]  # [1, 768]

# Generate conditioned samples with classifier-free guidance
output = pipeline(
    h0_mini_embeds=h0_mini_embeds,
    num_images_per_prompt=4,
    num_inference_steps=250,
    guidance_scale=2.5,
    guidance_low=0.0,
    guidance_high=0.75,
)

images = output["images"]

Warning: to achieve best results, the guidance images must be 224x224 images at 0.5 MPP (x20 magnification). If you need to resize an image, use the LANCZOS filter and the PIL resize implementation.

Data

In the Files and versions tab, you can find:

data/training_set_40M.parquet - training set for CytoSyn-v1
data/training_set_108M.parquet - training set for CytoSyn-v2
data/validation_set_in.parquet, data/validation_set_out.parquet - val-in and val-out sets - refer to the paper for more details
data/guidance_training_subset.parquet — subset of the training set used as guidance set for conditional generation
synthetic_samples/cytosyn_v1_uncond_100k.tar.gz, synthetic_samples/cytosyn_v2_uncond_100k.tar.gz - 100k images generated unconditionally with CytoSyn-v1 & v2.

Software Dependencies

torch>=2.0.0
diffusers>=0.35.1
timm>=0.9.0
pillow
huggingface-hub

Additional information

Paper: https://arxiv.org/abs/2603.18089

Summary: blog article

Acknowledgements

Computing Resources

This work was granted access to the High Performance Computing (HPC) resources of Meluxina, from LuxProvide, as part of a Euro-HPC grant under the allocation EHPC-AI-2024A04-020, and to the HPC resources of IDRIS under the allocations 2025-A0181012519 made by GENCI. The results published here are in part based on data and biosamples obtained from the IBD Plexus program of the Crohn’s & Colitis Foundation and in part based upon data generated by the TCGA Research Network: https://www.cancer.gov/tcga.

Code

CytoSyn was built using the REPA-E repository (MIT License).

License

The model is only available to academic and research institutions, for non-commercial use.

Contact

For questions, comments and issues, contact Thomas Duboudin (thomas.duboudin@owkin.com).

References

Leng, X., Singh, J., Hou, Y., Xing, Z., Xie, S., & Zheng, L. (2025). Repa-e: Unlocking vae for end-to-end tuning with latent diffusion transformers. arXiv preprint arXiv:2504.10483.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022). High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 10684-10695). arXiv:2112.10752
Ma, N., Goldstein, M., Albergo, M. S., Boffi, N. M., Vanden-Eijnden, E., & Xie, S. (2024, September). Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers. In European Conference on Computer Vision (pp. 23-40). Cham: Springer Nature Switzerland. arXiv:2401.08740
Filiot, A., Dop, N., Tchita, O., Riou, A., Dubois, R., Peeters, T., ... & Olivier, A. (2025, September). Distilling foundation models for robust and efficient models in digital pathology. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 162-172). Cham: Springer Nature Switzerland. arXiv:2501.16239 | HuggingFace
Saillard, C. and Jenatton, R. and Llinares-López, F. and Mariet, Z. and Cahané, D. and Durand, E. and Vert, J.P. (2024). H-optimus-0. URL: https://github.com/bioptimus/releases/tree/main/models/h-optimus/v0 | HuggingFace

Downloads last month: 112

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Papers for Owkin-Bioptimus/CytoSyn

CytoSyn: a Foundation Diffusion Model for Histopathology -- Tech Report

Paper • 2603.18089 • Published 14 days ago

REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformers

Paper • 2504.10483 • Published Apr 14, 2025 • 22

Distilling foundation models for robust and efficient models in digital pathology

Paper • 2501.16239 • Published Jan 27, 2025 • 1

SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers

Paper • 2401.08740 • Published Jan 16, 2024 • 14

High-Resolution Image Synthesis with Latent Diffusion Models

Paper • 2112.10752 • Published Dec 20, 2021 • 16