Instructions to use scenario-labs/ltx25-sdr-to-hdr-modular with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use scenario-labs/ltx25-sdr-to-hdr-modular with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("scenario-labs/ltx25-sdr-to-hdr-modular", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
LTX-2.5 SDR-To-HDR (Modular Diffusers)
Modular Diffusers blocks that run Lightricks'
LTX-2.5 SDR-To-HDR IC-LoRA on a stock diffusers
install. An SDR clip goes in; a scene-linear HDR clip (Rec.709 or ACEScg primaries) or its ACEScct codes come out.
Seam keyframes are on by default, as in Lightricks' reference: at every 24- or 32-frame segment seam of the clip, the
source frame is encoded on its own and held at keyframe_strength (0.95) as a guide, next to an empty keyframe slot
that is generated with the video and then anchors the diffusion decoder.
This repo holds code and a pipeline index only. The weights are loaded from
Lightricks/LTX-2.5-Diffusers (transformer, VAE, diffusion
decoder, scheduler) and the IC-LoRA repo (adapter and precomputed scene embedding). See
huggingface/diffusers#14981 for the discussion.
Requirements
- Access to the gated
Lightricks/LTX-2.5-DiffusersandLightricks/LTX-2.5-22b-IC-LoRA-SDR-To-HDRrepos (accept the LTX-2.x Community License, thenhf auth login). diffuserswith the LTX-2.5 modular pipelines (LTX25ModularPipeline),peftfor the LoRA.trust_remote_code=True: the blocks are defined inblock.pyhere, and the keyframe-aware diffusion decoder inkeyframe_decoder.py.kernels(pip install kernels) for the diffusion decoder's NATTEN attention on GPU.- Optional, for export:
avwith an FFmpeg build that includeslibx265(HLG.mp4),OpenEXR>=3.3(EXR frames).
Usage
import torch
from diffusers import ComponentsManager, ModularPipeline
from diffusers.models.autoencoders.ltx2_diffusion_decoder import LTX2VideoVaeNeighborhoodNattenProcessor
from diffusers.pipelines.ltx2.pipeline_ltx2_ic_lora import LTX2ReferenceCondition
from diffusers.utils import load_video
from diffusers.utils.dynamic_modules_utils import get_class_from_dynamic_module
repo_id = "scenario-labs/ltx25-sdr-to-hdr-modular"
components = ComponentsManager()
components.enable_auto_cpu_offload(device="cuda")
pipe = ModularPipeline.from_pretrained(repo_id, trust_remote_code=True, components_manager=components)
# The VAE and the diffusion decoder run in float32, as in the reference.
# `diffusion_decoder` loads into `LTX25KeyframeDiffusionDecoderModel` (keyframe_decoder.py), see below.
pipe.load_components(dtype={"default": torch.bfloat16, "vae": torch.float32, "diffusion_decoder": torch.float32})
pipe.diffusion_decoder.set_attn_processor(LTX2VideoVaeNeighborhoodNattenProcessor())
pipe.diffusion_decoder.enable_tiling()
pipe.load_lora_weights(
"Lightricks/LTX-2.5-22b-IC-LoRA-SDR-To-HDR",
weight_name="ltx-2.5-22b-ic-lora-sdr-to-hdr-1.0.safetensors",
adapter_name="sdr_to_hdr",
)
pipe.set_adapters("sdr_to_hdr", 1.0)
# An 8-bit sRGB clip of any size; 8k + 1 frames are generated (the largest count the clip covers by default).
sdr_video = load_video("/path/to/sdr.mp4")
frame_rate = 24.0
hdr = pipe(
reference_conditions=LTX2ReferenceCondition(frames=sdr_video),
frame_rate=frame_rate,
output_colorspace="rec709",
generator=torch.Generator("cuda").manual_seed(42),
output="videos",
)[0] # (F, H, W, 3) float32, scene-linear Rec.709
encode_hlg = get_class_from_dynamic_module(
repo_id, "hdr_export.py", class_name="encode_hdr_tensor_to_hlg_mp4", trust_remote_code=True
)
encode_hlg(hdr, "sdr_to_hdr.mp4", frame_rate=frame_rate)
Seam keyframes are on by default (keyframe_strength=0.95). Pass seam_keyframes=False for the plain IC-LoRA
(keyframe_strength=None in the reference; in Modular Diffusers a None input takes the default, so it cannot turn
them off), and high_quality_hdr=True to generate twice the frames and keep every second one.
For an ACEScg EXR sequence, run with output_colorspace="acescg" and use export_to_exr_sequence from
hdr_export.py (exr_colorspace="acescg").
To load from local copies, pass a per-component pretrained_model_name_or_path dict to load_components (add
subfolder={"diffusion_decoder": None} when the decoder folder is not a diffusion_decoder subfolder of a base
snapshot), and point the scene embedding at a local folder with pipe.register_to_config(scene_embedding_repo="/path/to/lora/folder") (or
pass scene_video_context=).
Keyframe-aware diffusion decoder
diffusers does not decode keyframe planes yet (Lightricks is adding it), so keyframe_decoder.py provides an
interim implementation: LTX25KeyframeDiffusionDecoderModel, a subclass of diffusers' LTX2VideoDiffusionDecoderModel
that adds decode(..., keyframe_latents=, keyframe_frame_indices=), a port of the reference's keyframe decode (joint
neighborhood attention included). Without keyframes, decode is the parent's. It also creates the checkpoint's learned
keyframe tag decoder.type_emb, which Lightricks/LTX-2.5-Diffusers/diffusion_decoder holds but the stock class
drops as an unexpected key, so every tensor of the checkpoint is loaded.
The blocks declare diffusion_decoder with that class and the base repo's diffusion_decoder subfolder, and this
repo's modular_model_index.json deliberately has no diffusion_decoder entry, so load_components loads it with the
keyframe-aware class. To load it yourself (e.g. from a local copy):
from diffusers.utils.dynamic_modules_utils import get_class_from_dynamic_module
KeyframeDecoder = get_class_from_dynamic_module(
repo_id, "keyframe_decoder.py", class_name="LTX25KeyframeDiffusionDecoderModel", trust_remote_code=True
)
decoder = KeyframeDecoder.from_pretrained(
"Lightricks/LTX-2.5-Diffusers", subfolder="diffusion_decoder", dtype=torch.float32
)
pipe.update_components(diffusion_decoder=decoder)
With the stock LTX2VideoDiffusionDecoderModel, the pipeline warns and decodes without the keyframe slots; without a
diffusion decoder, the VAE decodes without them. keyframe_decoder.py will be retired once diffusers ships
keyframe-aware decoding.
Validation
Plain IC-LoRA (seam_keyframes=False): on a B200 with the real weights, three clips (720x480 at 8 fps, 1000x560 padded
to 1024x576, and a 60 fps clip for the RoPE cap) give bitwise-identical latents, transformer steps and HDR output to a
classic LTX2HDRPipeline implementation of the same path, given the same input tensor.
Seam keyframes (49 frames with seams at 24 and 48, 97 frames with seams at 32, 64 and 96), high_quality_hdr and the
no-seam fallback (17 frames): on a B200 with the real weights, latents, keyframe slot latents, every transformer step and
the HDR output are bitwise identical to the classic implementation, given the same input tensor. A keyframe decode took
under 32 s per call at 720x480 with a 63 GB peak.
Bitwise reproducibility of the default sRGB input needs the same CPU family: the sRGB to ACEScct primaries conversion runs on the CPU, and its 3x3 matrix product rounds differently across CPU models (about 60 dB PSNR between AMD and Intel hosts).
Inputs
| Name | Default | |
|---|---|---|
reference_conditions |
required | The SDR clip: an LTX2ReferenceCondition, a list of them, or the frames (list of PIL images, (F, H, W, C) array, (F, C, H, W) tensor). |
input_colorspace |
"srgb_gamma" |
srgb_gamma (sRGB-encoded, e.g. 8-bit video), srgb (scene-linear Rec.709), acescg (scene-linear AP1), acescct. |
height, width |
the clip's | Output size. Generation runs at the size rounded up to 32 (reflect-padded); the output is cropped back. |
num_frames |
largest 8k + 1 in the clip |
Number of frames to generate. |
frame_rate |
24.0 |
Frame rate of the clip. RoPE is conditioned on at most 30 fps, as in the reference. |
sigmas / timesteps |
distilled schedule | The 8-step distilled schedule, used verbatim. |
keyframe_strength |
0.95 |
Strength of the seam guides, in [0, 1]. |
seam_keyframes |
True |
False runs the plain IC-LoRA, without seam keyframes. |
high_quality_hdr |
False |
Frame-doubled source, 2N - 1 generated frames, every second frame kept (about twice the cost). |
output_type |
"pt" |
pt, np, or latent (denormalized, padded latents). |
output_colorspace |
"rec709" |
rec709 / acescg (scene-linear, [0, inf)) or acescct (codes in [0, 1]). |
scene_video_context |
from the LoRA repo | Overrides the precomputed scene embedding (config scene_embedding_repo / _filename / _revision). |
generator, num_videos_per_prompt, latents, noise_scale, reference_downscale_factor, attention_kwargs |
As in the LTX-2 modular pipelines. |
Output: videos, a float32 (B, F, H, W, 3) tensor (or array).
Blocks
LTX25SDRToHDRBlocks
input LTX25HDRInputStep size from the clip, pad to 32, 30 fps RoPE cap, distilled sigmas,
seam positions, frame doubling
scene_embedding LTX25HDRSceneEmbeddingStep precomputed scene embedding instead of a text encoder
reference_encoder LTX25HDRReferenceEncoderStep ACEScct + reflect-pad, float32 VAE encode, seam frames encoded alone
seam_keyframes LTX25HDRSeamKeyframesStep appends [guides | slots] after the reference tokens
denoise
prepare_latents LTX2InContextPrepareLatentsStep (diffusers)
keyframe_marker LTX25HDRKeyframeMarkerStep first latent frame and seam slots marked
set_timesteps LTX2ConditionSetTimestepsStep (diffusers)
prepare_audio_latents LTX25HDRPlaceholderAudioStep one zero audio token, no RNG
prepare_coords LTX2ConditionPrepareCoordsStep (diffusers)
denoise LTX2ConditionLoopBeforeDenoiser -> LTX25HDRLoopDenoiser -> LTX2ConditionLoopAfterDenoiser
decode
seam_slots LTX25HDRSeamSlotsStep cuts out and denormalizes the generated seam slots
trim_condition_tokens LTX2TrimConditionTokensStep (diffusers)
video_decode LTX25HDRVideoDecodeStep float32 keyframe-aware diffusion decoder (VAE fallback),
crop, every second frame if frame-doubled, ACEScct -> HDR
The denoiser makes one unguided, video-only transformer pass per step (isolate_modalities=True), so neither a text
encoder, an audio VAE, a vocoder nor guiders are needed.
Limitations
- Seam keyframes need a single reference clip and a transformer with the LTX-2.5 keyframe position embedding; clips shorter than 25 frames have no seam and run the plain IC-LoRA.
- A keyframe decode always uses a pure-PyTorch joint attention (the NATTEN processor only serves plain decodes).
output_type="latent"returns the video latents only (withhigh_quality_hdr, those of the frame-doubled clip).- The scheduler must have
use_dynamic_shifting=False(true forLightricks/LTX-2.5-Diffusers); the blocks raise otherwise. - Output is HDR only (
pt/np/latent); usehdr_export.pyfor HLG.mp4or EXR files.
The model weights are subject to the LTX-2.x Community License; this repo's code is Apache-2.0.
- Downloads last month
- 18
Model tree for scenario-labs/ltx25-sdr-to-hdr-modular
Base model
Lightricks/LTX-2.5