Instructions to use remyxai/appearance-transfer-flux-modular with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use remyxai/appearance-transfer-flux-modular with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("remyxai/appearance-transfer-flux-modular", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
Appearance Transfer for FLUX β training-free reference appearance/texture transfer (Modular Diffusers custom block)
π€ Hub: remyxai/appearance-transfer-flux-modular Β· π Paper: arXiv:2603.26767 Β· π¦ Monorepo: flux-recipes
Transfer a reference image's appearance (color, texture, material) onto a source image while preserving the source's geometry β no fine-tuning, no new weights β as a Modular Diffusers custom block. Clean-room implementation of "A Training-Free Framework for High-Fidelity Appearance Transfer via Diffusion Transformers" (arXiv:2603.26767), built on FLUX.1-dev-Depth + a mask-weighted FLUX.1-Redux. Unlike our regional-prompting (soft prompt routing) or stitch (bounding-box placement) blocks, this is reference-driven material/texture transfer.
Usage
import torch
from diffusers import ModularPipeline
from PIL import Image
pipe = ModularPipeline.from_pretrained("remyxai/appearance-transfer-flux-modular", trust_remote_code=True)
pipe.load_components(dtype=torch.bfloat16)
pipe.to("cuda") # ~36GB resident; prefer an 80GB A100 (this block runs ~170 transformer passes)
img = pipe(
source_image=Image.open("source.png"), # geometry to keep
reference_image=Image.open("reference.png"), # appearance to transfer
reference_mask=None, # optional HxW [0,1] fg mask -> texture, not shape
blend_k=0.25, # lower = more appearance, higher = more structure
height=1024, width=1024,
output="images",
).images[0]
img.save("appearance_transfer.png")
Pass reference_image=None and the block is a bit-exact identity (returns the source unchanged).
No text prompt is required β appearance comes from the reference.
How it works
Three training-free ingredients on frozen FLUX.1-Depth (all validated on an A100):
- Source inversion (structure prior). The source is inverted along the FLUX.1-Depth flow with a
matched 2nd-order (RK2/midpoint) solver at guidance 1.0, giving a content-rich trajectory. A
blended initialization replays this trajectory for the first
blend_kfraction of steps, locking the source geometry; depth control anchors it every step. - Mask-weighted Redux (appearance). The reference is encoded through the FLUX.1 Redux image encoder; its SigLIP patch tokens are down-weighted by the reference foreground mask to suppress the reference's shape and keep its texture. This global embedding conditions generation throughout.
- Attention context expansion (fine detail). The reference's image-token Keys/Values are captured and concatenated onto the source's at the first-2 / last-2 blocks of both FLUX streams, so each source patch attends cross-image to the reference's appearance library. Query tokens are unchanged; capture is image-token-only so it composes with the Redux-extended conditioning.
The attention processors are swapped for the call and restored in finally; with reference_image
empty none are installed.
Validated (e2e, 50 steps): structure preserved (sourceβresult depth-map correlation 0.82β0.98) with
appearance moved toward the reference (+0.12 to +0.35 CLIP-to-reference gain). blend_k=0.25 balances
the two; strong-structure subjects tolerate 0.2 (more transfer), delicate close-ups prefer 0.3β0.4.
Compute: two RK2 inversions + generation β 170 transformer passes (2β3Γ a stock FLUX.1-Depth run).
Comfortable on an 80GB A100 with all components resident; CPU offload thrashes at this call count.
Key parameters
| arg | default | meaning |
|---|---|---|
source_image |
β | image whose geometry is preserved (required) |
reference_image |
None | image whose appearance is transferred; None β identity (returns source) |
reference_mask |
None | HxW mask in [0,1]; down-weights background reference patches (texture, not shape) |
blend_k |
0.25 | fraction of steps replaying the source (structure lock); lower = more appearance, higher = more structure |
num_inference_steps |
50 | total steps (matched RK2 both directions) |
guidance_scale |
10.0 | FLUX.1-Depth generation guidance |
invert_guidance |
1.0 | RF-inversion guidance (best reconstruction) |
use_kv_injection |
True | attention appearance channel (secondary; fine detail) |
redux_mask_floor |
0.1 | background patch weight floor for mask-weighted Redux |
height / width |
1024 | canvas size |
Dependencies
diffusers (main / β₯ 0.41), transformers, accelerate, sentencepiece, protobuf. A depth estimator
(depth-anything/Depth-Anything-V2-Small-hf) is lazily loaded for the source depth. Components:
FLUX.1-Depth-dev (transformer/vae/text-encoders/scheduler) + FLUX.1-Redux-dev (image encoder +
embedder). No new weights are trained. Prompt β source/reference is up to the caller (like a
reference-image API).
Attribution & AI assistance
Method by Shengrong Gu, Ye Wang, Song Wu, Rui Ma, Qian Wang, Lanjun Wang, and Zili Yi (arXiv:2603.26767). Clean-room implementation: no reference code was released, so this block was written from the paper's description alone (same discipline as our other FLUX ports; the method itself is not copyrightable). Authored with AI assistance (Claude) and validated by the Remyx AI team; method credit to the authors above. Uses FLUX.1-dev-Depth and FLUX.1-Redux-dev under their non-commercial license β this derivative inherits that restriction.
Citation
@misc{gu2026appearancetransfer,
title={A Training-Free Framework for High-Fidelity Appearance Transfer via Diffusion Transformers},
author={Gu, Shengrong and Wang, Ye and Wu, Song and Ma, Rui and Wang, Qian and Wang, Lanjun and Yi, Zili},
year={2026}, eprint={2603.26767}, archivePrefix={arXiv}, primaryClass={cs.CV},
url={https://arxiv.org/abs/2603.26767}
}
- Downloads last month
- 9
