EllipsesMark/minimax-h3-vr180-sbs-lora

MiniMax H3 · VR180 stereoscopic side-by-side LoRA

A LoRA for MiniMax H3 (Hailuo 3.0) that makes the model shoot VR180 stereo video: one frame holds the left-eye view on the left and the right-eye view on the right, each a 180°×180° half-equirectangular hemisphere, 24 fps, with H3's native audio. Un-squeeze the 21:9 output to 2:1, tag it with spherical metadata, and it plays in a Quest / DeoVR / Skybox as an immersive 3D-180 clip.

Same seed per prompt. s0 = LoRA scale 0 (base model), s1 = LoRA scale 1. All four s1 outputs are left/right pairs; near objects carry 3–7 px of horizontal disparity per 768-px eye, the far field ~0, matching real VR180 footage.

Use it on fal

POST https://queue.fal.run/minimax/h3/text-to-video/lora
{
  "prompt": "vr180sbs Stereoscopic VR180 video shown side by side: the left half is the left-eye view and the right half is the right-eye view of the same 180-degree scene, nearly identical with a slight horizontal offset. A quiet beach at golden hour, gentle waves rolling in, a child building a sandcastle a few meters away, seagulls calling",
  "loras": [{"path": "rehan-fal/minimax-h3-vr180-sbs-lora", "scale": 1.0}],
  "aspect_ratio": "21:9",
  "resolution": "768P",
  "duration": 5,
  "prompt_expansion_mode": "disabled"
}

• Trigger: vr180sbs at the start of the prompt, followed by the layout sentence above (it was in every training caption). Then describe the scene normally, including the sound.
• Geometry: generate at 21:9; the LoRA was trained on 2:1 side-by-side frames fitted into H3's 21:9 bucket (each eye 1.17:1). Render at 768P (native). 2K/4K on fal are SeedVR upscales of the 768P pass and were not evaluated for stereo consistency.
• Scale: 1.0. Lower values fade back toward a mono wide-angle look before losing the split.
• path accepts this repo id directly (or a direct .safetensors URL). Longer clips (up to 15 s) and the 2K/4K upscale paths keep the stereo pair intact; the 8-step minimax/h3-max LoRA route degrades it, so stay on minimax/h3.

Make it headset-ready

# 1536x672 (21:9) -> 1536x768 so each eye is square, then Spherical V2 stereo metadata (180° = crop 90° per side)
ffmpeg -i out.mp4 -vf "scale=1536:768:flags=lanczos,setsar=1" -c:v libx264 -crf 16 -pix_fmt yuv420p -c:a copy tmp.mp4
python spatialmedia -i --v2 -s left-right -p equirectangular -b 0:0:1073741824:1073741824 tmp.mp4 out_LR_180.mp4

(spatialmedia is Google's spatial-media tool.) Two finished samples are in samples/.

Training

Trained on fal's hosted trainer (minimax/h3/t2v/trainer, the fal-ai/minimax-h3-trainer app): PEFT LoRA, rank 32 (alpha = rank), on the qkv_proj and out_proj attention projections of all 50 DiT blocks plus the 2 text token-refiner blocks, rectified-flow loss on video and audio jointly, AdamW 2e-4 with linear decay, batch 1, 2500 steps (~3.9 h on one H200, US$12.50).

Bucket	896×384×124 frames (fal resolution=medium, aspect_ratio=21:9), 24 fps
Data	50 clips × 5.3 s from 27 real VR180 YouTube videos (first-person walks, nature, theme-park shows, trade-show vlogs), selected through Google's Stereo4D index of validated stereo footage
Prep	full side-by-side frame scaled exactly to the bucket (fal's cover+center-crop becomes a no-op), exact 24 fps CFR, 32 kHz stereo audio kept, letterbox bars removed, L/R correlation and aspect sanity checks
Captions	Gemini 3.5 Flash Lite via fal (openrouter/router/video, sees audio), one paragraph per clip describing the scene once; trigger via trigger_phrase; constant layout sentence prepended to every caption
Pilots	1000 steps at 21:9 → stereo pair on 1/4 prompts; 1000 steps at 16:9 (768×448) → 0/4. Production = 21:9 + layout captions + 2500 steps → 4/4

Full pipeline (download → prep → caption → zip → train → A/B eval → VR packaging) and the design notes live in the training/ folder and the source repo.

Known limits

• Over-emphasised dark "porthole" vignettes at the outer edge of each eye on some prompts (inherited from camera-native VR180 footage).
• One source channel supplies 20 of the 50 clips; expect a bias toward first-person outdoor walks.
• Text and faces are as good as base H3, no better. Fast pans can drift between eyes.
• Evaluated on 4 prompts × 1 seed; put on a headset before trusting it.

Licence and data notice

Weights derive from MiniMax H3 and are subject to the MiniMax Community License (attribution and territory/revenue clauses apply). Training clips were downloaded from YouTube under YouTube's standard licence for research; they are not redistributed here. The adapter learns a frame layout and projection, not the content of those videos.