Article
DeepSeek Engram × OLMo-core: Distributed Implementation
bird-of-paradise
• • 3
👋 Open to Work
Jen Wei
bird-of-paradise
NingXu24's profile picture
GaborMadarasz's profile picture
CESI-LINEACT-Laboratory2023's profile picture·
AI & ML interests
Training dynamics, optimization & algorithm–hardware co-design
Recent Activity
upvoted a paper 18 days ago
Dion3: Full-Stack Orthogonal Updates upvoted a paper 18 days ago
Weak-to-Strong Generalization via Direct On-Policy Distillation commentedon a paper 18 days ago
Dion3: Full-Stack Orthogonal UpdatesOrganizations
published an article 4 months ago
published an article 5 months ago
Article
The Three Horsemen of Numerical Divergence in Hybrid Models
bird-of-paradise
• • 3 published an article 6 months ago
Article
FlashAttention, Streaming Algorithms, and Numerical Stability in Modern ML Systems
bird-of-paradise
• • 1 published an article 7 months ago
Article
Hopper: The Optimizer That Learns Parallelism 2x Faster Than Adam
bird-of-paradise
• • 4 published an article 7 months ago
Article
Field Notes: Why Muon "Hollows Out" in RL (and What We Plan To DO Next)
bird-of-paradise
• • 6 published an article 8 months ago
Article
Field Notes: The Dilemma of Training Reasoning with Muon
bird-of-paradise
• • 2 published an article 8 months ago
Article
Scaling Is Not Plug-and-Play: What Muon Teaches Us About Optimizers at Scale
bird-of-paradise
• • 2 published an article 9 months ago
Article
Beyond the Wrapper: Building High-Throughput Reasoning Agents with Async Kernels
bird-of-paradise
• • 1 published an article 9 months ago
Article
Reproducing and Validating Distributed Muon 🐢✨: A Practical Verification of Communication Efficiency Claims
bird-of-paradise
• • 2