Towards Looped Models Done Right

IFM 's Collections

K2 Horizon

K2 Horizon Quantization

K2 Horizon GGUF

Arabic-Benchmark-Auditing

Towards Looped Models Done Right

Towards Looped Models Done Right

updated 21 days ago

Looped language model checkpoints from Part I and Part II (Rethinking at Fixed Points). Native xLLM format; loading requires the xLLM code.

Upvote

IFM/LoopedLM-P1-dense-d112-336b

Text Generation • 4B • Updated Oct 2 • 298

Note dense_d112 · 336B tokens
IFM/LoopedLM-P1-dense-d112-500b

Text Generation • 4B • Updated Oct 2 • 325

Note dense_d112 · 500B tokens
IFM/LoopedLM-P1-dense-d28-336b

Text Generation • 1B • Updated Oct 2 • 306

Note dense_d28 · 336B tokens
IFM/LoopedLM-P1-dense-huginn-336b

Text Generation • 1B • Updated Oct 2 • 239

Note dense_huginn · 336B tokens
IFM/LoopedLM-P1-dense-huginn-500b

Text Generation • 1B • Updated Oct 2 • 244

Note dense_huginn · 500B tokens
IFM/LoopedLM-P1-dense-middle-336b

Text Generation • 1B • Updated Oct 2 • 331

Note dense_middle · 336B tokens
IFM/LoopedLM-P1-dense-middle-prelude-injection-336b

Text Generation • 1B • Updated Oct 2 • 244

Note dense_middle_prelude_injection · 336B tokens
IFM/LoopedLM-P1-dense-ouro-336b

Text Generation • 1B • Updated Oct 2 • 327

Note dense_ouro · 336B tokens
IFM/LoopedLM-P1-dense-ouro-raw-injection-336b

Text Generation • 1B • Updated Oct 2 • 232

Note dense_ouro_raw_injection · 336B tokens
IFM/LoopedLM-P1-dense-shared-hl-336b

Text Generation • 1B • Updated Oct 2 • 166

Note dense_shared_hl · 336B tokens
IFM/LoopedLM-P1-dense-split-hl-336b

Text Generation • 1B • Updated Oct 2 • 246

Note dense_split_hl · 336B tokens
IFM/LoopedLM-P1-moe-ff112-336b

Text Generation • 33B • Updated Oct 2 • 262

Note moe_ff112 · 336B tokens
IFM/LoopedLM-P1-moe-ff112-500b

Text Generation • 33B • Updated Oct 2 • 387

Note moe_ff112 · 500B tokens
IFM/LoopedLM-P1-moe-ff112-prelude-injection-336b

Text Generation • 33B • Updated Oct 2 • 276

Note moe_ff112_prelude_injection · 336B tokens
IFM/LoopedLM-P1-moe-huginn-336b

Text Generation • 9B • Updated Oct 2 • 196

Note moe_huginn · 336B tokens
IFM/LoopedLM-P1-moe-huginn-500b

Text Generation • 9B • Updated Oct 2 • 263

Note moe_huginn · 500B tokens
IFM/LoopedLM-P1-moe-middle-336b

Text Generation • 9B • Updated Oct 2 • 279

Note moe_middle · 336B tokens
IFM/LoopedLM-P1-moe-middle-500b

Text Generation • 9B • Updated Oct 2 • 261

Note moe_middle · 500B tokens
IFM/LoopedLM-P1-moe-middle-input-injection-336b

Text Generation • 9B • Updated Oct 2 • 204

Note moe_middle_input_injection · 336B tokens
IFM/LoopedLM-P1-moe-middle-input-injection-500b

Text Generation • 9B • Updated Oct 2 • 308

Note moe_middle_input_injection · 500B tokens
IFM/LoopedLM-P1-moe-ouro-336b

Text Generation • 9B • Updated Oct 2 • 261

Note moe_ouro · 336B tokens
IFM/LoopedLM-P1-moe-ouro-500b

Text Generation • 9B • Updated Oct 2 • 343

Note moe_ouro · 500B tokens
IFM/LoopedLM-P1-moe-shared-hl-336b

Text Generation • 9B • Updated Oct 2 • 200

Note moe_shared_hl · 336B tokens
IFM/LoopedLM-P1-moe-shared-hl-500b

Text Generation • 9B • Updated Oct 2 • 189

Note moe_shared_hl · 500B tokens
IFM/LoopedLM-P1-dense-split-hl-x2-336b

Text Generation • 1B • Updated Oct 2 • 226

Note dense_split_hl_x2 · 336B tokens
IFM/LoopedLM-P2-d4-m

Text Generation • 0.8B • Updated 30 days ago • 466

Note Table 4, M, D4
IFM/LoopedLM-P2-d4-s

Text Generation • 0.3B • Updated 30 days ago • 528

Note Table 4, S, D4
IFM/LoopedLM-P2-huginn-l-fixed-pln5

Text Generation • 2B • Updated 30 days ago • 377

Note Table 2, L, Fixed PLN-5
IFM/LoopedLM-P2-huginn-l-fixed-r5

Text Generation • 2B • Updated 30 days ago • 531

Note Table 2, L, Fixed R=5
IFM/LoopedLM-P2-huginn-l-learned-entropy0

Text Generation • 2B • Updated 30 days ago • 433

Note Table 2, L, Learned lambda_H=0
IFM/LoopedLM-P2-huginn-l-learned-entropy0p01

Text Generation • 2B • Updated 30 days ago • 412

Note Table 2, L, Learned lambda_H=0.01
IFM/LoopedLM-P2-huginn-m-fixed-pln5

Text Generation • 0.8B • Updated 30 days ago • 443

Note Table 2, M, Fixed PLN-5
IFM/LoopedLM-P2-huginn-m-fixed-r5

Text Generation • 0.8B • Updated 30 days ago • 514

Note Table 2, M, Fixed R=5
IFM/LoopedLM-P2-huginn-m-huginn-linear

Text Generation • 0.8B • Updated 30 days ago • 406

Note Table 3, M, Huginn-Linear
IFM/LoopedLM-P2-huginn-m-learned-entropy0

Text Generation • 0.8B • Updated 30 days ago • 434

Note Table 2, M, Learned lambda_H=0
IFM/LoopedLM-P2-huginn-m-learned-entropy0p01

Text Generation • 0.8B • Updated 30 days ago • 484

Note Table 2, M, Learned lambda_H=0.01 (Table 4 M teacher)
IFM/LoopedLM-P2-huginn-m-orthoinj

Text Generation • 0.8B • Updated 30 days ago • 529

Note Table 3, M, OrthoInj
IFM/LoopedLM-P2-huginn-m-parcae

Text Generation • 0.8B • Updated 30 days ago • 575

Note Table 3, M, Parcae-Decay
IFM/LoopedLM-P2-huginn-s-fixed-pln5

Text Generation • 0.3B • Updated 30 days ago • 517

Note Table 2, S, Fixed PLN-5
IFM/LoopedLM-P2-huginn-s-fixed-r5

Text Generation • 0.3B • Updated 30 days ago • 525

Note Table 2, S, Fixed R=5
IFM/LoopedLM-P2-huginn-s-huginn-linear

Text Generation • 0.3B • Updated 30 days ago • 439

Note Table 3, S, Huginn-Linear
IFM/LoopedLM-P2-huginn-s-learned-entropy0

Text Generation • 0.3B • Updated 30 days ago • 417

Note Table 2, S, Learned lambda_H=0
IFM/LoopedLM-P2-huginn-s-learned-entropy0p01

Text Generation • 0.3B • Updated 30 days ago • 426

Note Table 2, S, Learned lambda_H=0.01 (Table 4 S teacher)
IFM/LoopedLM-P2-huginn-s-orthoinj

Text Generation • 0.3B • Updated 30 days ago • 479

Note Table 3, S, OrthoInj
IFM/LoopedLM-P2-huginn-s-parcae

Text Generation • 0.3B • Updated 30 days ago • 519 • 1

Note Table 3, S, Parcae-Decay
IFM/LoopedLM-P2-distilled-m-random-init

0.2B • Updated 30 days ago • 220

Note Table 4, M, Distilled, random init
IFM/LoopedLM-P2-distilled-m-teacher-init

0.2B • Updated 30 days ago • 398

Note Table 4, M, Distilled, teacher init
IFM/LoopedLM-P2-distilled-s-random-init

54.3M • Updated 30 days ago • 278

Note Table 4, S, Distilled, random init
IFM/LoopedLM-P2-distilled-s-teacher-init

54.3M • Updated 30 days ago • 256

Note Table 4, S, Distilled, teacher init

Upvote