Upload PCR + complex screening hybrid, Path-X test acc 0.9350

README.md

@@ -0,0 +1,103 @@
+---
+license: mit
+tags:
+  - pytorch
+  - long-range-arena
+  - path-x
+  - state-space-model
+  - linear-recurrence
+  - complex-valued-neural-network
+  - sequence-classification
+datasets:
+  - long-range-arena
+metrics:
+  - accuracy
+---
+
+# PCR + Complex Screening Hybrid — Path-X (Long Range Arena)
+
+A **phase-coherent linear recurrence (PCR)** model — a complex-diagonal
+LRU/S4D-style recurrence — combined with a **non-competing complex
+screening attention** module, trained on **Path-X** (Long Range Arena),
+the 16,384-token binary sequence-connectivity task.
+
+- **Task**: raw 1D token sequence in, single binary label out. No 2D
+  structure, no auxiliary supervision, no handcrafted features — the same
+  rule-compliant setting as S4/S5/LRU/MEGA on the LRA leaderboard.
+- **Test accuracy**: **0.9350** (n=20,000, full deterministic sweep)
+- **PCR-only ablation** (no screening attention): 0.9254 ± 0.0028 (N=2 seeds)
+
+| Model | Path-X (test) |
+|---|---|
+| S4D-Real (θ=0, no phase) | chance |
+| S4D-LegS | 91.9 |
+| **PCR (this repo, screening ablated)** | **92.54 ± 0.28** |
+| LRU | 94.2 |
+| MEGA-chunk | 93.81 |
+| **PCR + screening hybrid (this repo)** | **93.50** |
+| S4 | 96.35 |
+| MEGA | 97.98 |
+| S5 | 98.58 |
+
+## Architecture
+
+```
+tokens (B, 16384)
+  -> linear encoder (scalar pixel -> d_model)
+  -> 6 x PCRBlock:
+       [BatchNorm -> PCRLayer (complex diagonal LTI, bidirectional, FFT-conv)
+        -> half-GLU -> residual]
+     with ComplexScreenBlock inserted after layers 2 and 4:
+       [chunked (1024) non-competing complex screening attention:
+        L2-normalized complex q,k -> trim-and-square gate
+        (no softmax, no row-normalization) -> TanhNorm -> modReLU gate
+        -> complex Hadamard -> residual]
+  -> LayerNorm -> mean-pool -> linear head -> 2-class logits
+```
+
+**Design principle (Phase-Coherent Transformer / PCT)**: complex
+eigenvalues implement input-independent phase rotation as coherent
+long-range transport (a continuous analogue of RoPE); all input-dependent
+gating, normalization, and readout stay real-valued. ~94% of parameters
+are complex-valued (100% within the recurrence and attention score/value
+paths; the ~6% real-valued mass is the input-dependent gates, norms, and
+readout — kept real by design, not by omission).
+
+Full experimental record, ablations (phase-necessity via a real-eigenvalue
+control, phase-bandwidth-vs-generalization sweep), and the training/eval
+harness are described in the source repository (see below).
+
+## Files
+
+- `pytorch_model.pt` — `state_dict` only (2,013,716 tensor elements across
+  116 parameter tensors)
+- `config.json` — architecture + optimizer config used for this run
+
+## Usage
+
+Load with the `PCRClassifier` / `PCRBlock` / `ComplexScreenBlock` definitions
+from the source training script (`train_with_checkpoint.py`, `cell="pcr"`
+with `pcr_config` matching `config.json`'s `pcr_config` field). This repo
+ships raw weights, not a packaged Python module — see the config for exact
+hyperparameters to reconstruct the module before calling
+`model.load_state_dict(torch.load("pytorch_model.pt"))`.
+
+## Training details
+
+- Optimizer: AdamW, base lr 4.5e-4, recurrence/B/C params at 1/3 lr with no
+  weight decay, cosine-hold-then-linear-decay schedule (decay starts at
+  step 200,000), 250,000 steps total, batch size 32.
+- Eigenvalue init: ring `|λ| ∈ [0.999, 0.9999]`, phase restricted to
+  `θ ∈ [0, π/10]` — the phase bandwidth was found necessary for
+  generalization (a narrower `[0, π/50]` band memorizes train perfectly
+  but fails to generalize; a real-only ablation, θ=0, fails to learn at
+  all).
+- No dropout, weight decay 0.05, gradient clip 1.0.
+
+## Caveats
+
+- Single seed for the hybrid checkpoint in this repo (N=1); the PCR-only
+  ablation number (92.54 ± 0.28) is averaged over 2 seeds.
+- Not benchmarked beyond Path-X, LRA Text, and LRA Image; no
+  task-specific hyperparameter tuning was performed for those two
+  auxiliary benchmarks.

config.json

@@ -0,0 +1,40 @@
+{
+  "architecture": "PCRClassifier (phase-coherent linear recurrence + non-competing complex screening attention hybrid)",
+  "task": "pathx",
+  "seq_len": 16384,
+  "vocab_size": 256,
+  "n_params": 2011660,
+  "training_steps": 250000,
+  "pcr_config": {
+    "d_model": 128,
+    "n_layers": 6,
+    "d_state": 256,
+    "norm": "batch",
+    "block": "s5",
+    "encoder": "linear",
+    "dropout": 0.0,
+    "r_min": 0.999,
+    "r_max": 0.9999,
+    "max_phase": 0.31416,
+    "lr_rec_factor": 0.3333,
+    "lr_bc_factor": 0.3333,
+    "screen_layers": [
+      2,
+      4
+    ],
+    "screen_heads": 4,
+    "screen_dim_head": 32,
+    "screen_chunk": 1024
+  },
+  "optimizer": {
+    "lr": 0.00045,
+    "weight_decay": 0.05,
+    "warmup": 2500,
+    "lr_scheduler": "hold_then_linear",
+    "lr_decay_start": 200000,
+    "lr_final_ratio": 0.1,
+    "clip_grad": 1.0,
+    "batch_size": 32
+  },
+  "seed": 0
+}

pytorch_model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f3ab75e7aad52d1d20a8f06def2bf6c4a7b4b8b3425966b68781a722cd0a80b0
+size 8092339