complexedleo/pcr-screening-pathx

PCR + Complex Screening Hybrid — Path-X (Long Range Arena)

A phase-coherent linear recurrence (PCR) model — a complex-diagonal LRU/S4D-style recurrence — combined with a non-competing complex screening attention module, trained on Path-X (Long Range Arena), the 16,384-token binary sequence-connectivity task.

➡️ Code, mathematical documentation, and the paper section: github.com/leohio/phase-coherent-transformer-r-d/tree/main/pathx

• Task: raw 1D token sequence in, single binary label out. No 2D structure, no auxiliary supervision, no handcrafted features — the same rule-compliant setting as S4/S5/LRU/MEGA on the LRA leaderboard.
• Test accuracy: 92.71 ± 0.89 over 3 seeds (n=20,000, full deterministic sweep); the checkpoint in this repo is the best seed, 93.50
• PCR-only ablation (no screening attention): 92.54 ± 0.28 (N=2 seeds)

Model	Path-X (test)
Transformer / Reformer / Performer / Linformer / BigBird / Luna-256	chance (≈50)
S4D-Real (θ=0, no phase)	chance
S4-v1	88.10
DSS	89.72
S4D-LegS	91.95
PCR (this repo, screening ablated)	92.54 ± 0.28
S4D-Inv	92.80
PCR + screening hybrid (this repo)	92.71 ± 0.89 (best 93.50)
MEGA-chunk	93.81
LRU	94.20
S4 (S4-LegS)	96.35
MEGA	97.98
S5	98.58

Architecture

tokens (B, 16384)
  -> linear encoder (scalar pixel -> d_model)
  -> 6 x PCRBlock:
       [BatchNorm -> PCRLayer (complex diagonal LTI, bidirectional, FFT-conv)
        -> half-GLU -> residual]
     with ComplexScreenBlock inserted after layers 2 and 4:
       [chunked (1024) non-competing complex screening attention:
        L2-normalized complex q,k -> trim-and-square gate
        (no softmax, no row-normalization) -> TanhNorm -> modReLU gate
        -> complex Hadamard -> residual]
  -> LayerNorm -> mean-pool -> linear head -> 2-class logits

Design principle (Phase-Coherent Transformer / PCT) — see Complex-Valued Phase-Coherent Transformer, Hioki, arXiv:2605.10123: complex eigenvalues implement input-independent phase rotation as coherent long-range transport (a continuous analogue of RoPE); all input-dependent gating, normalization, and readout stay real-valued. The screening attention used here is the PCT paper's non-competing gate — a real-valued, element-independent, smooth gate on L2-normalized complex query-key similarities in place of softmax — applied on top of the PCR transport backbone. ~94% of parameters are complex-valued (100% within the recurrence and attention score/value paths; the ~6% real-valued mass is the input-dependent gates, norms, and readout — kept real by design, not by omission).

Full experimental record, ablations (phase-necessity via a real-eigenvalue control, phase-bandwidth-vs-generalization sweep), the derivations, and the training/eval harness are in the companion repository: phase-coherent-transformer-r-d/pathx.

Files

• pytorch_model.pt — state_dict only (2,013,716 tensor elements across 116 parameter tensors)
• config.json — architecture + optimizer config used for this run

Usage

This repo ships raw weights, not a packaged Python module. The model code (PCRClassifier / PCRBlock / ComplexScreenBlock, self-contained, torch only) and a ready-made loading example are here:

https://github.com/leohio/phase-coherent-transformer-r-d/tree/main/pathx

import json, torch
from pcr_screening import build_pcr_classifier   # pathx/code/pcr_screening.py

cfg = json.load(open("config.json"))["pcr_config"]
model = build_pcr_classifier(seq_len=16384, vocab=256, **cfg)
model.load_state_dict(torch.load("pytorch_model.pt", weights_only=True), strict=True)
model.eval()

Training details

• Optimizer: AdamW, base lr 4.5e-4, recurrence/B/C params at 1/3 lr with no weight decay, cosine-hold-then-linear-decay schedule (decay starts at step 200,000), 250,000 steps total, batch size 32.
• Eigenvalue init: ring |λ| ∈ [0.999, 0.9999], phase restricted to θ ∈ [0, π/10] — the phase bandwidth was found necessary for generalization (a narrower [0, π/50] band memorizes train perfectly but fails to generalize; a real-only ablation, θ=0, fails to learn at all).
• No dropout, weight decay 0.05, gradient clip 1.0.

Caveats

• The hybrid result is 92.71 ± 0.89 over 3 seeds (93.50 / 92.89 / 91.75); the checkpoint released here is the best of the three. The PCR-only ablation number (92.54 ± 0.28) is averaged over 2 seeds.
• Not benchmarked beyond Path-X, LRA Text, and LRA Image; no task-specific hyperparameter tuning was performed for those two auxiliary benchmarks.