If you can't kill it, it might actually be good.

Allowed transformations

Orthogonal + permutation Tucker rotations on layer or expert mode

Invariant

Frobenius norm; rank family preserved

Kill threshold

residual_ratio 0.5 fraction

Non-vacuity constraint

r_eff < dim(mode) when rotated to optimal basis

Killed at all layer + expert scopes on OLMoE-1B-7B. Orthogonal-only rotations reshape spectrum by less than 5%; minimum-residual basis still leaves ≥ 88.9% energy outside any rank-(r_eff−1) projection.

Empirical probe cross-check/ffn-quant/g0_orthogonal_tucker.py · JSON

Allowed transformations

Bounded conditioning (cond ≤ κ) non-orthogonal regauge

Invariant

Spectral envelope preserved up to κ

Kill threshold

vs_orth_advantage 0.05 fraction

Non-vacuity constraint

Learned gauge must beat random orthogonal at fixed cond

Reclassified after G₁.0' r2: 24/24 vs_orth fail at 0.50 gate; learned gauges beat zero-cost cond=1 orthogonal random by only 2–3%. Reframes Paper 2 narrative as SRHT/structured-rotation rather than learned gauge.

Empirical probe cross-check/ffn-quant/g1_0prime_gauge_quant.py · JSON

Allowed transformations

General invertible Tucker on (layer, expert, hidden) modes

Invariant

Multi-linear rank tuple

Kill threshold

r_eff 256 rank

Non-vacuity constraint

r_eff < dim across all 3 modes

Killed on OLMoE: minimum r_eff across layer/expert/hidden Tucker reshape is 262.773, exceeding the 256 budget. Cross-mode reshaping cannot compress past the hidden-dimension barrier without losing > 99.6% energy.

Empirical probe cross-check/ffn-quant/g2_cross_mode.py · JSON

Allowed transformations

Replace W_e by W_eff(e) = Σ_t π(t,e) W_e for routed expert e

Invariant

Routed-action equivalence (effective Gram preserved)

Kill threshold

r_eff_effective 16 rank

Non-vacuity constraint

Energy captured at q ≤ 32 must exceed 50%

Modest constructive at q ∈ [16, 32]: 58–80% energy captured. Below q=8 the effective rank floor (≥ 3.89) prevents further compression.

Empirical probe cross-check/ffn-quant/g3_routed_eff_weights.py · JSON

Allowed transformations

Per-task low-rank weight subsets (humaneval, gsm8k, wikitext)

Invariant

Task-conditional activation-weighted error

Kill threshold

uniform_rank_pass_rate 0.5 fraction

Non-vacuity constraint

Task-specific bases must beat uniform-rank null

Uniform rank-(16,24) killed with strong nulls. Task-conditional surviving: humaneval 4/4 cells pass, gsm8k 3/4, wikitext 2/4. Layer 3 + wikitext is the worst-case kernel.

Empirical probe cross-check/ffn-quant/g4_task_conditional.py · JSON

Allowed transformations

Sparse linear feature deflation per LRH

Invariant

Linear-representation hypothesis residual

Kill threshold

lrh_plausibility 0.5 fraction

Non-vacuity constraint

Post-deflation residual must remain LRH-plausible

LRH proxy collapses after deflation: 13/16 layers fall below the LRH-plausibility threshold once linear features are removed. Causal-circuit compression cannot rest on LRH for OLMoE.

Empirical probe cross-check/g5/lrh_deflation.py · JSON

Allowed transformations

Low-rank unary router (R = U_r V_r^T)

Invariant

Top-k expert membership

Kill threshold

route_flip_rate 0.1 fraction

Non-vacuity constraint

Route-flip < 10% at any rank below full

Killed at every tested rank. Route-flip rate exceeds the 10% tolerance even at r = router_dim − 4, ruling out unary low-rank routers.

Empirical probe cross-check/router/g6_unary_low_rank.py · JSON

Allowed transformations

Pair-co-occurrence router C_pair restricted to dominant pairs

Invariant

Pairwise selection preserved on hot set

Kill threshold

top100_coverage 0.95 fraction

Non-vacuity constraint

Hot set of dominant pairs must cover ≥ 95% of expert traffic

Hot-set is too dispersed: top-100 dominant pairs cover only 72% of router traffic, well below the 95% gate. Lean machinery deferred until coverage gate is reopened.

Empirical probe cross-check/router/g7_pairwise.py · JSON

Allowed transformations

Replace attention block by α-decayed N/D recurrence with bounded local defect δ; certified by discrete-Grönwall + scan-monoid associativity (parallel ≡ sequential).

Invariant

Geometric error envelope α^t · E₀ + δ/(1−α); scan composition associativity preserves recurrent ≡ parallel-segment equivalence.

Kill threshold

held_out_residual_H 0.5 ratio

Non-vacuity constraint

Decay 0 ≤ α < 1 with held-out gap H ≤ 0.5; otherwise the contractive bound is vacuous and the receptacle does not strictly transport state.

Mathematical foundation now spans 14 theorems (RWKVStability.lean, 0 sorry, 435 LoC): scalar Grönwall, per-channel envelope, sup-δ certification, L∞ + L2 aggregators, scan-monoid associativity, blockwise stability. Empirical viability ladder ran three iterations on real OLMoE-1B-7B: G_10.0 (scalar) had vacuous floor 12.07; G_10.0a (per-channel + cumulative-key receptacle) collapsed it to 0.045 on Layer 6; G_10.0b (sup-δ + trap repair + cross-corpus) widened to 0.628 with 4/6 ESCALATE gates passing; G_10.0c (geom-sum predictor) only 5% tighter on average — confirmed the block is mathematical not predictor-form. Worst-channel α≈0.984 produces 1/(1-α)≈60 regardless of T. To reach ESCALATE, needs channel sparsification, activation-weighted relative-L2 bound, or different Lean theorem family entirely. Stops here at conceptual block.

Empirical probe cross-check/rwkv-receptacle/rw_replacement_ladder_queue.py · JSON

Allowed transformations

Decompose A = A_smooth + A_sparse where A_sparse = top-k (k=4) routing mask per row; fit RWKV exponential basis to A_smooth only; preserve A_sparse verbatim.

Invariant

Row-stochastic mass conservation under split (sparse_smooth_mass_decomposition); attention output linearity under sum decomposition (attention_output_split).

Kill threshold

hybrid_vs_smooth_only_ratio 0.5 ratio

Non-vacuity constraint

≥1 routing-heavy head per layer (M_sparse > 0.5) AND routing stability > 0.5 across adjacent layers; otherwise the decomposition is either trivial or capturing noise.

Diagnostic-constructive rung that explains the G_10 family's failures by separating compressible transport from incompressible routing. Decomposition probe on real OLMoE-1B-7B: 48/64 heads are hybrid (significant sparse mass), 16 transport-dominated, 0 pure routing-dominated; sparse mass mean 0.593 across all heads/layers. Hybrid reconstruction recovers attention output 3.7× tighter than RWKV alone. The routing-stability gate failed at 0.23 Jaccard between adjacent layers — sparse routing is dynamic and content-addressed, so the static-overlap metric is the wrong test. Future iteration G_10.6 should replace Jaccard with a subspace-overlap routing metric or per-layer head taxonomy. NOT a standalone compression mechanism: A_sparse must be preserved verbatim, only A_smooth is RWKV-compressible.

Empirical probe cross-check/rwkv-receptacle/g10_5_hybrid_decomposition.py · JSON