If you can't kill it, it might actually be good.

Allowed transformations

Orthogonal + permutation Tucker rotations on layer or expert mode

Invariant

Frobenius norm; rank family preserved

Kill threshold

residual_ratio 0.5 fraction

Non-vacuity constraint

r_eff < dim(mode) when rotated to optimal basis

Killed at all layer + expert scopes on OLMoE-1B-7B. Orthogonal-only rotations reshape spectrum by less than 5%; minimum-residual basis still leaves ≥ 88.9% energy outside any rank-(r_eff−1) projection.

Empirical probe cross-check/ffn-quant/g0_orthogonal_tucker.py · JSON

Allowed transformations

Bounded conditioning (cond ≤ κ) non-orthogonal regauge

Invariant

Spectral envelope preserved up to κ

Kill threshold

vs_orth_advantage 0.05 fraction

Non-vacuity constraint

Learned gauge must beat random orthogonal at fixed cond

Reclassified after G₁.0' r2: 24/24 vs_orth fail at 0.50 gate; learned gauges beat zero-cost cond=1 orthogonal random by only 2–3%. Reframes Paper 2 narrative as SRHT/structured-rotation rather than learned gauge.

Empirical probe cross-check/ffn-quant/g1_0prime_gauge_quant.py · JSON

Allowed transformations

General invertible Tucker on (layer, expert, hidden) modes

Invariant

Multi-linear rank tuple

Kill threshold

r_eff 256 rank

Non-vacuity constraint

r_eff < dim across all 3 modes

Killed on OLMoE: minimum r_eff across layer/expert/hidden Tucker reshape is 262.773, exceeding the 256 budget. Cross-mode reshaping cannot compress past the hidden-dimension barrier without losing > 99.6% energy.

Empirical probe cross-check/ffn-quant/g2_cross_mode.py · JSON

Allowed transformations

Replace W_e by W_eff(e) = Σ_t π(t,e) W_e for routed expert e

Invariant

Routed-action equivalence (effective Gram preserved)

Kill threshold

r_eff_effective 16 rank

Non-vacuity constraint

Energy captured at q ≤ 32 must exceed 50%

Modest constructive at q ∈ [16, 32]: 58–80% energy captured. Below q=8 the effective rank floor (≥ 3.89) prevents further compression.

Empirical probe cross-check/ffn-quant/g3_routed_eff_weights.py · JSON

Allowed transformations

Per-task low-rank weight subsets (humaneval, gsm8k, wikitext)

Invariant

Task-conditional activation-weighted error

Kill threshold

uniform_rank_pass_rate 0.5 fraction

Non-vacuity constraint

Task-specific bases must beat uniform-rank null

Uniform rank-(16,24) killed with strong nulls. Task-conditional surviving: humaneval 4/4 cells pass, gsm8k 3/4, wikitext 2/4. Layer 3 + wikitext is the worst-case kernel.

Empirical probe cross-check/ffn-quant/g4_task_conditional.py · JSON

Allowed transformations

Sparse linear feature deflation per LRH

Invariant

Linear-representation hypothesis residual

Kill threshold

lrh_plausibility 0.5 fraction

Non-vacuity constraint

Post-deflation residual must remain LRH-plausible

LRH proxy collapses after deflation: 13/16 layers fall below the LRH-plausibility threshold once linear features are removed. Causal-circuit compression cannot rest on LRH for OLMoE.

Empirical probe cross-check/g5/lrh_deflation.py · JSON

Allowed transformations

Low-rank unary router (R = U_r V_r^T)

Invariant

Top-k expert membership

Kill threshold

route_flip_rate 0.1 fraction

Non-vacuity constraint

Route-flip < 10% at any rank below full

Killed at every tested rank. Route-flip rate exceeds the 10% tolerance even at r = router_dim − 4, ruling out unary low-rank routers.

Empirical probe cross-check/router/g6_unary_low_rank.py · JSON

Allowed transformations

Pair-co-occurrence router C_pair restricted to dominant pairs

Invariant

Pairwise selection preserved on hot set

Kill threshold

top100_coverage 0.95 fraction

Non-vacuity constraint

Hot set of dominant pairs must cover ≥ 95% of expert traffic

Hot-set is too dispersed: top-100 dominant pairs cover only 72% of router traffic, well below the 95% gate. Lean machinery deferred until coverage gate is reopened.

Empirical probe cross-check/router/g7_pairwise.py · JSON