+0.5776 bpb
Uniform HRR's validation result relative to softmax; lower was better.
EXP-006 · settled 2026-08-08
The run completed.
Softmax still led.
A public CPU/enwik8 execution independently verified the Holo HRR T=512 three-arm implementation. Both HRR variants ran successfully, but neither beat softmax in validation quality or speed.
Recovered · corpus proxy
Protocol outcomeRecovered
AuthorityCorpus proxy
ScopeSeed 0 · CPU
Decision
The implementation is executable. This run shows no HRR advantage.
The frozen public checkout completed 1,500 training steps for softmax, uniform HRR, and gated HRR under one seed and one public enwik8 split. The recovery verifier matched the captured result, source commit, source-file hashes, dataset hash, frozen configuration, and CPU device before ilXyr recorded evidence.
The scientific comparison is descriptive rather than a promotion gate: lower bits per byte and lower step time both favored softmax.
Measured result
One protocol, three attention arms.
| Arm | Parameters | Validation bits/byte | Median ms/step | Tokens/sec |
|---|---|---|---|---|
| Softmax | 4.330M | 2.9731 | 192.99 | 42,448 |
| Uniform HRR | 4.068M | 3.5507 | 226.95 | 36,096 |
| Gated HRR | 4.069M | 3.3308 | 296.08 | 27,668 |
+0.3577 bpb
Gated HRR's validation result relative to softmax.
0.850× / 0.652×
Uniform and gated training speed relative to softmax.
Claim boundary
What this result does—and does not—establish.
- It verifies that the public Holo HRR implementation executes its declared T=512 comparison end to end on the public enwik8 corpus.
- It does not reproduce the unpublished Telegram corpus or the upstream MPS hardware conditions.
- Its promoted protocol lane applies to the recovered evidence transaction, not to a claim that HRR outperformed softmax.
- It covers one seed on one CPU host; exact replication and broader performance claims remain open.
Recovery audit
The failed envelope stays visible.
1 · Training completed
All three arms finished and emitted metrics, checkpoints, a transcript, and a captured result under the frozen v2 experiment.
2 · Strict parser rejected
The wrapper added an unsupported top-level field. ilXyr retained the run but did not convert malformed output into settled evidence.
3 · Evidence recovered
A separate v3 verifier checked the captured result and provenance without rerunning training or changing parameters, then emitted a valid envelope.
Forecast chronology
| Stage | Forecaster | Probability on successful stage outcome | Settlement |
|---|---|---|---|
| Before training | Mechanistic | 96% completed | Not settled after invalid envelope |
| Before training | Empirical | 90% completed | Not settled after invalid envelope |
| Before recovery | Mechanistic | 98% recovered | Brier 0.0006 |
| Before recovery | Empirical | 97% recovered | Brier 0.0014 |
The recovery forecasts concern deterministic artifact verification after the training result existed. They are not forecasts that HRR would beat softmax.
Public evidence
Small summary here. Heavy artifacts stay out.
The public summary carries source, dataset, result, checkpoint, ledger, and event hashes without exposing raw executor output or workstation paths. The three retained checkpoints total about 48 MB and are deliberately not served by GitHub Pages.
Read the machine-readable evidence →
Inspect the frozen upstream source →