Benchmarking xLSTM and hybrid sequence models
Architecture comparisons become useful when capacity, task, repeated runs, and uncertainty stay visible together.
Compare the benchmark results.
Controlled experiments on hybrid xLSTM, LSTM, and Transformer blocks across associative recall and formal-language tasks.
MM6 runs92.0%± 13.0
TT6 runs28.0%± 7.8
At d = 8, the model choice exposes the largest capacity differences; by d = 16, the recorded configurations converge.Compare hybrid sequence blocks without reducing the result to a single score that hides width thresholds, task sensitivity, or run-to-run uncertainty.
- 01
Implemented repeatable benchmarks for Multi-Query Associative Recall and Chomsky-hierarchy formal languages.
- 02
Compared block composition and embedding capacity under fixed training and validation settings.
- 03
Stored individual experiment results as structured JSON and summarized repeated runs with mean and 95% confidence intervals.
The MQAR view reports the recorded N=128 ablation summaries with 95% confidence intervals over 5–7 runs per point. ST and TS have identical recorded values at each width, so their intervals have zero visible width. The formal-language view uses five runs per result.
- PyTorch
- Python
- xLSTM
- Transformers
- Jupyter
- CUDA