What it is
A transformer that chooses and reorders its layers at inference time, letting one model span dense, recurrent, and sparse expert regimes with input-dependent execution paths. Instead of fixed-depth order or routing only within a layer, a router picks the next transformer block from a global library at each depth. This makes layer order, reuse, and sparsity input-dependent, unifying dense, looped, and MoE-style models.
Key results
- Safety probe on sequence-level routing features reaches 0.95 AUROC
- Maze strategy control hits 99.79-100% correct forced-strategy traces
- Sudoku forced routing yields 99.5-100% requested-strategy accuracy
Shows that layer order can be a learned, controllable part of transformer computation rather than a fixed architectural choice.