This isn't a "flip a switch" proposal. The paper outlines a phased engineering curriculum to gradually move AI from purely probabilistic text prediction toward hybrid deterministic reasoning.
Forcefully reset and orthogonalise ~100 operator tokens so they have clean, unambiguous mathematical meanings β pristine coordinates 90Β° apart in the model's internal space.
Use The Weaver to generate millions of flawless, verified mathematical reasoning traces via rejection sampling β building the training dataset from scratch.
Fine-tune the model, but only penalise it for missing the exact mathematical operations. The English prompt stays in the input for context but is masked from the loss function β preventing distraction by word prediction.
Use PPO/GRPO with The Weaver as the reward model. Correct math = reward. Hallucinated English during a math operation = Cognitive Segfault penalty.
The ambitious end-goal: the model internalises the VSA algebra natively via "grokking" β a phenomenon where neural networks suddenly discover exact algebraic circuits after prolonged training. The Weaver is no longer needed because the model reasons directly in vector space, bypassing token generation entirely.
Stage 5 replaces cross-entropy loss entirely with a physics-inspired loss:
β_latent = ||h_predict β V_target||Β² + Ξ»(1 β cos(h_predict, V_target)).
The hypothesis rests on grokking (Power et al., 2022): over-parameterised networks, long after achieving zero training error by memorisation, can undergo sudden phase transitions where they discover exact, generalising algebraic circuits (e.g. modular arithmetic). Combined with Neural Algorithmic Reasoning (VeliΔkoviΔ et al., 2021), this suggests prolonged continuous distillation could induce native VSA internalisation β but this remains an unproven, open research question.
The complete paper includes detailed mathematical formulations, complexity proofs, data tables, and the full reference list.
Read the Full Research Paper β