- This event has passed.
Language Models Without Language: Teaching AI to Think in Vectors
October 26 at 3:30 pm - 4:30 pm

About the talk
Language models were built to read and write language, yet a new generation of them thinks without it. Instead of writing out their reasoning step by step, these models think in vectors, called thought representations, which pass from one reasoning step to the next and even from one AI agent to another. Thinking in vectors is faster than thinking in words, but no one has stated what a thought representation must contain to be useful. These models are trained only on their final answers, which remain correct even when a model ignores its own thought representations.
First, we define four properties of a useful thought representation. Next, we turn each property into a measurable criterion and evaluate existing methods. On at least one measure, every audited method performs worse than a baseline that never reasons and only encodes the question. Finally, we convert each property into a training loss. The resulting method, REpresentation-Supervised Thoughts (REST), teaches a model how to think, not only what to answer. Across seven benchmarks in mathematics, science, medicine and code, REST raises the accuracy of single- and multi-agent systems compared with standard training. Language models do not need words to think, but their thoughts still need rules.
About the speaker
Fahd Seddik
MSc researcher at UBC Okanagan
Preregistration not required. If you have questions, please email: yves.lucet@ubc.ca.