Loading Events

« All Events

  • This event has passed.

Language Models Without Language: Teaching AI to Think in Vectors

October 26 at 3:30 pm - 4:30 pm

About the talk

Language models were built to read and write language, yet a new generation of them thinks without it. Instead of writing out their reasoning step by step, these models think in vectors, called thought representations, which pass from one reasoning step to the next and even from one AI agent to another. Thinking in vectors is faster than thinking in words, but no one has stated what a thought representation must contain to be useful. These models are trained only on their final answers, which remain correct even when a model ignores its own thought representations.

First, we define four properties of a useful thought representation. Next, we turn each property into a measurable criterion and evaluate existing methods. On at least one measure, every audited method performs worse than a baseline that never reasons and only encodes the question. Finally, we convert each property into a training loss. The resulting method, REpresentation-Supervised Thoughts (REST), teaches a model how to think, not only what to answer. Across seven benchmarks in mathematics, science, medicine and code, REST raises the accuracy of single- and multi-agent systems compared with standard training. Language models do not need words to think, but their thoughts still need rules.

About the speaker

Fahd Seddik
MSc researcher at UBC Okanagan

Preregistration not required. If you have questions, please email: yves.lucet@ubc.ca.

 

Details

Date:
October 26
Time:
3:30 pm - 4:30 pm

Venue

Additional Info

Room Number
ASC 301
Registration/RSVP Required
No
Event Type
Presentation, Talk/Lecture
Topic
Research and Innovation, Science, Technology and Engineering
Audiences
Faculty, Staff, Undergraduate Students, Graduate Students, Postdoctoral Fellows and Research Associates