Please note: This master’s thesis presentation will take place in DC 2310 and online.
Isaac Joffe, Master’s candidate
David R. Cheriton School of Computer Science
Supervisor: Professor Chris Eliasmith
The Abstraction and Reasoning Corpus for Artificial General Intelligence (ARC-AGI) is a popular artificial intelligence (AI) benchmark comprising abstract reasoning tasks that test fluid intelligence in a generative, few-shot setting. Although humans solve ARC-AGI with ease, it remains extremely difficult for all but the most advanced AI systems.
Inspired by methods for modelling biological intelligence spanning psychology to neuroscience, we present a general framework for automated solving of ARC-AGI tasks as well as two specific ARC-AGI solvers that implement our framework. Our cognitive inspiration draws on dual process theory (System 1 and System 2) and vector symbolic algebras (VSAs), and our solvers apply neurosymbolic, object-centric program synthesis.
The first of our two ARC-AGI solvers, Solver 1, uses VSAs to represent objects. Solver 1 structures its solutions to ARC-AGI tasks as sets of conditional neurosymbolic rules implemented in a flexible domain-specific language (DSL), and synthesizes its solutions using System 1, VSA-enabled sample-efficient neural learning within System 2, VSA-powered heuristic search. Overall, Solver 1 scores $10.8\%$ on ARC-AGI-1-Train and $3.0\%$ on ARC-AGI-1-Eval. Additionally, Solver 1 scores $94.5\%$ and $83.1\%$ on the simpler Sort-of-ARC and 1D-ARC benchmarks---the latter of which outperforms GPT-4 without any computationally expensive pre-training.
The second of our two ARC-AGI solvers, Solver 2, uses VSAs to represent programs. Solver 2 structures its solutions to ARC-AGI tasks as sequential compositions of transformations implemented in a bespoke DSL, and synthesizes its solutions using System 1, VSA-powered neural guidance for System 2, VSA-mediated procedural reasoning. Overall, Solver 2 scores $13.5\%$ on ARC-AGI-1-Train and $3.5\%$ on ARC-AGI-1-Eval. Additionally, our best version of Solver 2 is $7\times$ more efficient than a naive version of Solver 2, and $235\times$ more efficient than brute-force search.
When combined into an ensemble, our two solvers score $21.0\%$ on ARC-AGI-1-Train and $6.2\%$ on ARC-AGI-1-Eval. Importantly, we believe that we are the first to apply VSAs to ARC-AGI and, in doing so, have developed two of the most efficient, interpretable, and cognitively plausible ARC-AGI solvers yet.
To attend this master’s thesis presentation in person, please go to DC 2310. You can also attend virtually on Zoom.