Are You Smarter Than AI?

Despite significant AI advancements, a substantial gap remains between current AI capabilities and true human-level intelligence.

The New York Times recently posed this question in an article exploring how to measure the true capabilities of AI, particularly its progress toward achieving human-level intelligence, also known as artificial general intelligence (AGI). The authors focused on a puzzle game called the Abstraction and Reasoning Corpus (ARC), which is designed to be deceptively simple for humans but quite challenging for machines. These colorful, grid-based puzzles require solvers to identify abstract visual patterns and apply logical reasoning—skills that humans tend to possess naturally.

To further advance AI development in this area, the ARC Prize was established and included a new, more challenging benchmark called ARC-AGI-2. After reading the article and playing the games, this author’s cursory cost-benefit analysis indicates that progress in this field may be slower than anticipated. The ARC Prize offered $1 million to anyone who could develop an AI system that exceeded human performance on the benchmark; a score of 85 percent is considered indicative of “smart” human performance. However, more than 1,400 AI systems attempted to win the prize but were unsuccessful. Later, OpenAI’s o3 system achieved a score of 87.5 percent on the puzzles but violated competition rules by spending nearly $1.5 million on electricity and computing resources to complete the test.

The article suggests that while AI is making impressive advancements, it still has a considerable way to go before it can fully replicate the complexities of human intelligence. The ability to efficiently learn new skills and navigate the intricacies of the real world remains a significant challenge. The pursuit of AGI is an ongoing process of refinement, with each new benchmark pushing the limits of what machines can accomplish.

The article is interesting, and the accompanying puzzles can be fun, at least the first two were. The last one may be fun (if you consider taking practice LSAT exams on a Sunday afternoon to be). UIC Law students, faculty, and staff can access the article with their credentials (click here for access info).