Google's Gemini 4 Argon scores 97.5% on a tough new test, but only with a special setup

Using the standard setup, the score in ARC Prize's independent test was 49.8%.

Confirmed

Google's Gemini 4 Argon has scored 97.5% on a tough new test called ARC-AGI-3. But it only got that score using a special setup, the ARC Prize Foundation says. The foundation is a non-profit group that runs the tests independently. ARC-AGI-3 is a set of new puzzles. It is built to check whether a super intelligence (SI) system can work out tasks it has never seen before. A "harness" is the setup that connects the SI system to the test. Gemini 4 Argon scored 97.5% using a special harness called the "Provider Adapter". On the standard harness, it scored 49.8%. Those standard runs were not complete, ARC Prize said. Only 42 of the 55 tasks were finished at the low reasoning level. That means the two scores are hard to compare fairly. ARC Prize published the results on a page dated 13 October. Test scores are often used to claim SI is catching up with people. This result shows how much the way a test is run can change the score.