System1 AI tool matches human testers on 9 of 10 ads, refuses to score the worst

September 13, 2026 · thedrum.com

Andrew Tindall, who leads growth at System1, tested ten well known ads for AI brands through the company's Test Your Ad Screen tool and checked its verdicts against System1's existing human testing data on the same ads. The AI matched the human Fluency Rating for all ten ads and landed in the correct Star Rating band for nine of them. Its single miss came from a refusal: fed Claude's 'Can I Get a Six Pack' Super Bowl ad, which had scored the lowest possible rating with human testers, the tool declined to assign a score. It correctly identified the weakest ads in the set, Otter.ai's campaign, ChatGPT's pre pivot Super Bowl ad and Apple's email rewriting ad, all rated one star by humans, and correctly picked the strongest two, Coca-Cola's AI generated Christmas ad and Google Gemini's 'Dream Job', which scored 5 stars and 3.1 stars respectively. Samsung's Galaxy S26 launch and the Meta x Ray-Ban campaign proved harder to place precisely, with the AI only able to narrow Samsung's ad to 'not a 1-Star' and Meta's to the bottom 40% of ads.

Creative pre-testing budgets can run through an automated first pass before committing to expensive human panels, provided the tool used is built to say when it does not know. Tindall points to NYU Stern and Emory's finding that AI ads outperformed human ads by up to 19% in controlled tests, a lead that dropped by 31.5% once AI involvement was disclosed, as evidence that consumers rarely notice or care how an ad was made. Media teams evaluating growing volumes of AI assisted creative can use tools like Test Your Ad Screen to triage which executions merit full human testing, while the same research warns that AI prediction breaks down exactly where creative is most novel, the ads most likely to move a brand forward.