polarisfactor – Digital Architects
18 Nov 2026·Retail & Commerce Media·Source: Linas Newsletter

Visa Benchmark: AI Shopping Agents Fail 23% of Purchases

Visa tested 22 AI shopping agents built on models from Google, Meta, OpenAI, Anthropic, and eight other labs. In 23.1% of runs, the agent bought a product that did not meet the shopper's needs, leading to potential disputes.

What happened

Visa tested 22 AI shopping agents built on models from Google, Meta, OpenAI, Anthropic, and eight other labs. In 23.1% of runs, the agent bought a product that did not meet the shopper's needs, leading to potential disputes. Visa's Agentic Commerce Benchmark highlights that even top models fail and that a more expensive model doesn't solve consistency issues. The company aims to establish itself in the agentic payments landscape, where liability for agent-initiated purchases remains unresolved. The test suggests a significant financial impact from disputes, positioning Visa as a potential arbiter in this emerging space.

Key facts & figures

  • ·Visa tested 22 AI shopping agents built on models from Google, Meta, OpenAI, Anthropic, and eight other labs.
  • ·In 23.1% of runs, the AI agent purchased a product that did not meet the shopper's needs.
  • ·Visa's Agentic Commerce Benchmark reported a 57.2% pass rate for the agents.
  • ·The article highlights that no party has yet agreed on liability when a shopper disputes an agent-initiated transaction.
  • ·Visa aims to position itself as a central figure in the agentic payments disputes landscape.

Why it matters — our perspective

Visa's benchmark provides crucial data on AI agent reliability, highlighting liability and dispute risks that will shape the future of agentic commerce and payments.

Key questions for investors and operators

When evaluating transactions, technology stacks, or commercial partnerships in this space, we recommend looking closely at three questions:

Should the retail media stack be built in-house, licensed as a white-label platform, or operated via federated partnerships?
How quickly can first-party shopper segments be activated off-site (CTV, open web) without data leakage or latency bottlenecks?
Does the attribution setup provide closed-loop, SKU-level incrementality reporting within advertisers' expected turnaround times?

Companies & platforms involved

Visa

Visa tested 22 AI shopping agents and announced the Agentic Commerce Benchmark.

Google

The AI shopping agents were built on models from Google.

Meta

The AI shopping agents were built on models from Meta.

OpenAI

The AI shopping agents were built on models from OpenAI.

Anthropic

The AI shopping agents were built on models from Anthropic.

Receive Insights & Market Signals by email.

Independent analysis on AdTech architecture, Connected TV, commerce media monetization, M&A transactions, and new research playbooks.

Privacy Policy

Let’s talk about your project.

Have a question about an upcoming transaction, your current ad stack, or a new monetization initiative? Feel free to reach out for a conversation.