SIMULATE BEFORE REALITY

Evaluating the AI Products
with 8.3 Billion Persona Agents

MatrAIx began as a way to run an A/B test before any real user sees the change: a population of 8.3 billion persona agents tries each variant first, so teams learn what works, and for whom, before launch.

01 — WHERE IT STARTED

It started with the A/B test.

A new checkout flow, a pricing page, a chatbot's reply: product teams settle most of these decisions with an A/B test. MatrAIx began with one question. What if you could run that experiment before any real user sees it?

TODAY · A/B TEST IN REALITY

Ship it, split real traffic, wait.

  1. Build both variants and put them in front of real users.
  2. Split live traffic. Some users get the variant that turns out worse.
  3. Wait for significance: days or weeks, and only if you have the traffic.
  4. Read one average. You learn which variant won, not for whom or why.
MATRAIX · SIMULATE BEFORE REALITY

Run the experiment on simulated users first.

  1. Choose a population of persona agents that matches your users.
  2. Let them use each variant: a survey, a chatbot, a website, or an app.
  3. Read the results by persona, with the reasons behind each choice.
  4. Send only the promising ideas on to a live A/B test.

Running that experiment well needs three things: a realistic population of users, environments they can act in, and tasks that reflect real decisions. Those became the layers of the MatrAIx stack below. The same question, how does this work for this user, now drives our work on personal agent evaluation.

THE MATRAIX THESIS

The next frontier for personal agents is understanding humanity—and learning to behave like us.

01 · WHOPersona
02 · WHERE / HOWEnvironment
03 · WHAT / WHYApplication
OUTPUTEvaluation Results
02 — APPLICATION TASKS

Four types of evaluation tasks.

MARKET RESEARCH · PROMOTION TEST “Which offer would make you buy today?”
Free ship
82%
10% off
74%
Returns
61%
2× points
45%
CUSTOMER SERVICE · PRODUCT RETURN
These shoes don't fit. Can I return them?
Yes. Were they worn outside?
No, only tried on.
Your return is approved. Here's the label. ✓
shop.test/checkout
SHOPPING WEBSITE · CHECKOUT
▲
SOCIAL APP · CREATE A GROUP

Survey

// questionnaires & feedback

Collect structured and open-ended user feedback for market research, concept testing, and preference analysis.

Chatbot

// conversational AI evaluation

Evaluate AI chatbots across task completion, customer satisfaction, helpfulness, safety, and multi-turn reliability.

Web

// web prototype evaluation

Evaluate web prototypes and features across usability, presentation, navigation, latency sensitivity, and task completion.

App

// app product evaluation

Evaluate app features and workflows across functionality, responsiveness, task success, and user preference.

03 — WHAT MATRAIX PROVIDES

From simulated users to actionable evaluation results.

04 — DEMO

See MatrAIx in action.

05 — EVAL INFRA

Start exploring the MatrAIx eval infra.

◉ MatrAIx Eval InfraPERSONAENVEVAL
8.3Bpersona corpus
1,290persona attributes
1,000+applications
06 — WHAT CAME NEXT

From products to personal agents.

The persona population behind these studies now powers our work on personal agent evaluation, including the PersonalAgentEval benchmark.