← THE JOURNEY
PHASE 03TRIAL-SET OPTIMISATION

Building the
Machine

How do you pick 200 trials out of 17,955 possible choices?

The problem

The design calls for a few hundred trials. But which ones?

Investments range from 1 to 10 tokens. Probabilities from 5% to 95%. Outcomes from 0 to 39 tokens. The full factorial space contains 17,955 possible trials.

The challenge

Not all trials are equally informative. Some tell us a lot about Prospect Theory parameters. Others are redundant or uninformative. We need a systematic method to select the best ones.

STEP 1

Generate the trial space

First, create all possible trials. For each investment level (1 to 10 tokens), generate all combinations of outcomes and probabilities.

1–10
tokens invested
5–95%
probabilities, in 5% steps
17,955
possible trials, net losses and gains
full factorial
STEP 2THE PARAMETER SPACE

A fine grid over four dimensions

Define the complete 4D parameter space for Prospect Theory. For each parameter, a grid of 50 values.

α outcome sensitivity0.5 → 1.0
γ+ probability weighting, gains0.4 → 3.0
γ− probability weighting, losses0.4 → 3.0
λ loss aversion1.0 → 6.0
6.25M
parameter combinations
250
parallel jobs
STEP 3

Certainty equivalents

For every single trial (17,955) and every single parameter combination (6.25M), calculate the certainty equivalent: how much is this gamble worth to a person with these specific risk preferences?

This required massive parallel computing. We computed 112 billion certainty equivalents to build a complete map of predicted behaviour.

Ina trial and a set of parameters
Outits certainty equivalent
STEP 4

Filter to discriminative trials

Eliminate trials that don't vary with parameter values. If everyone accepts a trial (or everyone rejects it) regardless of their parameters, it provides no information.

  1. 1Threshold crossing (CE crosses 0)
  2. 2Sufficient range (range ≥ 0.5)
  3. 3Avoid consensus (< 95% agreement)
filtering results
17,955 → 7,169
discriminative trials retained
STEP 5DERIVATIVES

Fisher information, one derivative at a time

Fisher information quantifies how much a trial tells you about a parameter. It is based on derivatives: how much does the certainty equivalent change when you tweak α, λ, γ+ or γ−?

179B
derivatives computed, central differences
15,000
CPU-hours on an HPC cluster
STEP 6

D-optimal sequential selection

We don't just pick random trials. We use an algorithm that selects trials one by one.

  1. 1Pick the trial that maximises information about the parameters, given what we already know
  2. 2Add it to the set. Update the information matrix.
  3. 3Repeat, towards a target of 300. Keeping the set balanced stopped it at 286.

The 286 trials ran in the first pilot, each participant in one condition. For the second pilot, in both conditions, the set grew into a pool of 403. From that pool the same selection kept the final 200 trials, with at least five trials at each of the 19 probability levels. Every participant plays these 200 trials twice: once with a person, once with a lottery.

286
trials in 2025, first pilot
403
trial pool, second pilot
200
final trials, played in both conditions
The result

A set of trials that is the most efficient way we know to estimate Prospect Theory parameters with this design

Precision estimates

The Fisher information matrix tells us how precisely we can estimate each parameter. Precision is measured by standard errors (SE). Smaller is better. These are the standard errors of the 2025 set of 286 trials.

αoutcome sensitivity
0.0147
excellent
γ+weighting, gains
0.0296
very good
γ−weighting, losses
0.0538
good
λloss aversion
0.0761
acceptable

Validation

We didn't just trust the maths. We ran simulations to check that it works.

502 SIMULATED AGENTS
Parameter recovery
Agents with known parameters played our game, then we tried to recover those parameters from their choices. Every fit converged, with moderate errors: about 20 to 35% of each parameter's range.
NEXT Building the experiment itself
MAKING IT REAL →
← PHASE 02The Hypothesis