# Experiment 4: exploratory RL, mechanism and constitutional-market pilot

**Date:** 2026-10-31. **Status:** DEVELOPMENT ONLY, five synthetic independent seeds (11–15); no live vendor calls, real human briefs or observed payment calibration. A frozen prereg protocol exists but the pilot exposed two implementation issues corrected with separately documented amendments. These numbers must **not** be reported as confirmatory tests of H1–H7.

**Interpretation:** regret is 100 × (infeasible ex-post flexible-capital welfare minus realized welfare) / infeasible benchmark. Smaller is better. The baseline comparison includes the optional common ex-ante price guard and the equal-budget exploration floor (Amendments 01 and 02).

| Arm | Mean regret (%) | SE (pp) | Mean markup | Discovery delay (periods) | Average capital sorting (Spearman) |
|---|---:|---:|---:|---:|---:|
| MKT-RL | 16.142 | 1.805 | 0.391 | 2.20 | 0.510 |
| PLAN-RL | 20.033 | 2.602 | 0.068 | 4.80 | 0.479 |
| HYB-C | 21.880 | 2.576 | 0.029 | 4.80 | 0.484 |
| COLL-RL | 13.554 | 1.869 | 0.422 | 2.20 | 0.592 |
| CONST-RL | 13.535 | 1.825 | 0.414 | 2.20 | 0.593 |

## Matched mean comparisons (uncorrected, descriptive)

- **H1 market minus planner:** -3.891 percentage points regret (paired SE 2.208); a negative difference is lower regret.
- **H2 collusion minus competition:** -2.588 percentage points regret (paired SE 1.285); a negative difference is lower regret.
- **H3 constitution minus collusion:** -0.020 percentage points regret (paired SE 0.084); a negative difference is lower regret.

## Hybrid C empirical mechanism certificate

- Allocation feasibility residual: 2.86e-06.
- Mean sampled unilateral deviation regret: 0.0268 utility units, versus 3.4111 for zero-transfer proportional allocation.
- Maximum sampled deviation regret: **6.186** utility units (large tail warning). This is not a strategyproofness proof.
- Training uses independent synthetic type profiles, not seed 11–15 utility observations. Distinct **mechanism-validation** profiles are used for the certificate, but economic evaluation remains developmental.

## Stress contrasts H5–H7

- H5_endogenous_no_audit: regret 18.723%, true consumer-principal net utility 824.50, mean absolute log-report distortion 0.150, capital sort 0.457.
- H5_honest_no_audit: regret 18.009%, true consumer-principal net utility 833.01, mean absolute log-report distortion 0.000, capital sort 0.458.
- H5_endogenous_audit: regret 18.788%, true consumer-principal net utility 824.50, mean absolute log-report distortion 0.150, capital sort 0.457.
- H6_no_drift: regret 13.930%, true consumer-principal net utility 905.78, mean absolute log-report distortion 0.000, capital sort 0.522.
- H6_high_drift: regret 17.714%, true consumer-principal net utility 819.76, mean absolute log-report distortion 0.000, capital sort 0.522.
- H6_rebrief: regret 13.766%, true consumer-principal net utility 897.15, mean absolute log-report distortion 0.000, capital sort 0.531.
- H7_flexible: regret 14.293%, true consumer-principal net utility 910.95, mean absolute log-report distortion 0.000, capital sort 0.503.
- H7_irreversible: regret 17.936%, true consumer-principal net utility 831.30, mean absolute log-report distortion 0.000, capital sort 0.509.

**Adverse/null results:** Collusion raises the modeled markup but may improve welfare by counteracting costly overproduction in this pilot. The constitutional audit rule barely changes net surplus relative to collusion, despite substantial audit and slash volume. Report learning did not measurably reduce report distortion in the adaptive-audit stress cell. These may be structural failures of the simplified policies, not general economic effects. A handful of seeds cannot support causality claims.

## Critical design qualifications

1. Online tabular action learners are genuine RL but not state-of-the-art deep RL, and their discrete policies are deliberately simple. Common bandit action lists and ex-ante price bounds constrain all arms. The initial uncapped/unguided variants remain available via `forecast_cap_multiplier=0` and `exploration_floor_interval=0`.
2. The RegretNet-style welfare-plus-regret architecture differs from the original auction-revenue objective and employs a **finite** deviation grid. Payments redistribute utility; Hybrid C budget shortfalls are explicitly logged. The learned mechanism does not optimize production or capital transfers intertemporally.
3. The planner's audit policy learns a crude reward tied to penalties and audit cost; it is not a calibrated optimal enforcement model. Producer collusion is a shared RL withholding policy, **not** evidence of coordination by actual commercial LLMs. Producer and consumer utility accounts are not identical to aggregate social welfare because of transfers, budget gaps and ownership.
4. Discovery success is stochastic, search slots are equalized, and demand is not advertised in advance; the amended common exploration floor can affect results. No observation of true *current* tastes changes current production (regression tested). The ex-post capital benchmark is infeasible and not a counterfactual planner.
5. **No live LLM claims:** adapters support OpenAI, Anthropic and Google only when explicitly configured, budget-approved and runnable on an internet-enabled host. This local run made zero provider calls. No genuine human-briefing trial or transaction calibration has been performed.
6. Prereg H1–H7 and 320 factor-grid cells are generated, but the 200-seed holdout and whole factor grid are **not executed**. The registered hypotheses must be tested with new holdout data after a new freeze of architecture and hyperparameters.

## Replication

```bash
pip install -r requirements-phase2.txt
python -m unittest discover -s tests -v
python sim/experiment4.py --manifest
python sim/experiment4.py --seeds 5 --train-steps 70
python sim/contrasts4.py
python charts/make_experiment4_chart.py
python sim/grid4.py --cell-start 0 --cell-count 4 --seed-start 11 --seed-count 2
python sim/calibration.py /path/to/observed_anonymized_payments.csv --period-minutes 60
```

### Reference

Paul Dütting et al. (2019), *Optimal Auctions through Deep Learning*, ICML. https://proceedings.mlr.press/v97/duetting19a.html