Human-centred mechanism design with Democratic AI

The study addresses the challenge of value alignment in AI, where systems should behave in ways preferred by humans despite diverse and sometimes conflicting human values. The authors propose Democratic AI, a human-in-the-loop framework that trains AI to optimize a democratic objective: design policies that humans will prefer in a majority vote. The work situates itself within mechanism design—controlling the allocation of wealth, information, or power among strategic actors—and tests whether a deep reinforcement learning agent can design an economic redistribution mechanism that groups of incentivized humans prefer. Using an online public-goods-style investment game with unequal and equal endowments, the research question asks: can an AI discover a redistribution mechanism that promotes fairness and productivity and is preferred by a majority of human players?

The paper builds on mechanism design in economics and game theory, including classic treatments of public goods, distributive justice, and market design. It connects to AI value alignment and participatory approaches that incorporate human feedback into learning objectives. Prior experiments on public goods examine contribution thresholds, sanctioning, exclusion, and effects of heterogeneity in endowments and marginal per capita returns. Related AI work includes learning from human preferences and AI-driven policy design (e.g., tax policies). The authors extend these lines by optimizing policies directly for democratic preference (majority vote) and by comparing AI-designed mechanisms to canonical distributive philosophies: strict egalitarian, libertarian, and liberal egalitarian. They also contrast mechanisms trained with human-imitating virtual players versus rational utility-maximizing simulated players, informing when human cognitive modeling is necessary.

Study design: A multi-stage human-in-the-loop pipeline was implemented: (1) Acquire: collect human gameplay data in a 10-round investment game with varied endowments and redistribution mechanisms; (2) Model: train virtual human players (recurrent neural networks) to imitate human contribution behavior and a voting model to predict preferences; (3) Optimize: train a mechanism designer (deep RL with a Graph Network architecture) to maximize expected votes from virtual players in head-to-head elections; (4) Repeat: evaluate with new human participants, augment training data, and iterate.

Game: Groups of four players played 10-round blocks. Each round, player i received endowment e (head: 10; tails: {2,4,6,8,10} depending on condition) and chose integer contribution c to a public fund, retaining e−c privately. Public contributions were multiplied by r=1.6 and redistributed according to a mechanism that outputs fractional payouts y_i summing to 1 of the public fund. Players experienced two rival mechanisms in counterbalanced order (blocks 2 and 3) and then voted for their preferred one to replay in a bonus block (block 4). A tutorial block with equal redistribution by default preceded these.

Participants and ethics: Approved by HuBREC. Recruited via two crowdsourcing platforms (UK/USA). In Experiments 1–3, n=4,776 participants; pilot data included ~4,000 additional datasets. In Experiment 4, 61 trained human referees and n=244 new players. Sessions lasted ~20–30 minutes, with up to £8 compensation. Timeouts were replaced by bots (affected games excluded). Voting analyses used permutation tests preserving within-group covariance.

Experiments:

• Experiment 1 (n=756): Contributions under three baselines (strict egalitarian, libertarian, liberal egalitarian) and endowment configurations (10,2,2,2), (10,4,4,4), (10,10,10,10); illustrates behavioral richness under canonical mechanisms.
• Ideological manifold: Generalized baselines via parameters v (absolute vs relative payout mix) and w (self vs others weighting) to map continuous mechanism space. Two-player tournament among virtual players identified liberal egalitarian as a Nash equilibrium and strongest competitor.
• Experiment 2 (n=2,508): Head-to-head votes between AI-designed HCRM and each baseline across five endowment conditions: tails in {2,4,6,8,10}.
• Experiment 3 (n=736): HCRM vs a Rational Mechanism (RM) trained against rational players maximizing expected returns (no human imitation).
• Experiment 4: Human referees trained (~1 hour) to allocate payouts to maximize votes versus HCRM in condition (10,4,4,4); evaluation with new human players.

Virtual human players: LSTM-based networks imitated human contributions. Inputs per round: each player’s endowment, previous contribution, previous relative contribution, and payout (scaled). Architecture: linear(64,tanh) → LSTM(hidden=16) → linear(11) yielding logits for contributions 0–10, masked by endowment. Trained with cross-entropy plus entropy (0.1) and L2 (1e−5) regularization using Adam (lr 0.0004). ~30,000 updates with minibatch size 512; validated on a held-out dataset (n=384).

Voting model: Probability of voting for mechanism A over B was logistic in the difference of cumulative relative payouts across blocks, p(A)=σ(s·rpay_A−s·rpay_B) with s=1.4.

Mechanism designer: A memoryless Graph Network policy mapping current-round endowments, contributions, and relative contributions (12 inputs) to 4 redistribution weights via softmax. Two GN blocks with tanh activations (edge/node/global layers, 32 units). Permutation-equivariant over players; no recurrence; slot-equivariant and privacy-preserving inputs (distributions rather than identities).

Training the mechanism: Used Stochastic Computation Graphs to estimate policy gradients of expected votes, training with RMSProp (lr 0.0004, epsilon 1e−8, decay 0.99) for 10,000 updates. On each update: simulate two batches of 512 games/10 rounds with virtual players; pair HCRM against a competitor (liberal egalitarian from the metagame) to generate 2,048 votes per update. Tail endowments sampled from {2,3,4,5,6,7,8,10} to improve generalization. Surrogate objective incorporated log-probabilities of discrete player actions; mean-centering reduced gradient variance. Trained HCRM exported for deployment with humans.

Analysis: Per-round and per-block metrics included contributions, payouts, returns, surplus (sum of returns/sum of endowments), and Gini coefficients. Voting significance assessed with group-level permutation tests (10,000 shuffles). Additional interpretable analyses used “beach plots” mapping relative contributions to fractional payouts.

• Experiment 1 (behavior under canonical mechanisms): Under strict egalitarianism (equal redistribution), contributions declined over time (head player: 2 coins F9,540=2.51, P=0.023, η²=0.017; 4 coins F9,540=5.5, P<0.001, η²=0.041; 10 coins F9,599=47.27, P<0.001, η²=0.056). Under libertarianism (payout y_i=r·c_i), contributions increased over time (head: 2 coins F9,297=9.96, P<0.001, η²=0.062; 4 coins F9,234=9.55, P<0.001, η²=0.073; 10 coins F9,270=12.56, P<0.001, η²=0.013). Under liberal egalitarianism (relative to endowment), tail players increased contributions (2 coins F9,720=4.79, P<0.001, η²=0.025; 4 coins F9,509=15.74, P<0.001, η²=0.043) while head contributions remained flat (2 coins F9,234=1.84, P=0.139; 4 coins F9,297=0.62, P=0.601). Head players contributed less when others were less well-off under liberal egalitarianism (F2,377=14.10, P<0.001, η²=0.070); no effect under strict egalitarianism (F2,188=0.29, P=0.745).
• Ideological manifold tournament: Among a continuous family of mechanisms parameterized by v and w, liberal egalitarian emerged as the Nash equilibrium and strongest competitor to HCRM for virtual players.
• Experiment 2 (votes for HCRM vs baselines, n=2,508): HCRM won majority votes against strict egalitarian (513/776, 66.2%, P<0.001), libertarian (450/740, 60.8%, P<0.001), and liberal egalitarian (951/1,744, 54.5%, P<0.001). By endowment conditions, HCRM’s vote share ranged 56.0–67.0% vs strict egalitarian and 57.5–66.7% vs libertarian. Against liberal egalitarian, HCRM prevailed under equality (64.5%, P<0.001) and moderate inequality ((10,8,8,8) and (10,6,6,6): 54.5%, P<0.006), but not under the most unequal (10,2,2,2) condition (47.4%, P=0.897).
• Experiment 3 (HCRM vs RM, n=736): HCRM preferred overall (421/736, 57.2%, P<0.001). Under equal endowment, no difference (71/148, 47.9%, P=0.617). RM adopted a policy that heavily favored tails under inequality, causing head players to cease contributing, lowering group surplus relative to HCRM (t183=7.96, P<0.001).
• Mechanism properties and outcomes: HCRM is a progressive hybrid that reduces pre-existing disparities by rewarding contributions relative to endowment, while sanctioning free-riding by returning little unless players contribute roughly half their endowment. It achieved a favorable trade-off between productivity (surplus) and equality (lower Gini) compared with baselines. Beach plots reveal interpretable payout surfaces responsive to both head and tail contributions.
• Experiment 4 (HCRM vs trained human referees): Human players preferred HCRM (62.4%, P<0.001). Human referees were less willing to sanction low-contributing head players and less responsive to high relative contributions from tails (government × payout sextile interaction F2,128=5.541, P<0.005, η²=0.125).

The findings show that an AI mechanism designer can optimize for a democratic objective—majority preference—yielding a redistribution policy that incentivized contributions, addressed initial inequality, sanctioned free riders, and won majority votes across a range of endowment distributions. By learning directly from human behavior (via virtual human players) and optimizing expected votes, Democratic AI aligns policy design with revealed human preferences under incentive-compatible conditions.

This approach mitigates researcher-imposed value choices by using voting as the arbiter, but raises considerations about the tyranny of the majority and potential neglect of minorities. The authors note this risk and suggest augmenting objectives to protect minority interests, analogous to legal safeguards. The designed HCRM is readily interpretable due to architectural constraints (no memory, slot- and permutation-equivariance), enhancing transparency and privacy while still outperforming baselines and trained human referees. Comparisons with a mechanism trained against rational agents suggest human data are especially important under unequal initial conditions; rational models can approximate human preferences under equality but may fail under inequality where contribution dynamics and cognitive biases matter. Overall, the results support Democratic AI as a method for value-aligned mechanism discovery in social dilemmas.

The paper contributes a human-in-the-loop pipeline—Democratic AI—that leverages imitation learning and deep RL to design redistribution mechanisms preferred by human majorities. The discovered HCRM balances productivity and equality, redresses initial wealth imbalances, and sanctions free riding, outperforming canonical mechanisms, a rationally trained mechanism, and trained human referees in head-to-head votes. The mechanism is interpretable and privacy-preserving by design.

Future directions include: integrating explicit constraints to protect minority interests; extending to richer social mechanisms (e.g., dynamic, multi-issue, or multi-round memory-based policies) while maintaining interpretability; testing across broader populations and contexts; comparing experience-based versus description-based deployment; and exploring real-world policy co-design workflows that keep policy implementation with human decision-makers while using AI to aid policy development.

• Democratic objective risks favoring majority preferences over minority welfare (potential tyranny of the majority); mitigation may require augmented objectives or constraints.
• Participant pool from online platforms in the UK/USA may limit generalizability to other cultures or demographics; uniqueness of participants could not be guaranteed across months.
• Mechanism was constrained to be memoryless and slot-equivalent; while aiding interpretability and privacy, this may preclude beneficial history-dependent policies.
• Players learned mechanisms by experience rather than description; behavior might differ if mechanisms were verbally specified.
• Virtual human players must generalize beyond observed data to new mechanisms; modeling errors could bias optimization. Under full equality, rational-player–trained mechanisms performed comparably, suggesting model dependence on context.
• Some procedural factors: timeouts replaced by bots (affected games excluded), and data collection/analysis were not blinded to conditions.
• Results pertain to a specific public-goods investment game with fixed r=1.6 and group size; outcomes may differ under other game structures or parameters.