To stay, remain or leave: how verbal concepts as response options in political referendums such as the Brexit polls might bias voting outcomes

The paper investigates whether replacing neutral yes/no response options in referendums with verbal alternatives (e.g., remain/leave) introduces bias via framing effects. Using the UK’s 1975 (yes/no; verb in question: stay) versus 2016 (remain/leave) EU membership referendums as a case context, the authors focus on the evaluative differences among the verbs stay, remain, and leave and whether such verbs are suitable as response options. The study does not analyze the full Brexit campaign; rather, it probes implicit, potentially unconscious framing effects arising from the verbs’ etymology and conceptual structures. The central research question is whether near-synonymous verbs (stay vs remain) are evaluatively equivalent when contrasted with leave, and more broadly whether verbs should replace yes/no response options in political decision-making.

Theoretical and linguistic background highlights extensive evidence that small wording changes can strongly affect responses in decision-making and survey contexts. The Electoral Commission (2015) recognized potential unconscious impacts of wording but mainly relied on qualitative testing. Literature on framing effects in psychology, marketing, communication, and cognitive science shows language shapes perception and judgments. The authors contrast synchronic (current use) and diachronic (etymology) perspectives, arguing that etymologically anchored parameters in words can trigger framing effects even if speakers lack explicit etymological knowledge. Talmy’s force dynamics and status-quo framing suggest static (remain/stay) versus dynamic (leave) constructions can influence responses. Survey research on polarity and negation underscores complexity of verb framing. The paper details diachronic and sound-symbolic analyses: remain (Latin remanere) suggests being left behind in a location, with negative associations; stay (PIE *sta-) implies fixed position, stability, and positive sound-symbolic cues (/st-/); leave (OE læfan; PIE *leip ‘stick, adhere’) evolved toward dissolving contact and change of location, with sound-symbolic /l-/ signaling connection and potential phonological priming (BeLEAVE). The authors argue that these verbs activate sensorimotor-based concepts ([position], [location], [contact], [change of location]) that can elicit embodied framing effects. This motivates empirical testing via an IAT to capture implicit evaluative associations.

Design: Two Implicit Association Test (IAT) studies assessed implicit evaluative associations of stay and remain relative to leave. Participants were randomly assigned to one of two target concept pairings: remain vs leave or stay vs leave. Attribute categories were positive vs negative. Stimuli: Target concepts: remain, stay, leave (tested as pairs). Attributes: five positive adjectives (good, outstanding, fantastic, wonderful, excellent) and five negative adjectives (bad, dreadful, nasty, terrible, faulty), selected from prior IAT studies and matched morphologically. Procedure: A standard seven-block IAT was used (Greenwald et al., 2015). Participants classified stimuli via keyboard keys (‘e’=left, ‘i’=right). Blocks 1–2 trained target concepts and attributes separately (20 trials each). Blocks 3–4 combined one target with one attribute on each side (e.g., remain+good vs leave+bad) with 20 and 40 trials respectively. Block 5 reversed left/right key mapping. Blocks 6–7 repeated combined tasks with reversed pairings (e.g., leave+good vs remain+bad) with 20 and 40 trials respectively. Stimuli appeared centered on a light grey background in black, remained until response, with 400 ms ISI. Each participant completed 180 trials. Orders and pairings were counterbalanced across participants. Measures and scoring: The D-measure (Greenwald, Nosek & Banaji, 2003; Greenwald et al., 2009) was computed using the improved scoring algorithm: (1) delete trials >10,000 ms; (2) exclude respondents with >10% trials <300 ms; (3) compute pooled SDs for blocks 3&6 and 4&7; (4) compute mean correct latencies per block; (5) replace each error latency with block mean +600 ms; (6) recompute block means; (7) compute differences: block 6 – block 3 and block 7 – block 4; (8) divide each difference by its associated pooled SD; (9) average the two standardized differences to yield D. Positive D indicates target concept (stay or remain) is more positively associated (and leave more negative) relative to leave; negative D indicates the reverse. Participants: Study 1 (10/2019): Convenience sample recruited via professional networks across countries (emphasis on English-speaking regions). N=185; 121 female; mean age 33.42 (SD 14.63; range 17–83). English native speakers: 46. Conditions: remain vs leave (n=92); stay vs leave (n=93). Study 2 (5/2021): Recruited via Clickworker; English native speakers from UK and USA; approximate compensation €1 for 4–6 min. N=355; 209 female, 141 male, 5 diverse; age 18–70; UK n=181. Conditions: remain vs leave (n=182); stay vs leave (n=173). Ethics: informed consent obtained; Helsinki compliant; minimal risk; adults only. Analytic approach: Within each study, within-group one-sample t-tests tested mean D against 0 for each pairing. Between-group independent t-tests compared D across pairings. Robustness checks in Study 1 included analyses restricted to English L1 participants, two-way ANOVA (verb pairing × native language), and OLS regression controlling for age, gender, and native language.

Study 1 (N=185): - Remain vs leave: Mean D=0.189 (SD=0.486), t(91)=3.739, p<0.001, r=0.365. On average, remain was evaluated more positively (and leave more negatively) than leave, with medium effect; individual D ranged −1.159 to 1.077. - Stay vs leave: Mean D=0.515 (SD=0.331), t(92)=12.979, p<0.001, r=0.804. Stronger positive association for stay relative to leave; range −0.327 to 1.268 (fewer negative associations). - Between pairings: D was significantly higher for stay vs leave than for remain vs leave, t(183)=5.329, p<0.001, r=0.367 (medium effect). Thus, relative to leave, stay is more positively connoted than remain. - Robustness (English L1 only): Remain vs leave: D=0.264 (SD=0.527), t(25)=2.551, p<0.05, r=0.454; Stay vs leave: D=0.547 (SD=0.330), t(19)=7.428, p<0.001, r=0.862; Between pairings: t(44)=−2.097, p<0.05, r=0.301. - ANOVA: Main effect of verb pairing F(1,181)=19.48, p<0.001, ω²=0.092; no main effect of native vs non-native English (F(1,181)=0.15, ns) and no interaction (F(1,181)=0.20, ns). - OLS regression: Model F(4,180)=8.19, p<0.001, ω²=0.154; age b=−0.003 (ns), gender b=−0.051 (ns), English L1 b=0.125 (ns); verb pairing b=0.319, p<0.001. Study 2 (N=355; UK and USA English L1): - Remain vs leave: Mean D=0.526 (SD=0.410), t(181)=17.309, p<0.001, r=0.790; range −0.908 to 1.373; no difference UK vs USA (t(180)=−0.7114, ns). - Stay vs leave: Mean D=0.592 (SD=0.351), t(172)=22.213, p<0.001, r=0.861; range −0.391 to 1.319; no UK vs USA difference (t(171)=−1.134, ns). - Between pairings: Smaller difference than Study 1: t(353)=1.619, two-tailed p≈0.106 (reported as p<0.53 likely typographical; authors note directed hypothesis yields one-tailed p<0.027). Pattern aligns with Study 1: stay more positive than remain relative to leave. Overall: Both studies show that verb wording affects implicit evaluative associations: stay is more positively evaluated relative to leave than remain is, and both stay and remain are more positive than leave. Effects do not differ by native language, country (UK vs USA), age, or gender.

Findings indicate that near-synonyms stay and remain differ in evaluative structure when contrasted with leave, consistent with etymological and sound-symbolic framing: stay (from PIE *sta-) evokes stability, strength, and a fixed position; remain (from Latin remanere) suggests being left behind in a location; leave (from OE læfan) evolved toward separation from contact and change of location. The verbs’ sensorimotor grounding likely triggers embodied simulations, producing language-, age-, and gender-independent effects, explaining the absence of demographic differences. Variability between Study 1 and Study 2, including stronger negativity toward leave over time, may reflect contemporaneous socio-political connotations (e.g., Brexit) that can suppress or amplify diachronic associations. While the IAT provides relative measures and individual-level diagnosticity is limited, small implicit effects can translate into meaningful aggregate impacts in elections. The results question the neutrality of verb-based response options and suggest potential bias when replacing yes/no with verbs in referendums.

The study combines linguistic analysis with two IAT experiments (Study 1: N=185; Study 2: N=355) to demonstrate that wording of dichotomous response options can influence evaluations: stay is implicitly more positive relative to leave than remain is, and both are more positive than leave. Given such framing effects, verbs appear less suitable as neutral alternatives to yes/no response options in political referendums. The authors recommend limiting verbs to question formulation rather than using them as response anchors to reduce bias. Future research should compare sensorimotor vs non-sensorimotor concepts, simple vs complex predicates (e.g., hold/keep vs give away/let go), the role of negation (do vs do not), and verbs sharing etymology but differing in scalar orientation (decrease vs increase).

- The IAT captures relative, not absolute, evaluative structure; individual-level reliability and predictive validity can be limited, and links from implicit measures to overt behavior may be weak in some contexts. - Study 1 used convenience sampling with a likely academic skew; generalizability may be constrained. - Contemporaneous socio-political associations (e.g., Brexit) may have influenced implicit evaluations, potentially suppressing or amplifying etymology-based effects. - The study did not analyze the full referendum campaign or broader contextual messaging effects. - Data sharing is restricted due to EU data privacy constraints; replication with open datasets would strengthen confidence.