Revealing nonlinear neural decoding by analyzing choices

The paper investigates how the brain decodes information from neural populations when stimulus-relevant variables are entangled with nuisance variables that can abolish mean tuning and necessitate nonlinear computation. It distinguishes encoding (how stimuli drive neural responses) from decoding (how responses drive behavior) and asks whether the brain uses nonlinear information efficiently. The authors unify concepts of tuning curves, nuisance variables, nonlinear computation, and redundant population codes, and propose that if decoding is efficient, neural response statistics that are informative about a stimulus should also correlate with behavioral choices. They provide a testable quantitative relationship between neural fluctuations and choices to assess decoding efficiency, and apply it to macaque V1 during an orientation variance discrimination task.

Prior theories of nonlinear population codes often assumed specific covariability structures, underestimating redundancy in large cortical populations. Information-limiting correlations have been developed for linear codes to account for redundancy and bounds on information. Work on choice-related activity (choice probability/correlation) links neurons and behavior but is hard to interpret due to shared variability and unobserved populations; previous optimality tests exist for linear decoding. Empirical studies have shown stimulus-dependent internal noise correlations and nonlinear selectivity in various areas, and theoretical/empirical work on quadratic/nonlinear decoding (e.g., energy models, quadratic discriminant analysis) motivates extending optimality tests to nonlinear codes. The authors generalize information-limiting correlations and choice-correlation optimality relations to nonlinear statistics, addressing biological constraints of cortical expansion and redundancy.

Theory and modeling: The authors formalize a feedforward chain (s, v) → r → R(r) → ŝ where s is the task-relevant stimulus, v nuisance variables, r neural responses, R(r) nonlinear statistics, and ŝ the estimate used for behavior. They broaden 'signal' to include any stimulus-dependent statistic of responses (e.g., means, covariances), and distinguish internal noise (variability at fixed s, v) from external 'nuisance' variability (averaging over v at fixed s). They define nuisance correlations Corr(r|s) arising from averaging over v, which can carry signal when statistics (e.g., covariance) depend on s. They discuss nonlinear codes where information resides in nonlinear statistics (e.g., covariance, higher moments), illustrated by polarity/phase-invariant orientation coding and XOR examples. Decoding and optimality test: For linear codes, optimal decoding predicts a relationship between choice correlation and discriminability (C_opt = d′_k/d′). They generalize to nonlinear choice correlations C_R = Corr(R(r), ŝ) with optimal decoding ŝ = w ⋅ R(r) + c, yielding the key prediction C_R(opt) = d′_R/d′, where d′_R is the discriminability provided by statistic R. For coarse tasks and binary choices, they introduce the Normalized Average Conditional Choice Correlation (NACCC) to remove stimulus-induced covariation, with small correction factors for binarized behavior. They derive optimal weights in exponential-family models with nonlinear sufficient statistics and show cancellation of nuisance/internal covariance under optimal decoding. Information-limiting correlations: They generalize information-limiting correlations to nonlinear statistics, decomposing Γ = Cov(R|s) into an information-limiting component ε F′F′^T and residual Γ_0. This bounds estimator variance and Fisher information, explaining redundancy and why many nonlinear statistics can be informative yet yield measurable choice correlations. Simulation tests: They simulate networks with linear vs quadratic decoding, and generic neural networks (ReLU; polynomial nonlinearities) decoding stimuli encoded in polynomial sufficient statistics up to cubic, verifying that measured nonlinear choice correlations match optimal predictions even when the decoder’s internal nonlinearities differ from the tested basis. Experimental application: Two male rhesus macaques performed a 2AFC task: categorize drifting gratings as drawn from narrow (σ=3°) or wide (σ=15°) orientation distributions at 64% contrast. Utah arrays (96 electrodes) recorded V1 multiunit spike counts over a 500 ms window before target onset. Sessions with performance >0.7 (monkey 1) or >0.75 (monkey 2) were analyzed. They computed NACCC-based choice correlations for linear (r_i) and quadratic statistics (r_i^2, r_i r_j), and predicted optimal NACCC from measured discriminability (using total correlations Corr(R, s)), with binary-choice correction factors estimated via logistic regression. Shuffle controls: (1) remove internal noise correlations while preserving nuisance correlations (matching s, orientation, choice); (2) remove nuisance correlations by mismatching orientations across neurons while matching s and choice. Statistical significance was assessed with nulls from shuffled choices/stimuli and bootstrapping for variability.

• Central theoretical result: Under optimal decoding of nonlinear population codes, nonlinear choice correlations equal the ratio of discriminabilities, C_R(opt) = d′_R/d′. This holds for feedforward networks under natural nuisance variation and is robust to noise structure except for the information-limiting component.
• Redundancy and information-limiting correlations constrain total information and make optimality more detectable: even with many nonlinear statistics, choice correlations can be sizable due to redundancy when cortical populations expand inputs from smaller sensory populations.
• Simulations: Linear decoders produce strong linear choice correlations and negligible nonlinear ones; quadratic decoders show the opposite. Generic networks (ReLU) decoding polynomially encoded signals produce nonlinear choice correlations consistent with optimal predictions despite mismatched implementations.
• Experimental V1 data (two monkeys) in variance discrimination: • Neural responses contained linear and quadratic information about the category (variance). Quadratic statistics (r_i^2 and r_i r_j) carried substantial stimulus information beyond linear terms. • Measured NACCC (choice correlations) for linear and quadratic statistics were strongly correlated with predicted optimal NACCC (from discriminability), with slopes near 1 after binary corrections, indicating near-optimal nonlinear decoding. • Reported coefficients of determination (R^2) between measured and predicted NACCC: Monkey 1: 0.50 (linear), 0.33 (square), 0.12 (cross); Monkey 2: 0.61 (linear), 0.64 (square), 0.40 (cross). Correlations between the quantities were high (e.g., 0.76/0.65/0.53 for Monkey 1; 0.80/0.83/0.72 for Monkey 2). • Behavioral performance: ideal observer ~0.82 fraction correct; Monkey 2 achieved 0.76; Monkey 1, 0.74. • Decoding efficiency (slope α ≈ ε/σ_ŝ^2 from Eq. (9)): Monkey 2 near-optimal, 0.96 ± 0.04 (95% CI), not different from 1 at session level (p=0.26); suggests encoding-limited performance. Monkey 1 lower, 0.75 ± 0.08, significantly different from 1 (p<1e-6), suggesting downstream inefficiencies. • Shuffle controls: Preserving nuisance correlations while removing internal noise preserved the relationship between measured and predicted nonlinear choice correlations, while removing nuisance correlations abolished it. This indicates nonlinear information and decoding were driven by external nuisance variation, not stimulus-dependent internal noise correlations. • Direct tests found weak stimulus-dependent internal noise covariances; internal-noise-induced nonlinear choice correlations were not significant under the analyzed conditions.

The findings support the hypothesis that the brain decodes nonlinear population codes efficiently: neural statistics that best discriminate stimuli are most correlated with choices, in proportion to their discriminability. By extending information-limiting correlations to nonlinear statistics, the theory accommodates biological redundancy from cortical expansion and explains why optimality can be detected via choice correlations without recording all neurons. In macaque V1 during a variance-categorization task dominated by nuisance variation (orientation, contrast), nonlinear statistics of population activity carried task-relevant information and correlated with choice as predicted. Differences between monkeys suggest that performance limits may arise in encoding (monkey 2) or partially in downstream decoding (monkey 1). The approach clarifies that many stimulus-dependent correlations seen experimentally can arise from external nuisance variation rather than internal noise, and that optimal decoders focus on information-limiting fluctuations, canceling other shared variability. The method provides a practical tool to quantify nonlinear decoding efficiency from single-neuron or pairwise statistics, avoiding infeasible full-population recordings. The results are relevant for modest-complexity tasks requiring invariances (e.g., polarity/phase invariance, texture statistics) and suggest broader applicability to other sensory computations (motion energy, auditory localization). However, for high-dimensional natural tasks with complex nonlinearities, the approach may lack statistical power or suitable feature bases, as relevant information is distributed across many small-contribution statistics.

The paper introduces a unified theory connecting nonlinear encoding and decoding in neural populations, generalizing information-limiting correlations and deriving a simple optimality prediction: nonlinear choice correlations equal the ratio of discriminabilities. This yields an operational test of decoding efficiency using easily computed statistics. Applying the method to macaque V1 during orientation-variance categorization reveals that nonlinear information induced by nuisance variation is decoded near-optimally, providing direct evidence for efficient nonlinear computation in cortex. The work advances quantitative tools for relating neural population statistics to behavior and highlights the role of redundancy and nuisance variables in shaping neural codes. Future work should extend the framework to recurrent and spatiotemporal settings, develop richer nonlinear bases (e.g., deep neural features) for complex tasks, and test the approach across modalities and task structures, including conditions emphasizing internal-noise-driven variability.

• Assumes primarily feedforward processing and spatial statistics; recurrent dynamics and feedback are not explicitly modeled and may confound choice correlations.
• Optimality relations are cleanest for fine discriminations; coarse binary tasks require corrections (e.g., NACCC, scaling factors), adding complexity.
• Best suited to tasks of modest complexity with low-dimensional nuisance; for high-dimensional natural tasks, suitable nonlinear bases may be hard to identify and statistical power may be limited.
• Analysis focused on high-contrast trials; internal-noise effects might be stronger at lower contrasts or with different task demands.
• Multiunit recordings and partial population sampling limit direct assessment of full population information; method infers optimality indirectly.
• The approach may be insensitive to subtle, large-scale internal-noise patterns requiring simultaneous population recordings and complex decoders to detect.