Neurons with graded response have collective computational properties like those of two-state neurons

The paper investigates whether the collective computational properties—especially content-addressable memory (CAM)—observed in networks of idealized two-state McCulloch-Pitts neurons persist when neurons have more biologically realistic graded (sigmoidal) input-output responses and dynamical (RC) time constants. Previous stochastic, asynchronous-update models with binary outputs differ significantly from real neurons and electronic analog circuits, prompting skepticism about their biological and physical realizability. The study aims to eliminate two major simplifications (binary outputs and stochastic state updates) by formulating a continuous, deterministic network model and examining whether it retains the associative memory and stability properties of the original model. Demonstrating this would support the relevance of these collective computations to neurobiology and to realizations in analog electronic hardware.

The work builds directly on Hopfield’s earlier models of associative memory in networks of binary neurons with symmetric interconnection matrices (1–3), which established energy (Lyapunov) functions guaranteeing convergence to stable states. It contrasts asynchronous stochastic updating with synchronous systems that may exhibit different collective dynamics (5, 6). The paper situates its stability analysis within broader results on stability and dynamical systems across chemistry, circuits, and biology (8–12). It also notes that more complex energy formulations used in constraint satisfaction and optimization (e.g., simulated annealing and Markov random fields; 13, 14) can produce interior minima in continuous spaces without discrete counterparts, highlighting the special simplicity of quadratic interaction terms used here.

• Original stochastic model: Neurons i take binary values V_i ∈ {V_i^−, V_i^+} (often 0/1). Total input H_i = Σ_j T_ij V_j + I_i. Asynchronous stochastic updates occur: each neuron samples its input at random times with mean rate ω and switches according to a threshold rule with thresholds U_i. An energy function E = −(1/2) Σ_i Σ_j T_ij V_i V_j − Σ_i I_i V_i + Σ_i U_i V_i strictly decreases under updates for symmetric T with zero diagonals, ensuring convergence to fixed-point attractors (memories).
• Continuous, deterministic model: Each neuron has a graded, monotone sigmoidal output V_i = g_i(u_i), with asymptotes V_i^− and V_i^+. The input u_i evolves via RC dynamics: C_i (du_i/dt) = Σ_j T_ij V_j − u_i/R_i + I_i with u_i = g_i^{-1}(V_i). The same equations describe an analog circuit of fast nonlinear amplifiers resistively interconnected (T_ij ~ 1/R_ij), with input resistances/capacitances included in R_i and C_i.
• Lyapunov (energy) function for continuous model: E = −(1/2) Σ_i Σ_j T_ij V_i V_j + Σ_i (1/R_i) ∫^{V_i} g_i^{-1}(V) dV + Σ_i I_i V_i. For symmetric T, dE/dt = −Σ_i C_i g_i^{-1′}(V_i) (dV_i/dt)^2 ≤ 0, with equality only at equilibria, proving convergence to stable fixed points (no oscillations/chaos) when amplifiers are fast relative to RC dynamics.
• Correspondence of stable states: Analyze simplified case with V_i^− < 0 < V_i^+, g_i(0)=0, I_i=0, T_ii=0. Introduce gain scaling V_i = g(λ u_i), u_i = (1/λ) g^{-1}(V_i). As λ → ∞ (steep gain), the integral term becomes negligible; minima/maxima of E coincide with those at the corners V_i ∈ {−1, +1} of the hypercube, matching the discrete model’s stable states. For finite λ, minima shift inward; as λ decreases, minima can annihilate with saddles, reducing their number; for small λ, only the origin remains.
• Illustrative example: Two-neuron system (T_12=T_21=1, λ=1.4, g(u)=(2/π) tan(π λ u / 2)) shows two stable minima near opposite corners; energy contours depict gradient-like flow.
• Hardware mapping: Presents an amplifier-resistor network realizing Eq. 5, including handling of inhibitory/excitatory signs via inverting stages, and conditions on symmetry and amplifier speed for guaranteed convergence.
• Extension to action potentials: Model spiking as a stochastic point process with rate F g(u). Each spike from neuron j injects quantal charge V_0 T_ij to neuron i’s capacitance. Derive a master equation for the joint density P(u_1,...,u_n,t). For small V_0, expand to obtain a Fokker–Planck-like equation with drift reproducing the continuous deterministic dynamics in the diffusionless limit (V_0 → 0, F → ∞, FV_0 constant), and a diffusion term capturing spike-induced noise. The noise lacks detailed balance and is not equivalent to thermal noise.

• A continuous, deterministic network of neurons with sigmoidal input-output functions and RC dynamics possesses a Lyapunov (energy) function for symmetric T, guaranteeing monotone decrease of energy and convergence to fixed points (stable memories), mirroring the original stochastic binary model.
• There is a direct mapping between stable states of the graded-response model and the binary model: in the high-gain limit (λ → ∞), there is a one-to-one correspondence; for finite gain, stable points shift inward from hypercube corners; as gain decreases further, some minima vanish via saddle-node collisions, reducing the number of memories, and for very low gain only the trivial state remains.
• The quadratic interaction term −(1/2) Σ_ij T_ij V_i V_j is key to the correspondence; more complex energy forms can admit interior minima without binary counterparts.
• Symmetric T and sufficiently fast amplifiers (relative to RC time constants) preclude oscillations and chaos; approximate symmetry is sufficient in practice.
• An explicit analog electronic implementation using operational amplifiers, resistive interconnections, and capacitive inputs will function as a CAM under the same symmetry and speed conditions.
• A master-equation/Fokker–Planck formulation incorporates action potentials as quantal stochastic events; in the limit of infinitesimal quanta with high rates (FV_0 constant), the deterministic continuous model is recovered; finite quanta introduce diffusion-like noise that does not correspond to thermal equilibrium (no detailed balance).

The work addresses whether associative memory and related collective computations survive when neurons have graded outputs and dynamical integration, as in biology and analog circuits. By constructing a Lyapunov function for the continuous, deterministic system with symmetric connections, the study shows that convergence to stable memories persists without relying on binary states or stochastic updates. The mapping between stable states of the two models clarifies that the original stochastic binary network remains a valid and efficient proxy for analysis and simulation, especially at high gain, while the graded model captures more biological realism. The hardware correspondence demonstrates practical realizability in analog circuits, with concrete conditions for stability. Incorporating action-potential-based stochasticity through a master equation establishes how spike noise perturbs, but does not fundamentally negate, the minimum-seeking behavior, enabling future quantitative analyses of robustness and memory lifetimes under noise.

The paper establishes that networks of neurons with graded (sigmoidal) responses and RC dynamics exhibit the same core collective computational properties—especially content-addressable memory and convergence to stable states—as the earlier stochastic binary Hopfield networks, provided the interconnection matrix T is symmetric. There is a clear correspondence between the stable states of both models, exact in the high-gain limit and approximate at finite gain. The analysis supports the feasibility of implementing CAM in analog electronic circuits and strengthens the plausibility of such mechanisms in biological systems. Future directions include quantitative study of the effects of spike-induced (quantal) noise on stability via the derived master/Fokker–Planck equations, exploration of antisymmetric components in T to realize sequence generation in the continuous model, and extending the framework to more detailed neuron models and dendritic computations.

• Exact convergence guarantees require symmetric T and fast amplifiers relative to RC dynamics; deviations from symmetry may degrade performance or introduce complex dynamics.
• The graded-response model abstracts neuron behavior to a static sigmoid and linear summation; detailed nonlinear dendritic processing and conductance-based dynamics are not modeled.
• Finite gain can reduce the number of stable memories relative to the binary model; very low gain collapses memory capacity to a trivial state.
• The action-potential extension treats spike timing probabilistically via rate coding; precise spike timing effects are not captured. The resulting noise does not satisfy detailed balance and lacks a temperature interpretation, complicating equilibrium analysis.
• Biological applicability remains tentative; propagation delays must be short relative to integration times, and the degree to which real neural circuits meet symmetry and speed assumptions is uncertain.