An exact mathematical description of computation with transient spatiotemporal dynamics in a complex-valued neural network

Spatiotemporal dynamics are a powerful computational substrate used in various physical and biological systems, including neural networks and wave-based computations. However, the relationship between network structure and the computations performed through these dynamics remains poorly understood, especially in nonlinear systems where dynamics are difficult to control and predict. This paper addresses this gap by introducing a complex-valued neural network (cv-NN) that allows for the exact mathematical design of specific computations through its spatiotemporal dynamics. The use of a linear network with a nonlinear readout allows for a closed-form solution of the computation, providing fundamental insight into how spatiotemporal patterns can be used to perform complex tasks. The system's ability to generate and utilize transient spatiotemporal patterns, especially chimera states, is a key aspect of its computational power. This approach offers a significant advancement over existing nonlinear systems by enabling precise control and prediction of the system's dynamics.

Existing research has shown that spatiotemporal dynamics are leveraged for computation in various systems, ranging from neural networks performing sensory computations to optical or electromagnetic wave systems performing complex transformations. However, a precise mathematical understanding of these computations, especially in nonlinear systems, is lacking. The difficulty lies in mapping a single input to a single output through a specific dynamical trajectory. Previous work demonstrated spatiotemporal dynamics for computations like logic operations and speech recognition, but a complete mathematical description was absent. This paper builds upon recent work showcasing the hallmarks of canonical synchronization behavior in complex-valued systems, extending it to develop a framework for exact computation through spatiotemporal dynamics.

The researchers designed a complex-valued neural network (cv-NN) with linear dynamics and a nonlinear readout. The cv-NN's dynamics are governed by a differential equation: ẋ(t) = (iωI + K)x(t), where x(t) ∈ CN is the network state vector, ω is the frequency, I is the identity matrix, and K is a connectivity matrix incorporating coupling strength and phase-delays. The connectivity matrix K = ee^iA represents interactions between nodes, with e representing coupling strength, φ as a phase-delay, and A representing connection weights. The nodes are coupled in a one-dimensional ring with periodic boundary conditions, and connection weights decay as a power-law with distance. The computations are implemented using phase offsets between nodes. The entire computation is captured in a closed-form expression: o_k(t) = R_kD_tx(0), where o_k(t) is the decoder output, D_t represents the linear dynamics, R_k quantifies synchronization in a local network patch, and θ_k is a Heaviside function. To design computations, the inverse operator D^-1 is used to calculate the input required to evolve the network to a target state. A similarity measurement S quantifies the match between the target and actual phase patterns. The cv-NN is used to implement logic gates (e.g., XOR), short-term memory, and a symmetric-key encryption system. The short-term memory task involves holding one of eight items in memory for 3 seconds, with online updating demonstrated. The encryption system uses chimera states to encode letters, with the network structure and chimera alphabet as public information, and the frequency ω and initial conditions x(0) as the secret key. Finally, the researchers tested the decodability of cv-NN dynamics by biological neurons using intracellular recordings, injecting cv-NN dynamics as current into neurons and using neuronal spiking as a decoder. The mathematical analysis included the spectral properties of the network (eigenvalues and eigenvectors of the matrix K), particularly focusing on the effect of phase-delays on the eigenvalues and the network's ability to generate transient spatiotemporal patterns.

The study's key findings include: 1. The development of a complex-valued neural network (cv-NN) with linear dynamics and nonlinear readout that can perform sophisticated computations through transient spatiotemporal dynamics, including chimera states. This is significant because it demonstrates that complex computations can be achieved even with linear dynamics, which are typically considered too simple for such tasks. 2. The derivation of an exact, closed-form mathematical expression describing the complete computation performed by the cv-NN. This allows for precise design and prediction of the network's behavior. 3. The successful implementation of various computational tasks using the cv-NN, including logic gates (XOR), short-term memory with online updating capabilities, and a secure message passing system using a symmetric-key encryption scheme based on chimera states. 4. The experimental demonstration that biological neurons can successfully decode the spatiotemporal dynamics of the cv-NN, indicating the potential for seamless integration between artificial and biological neural networks. The results showed that the biological neuron successfully identified the stored item based on the location of the phase-coherent cluster in the cv-NN dynamics. This confirms the feasibility of using such networks in bio-hybrid computing systems. 5. The findings highlight the importance of phase delays in extending the window for bounded amplitude, enabling the creation of rich and transient spatiotemporal patterns suitable for computation. 6. The successful demonstration of a robust symmetric-key encryption system highlights the potential of the cv-NN for secure communication. The system demonstrated resistance against random attacks.

The findings address the research question by demonstrating that a complex-valued linear neural network can perform complex computations using transient spatiotemporal dynamics. The exact mathematical solvability of the system provides a powerful tool for understanding and designing computations in neural networks. The significance of the results lies in the potential to design highly adaptable and efficient bio-hybrid computing systems. The ability to seamlessly interface the cv-NN with biological neurons opens new possibilities for developing neuromorphic computing architectures and brain-computer interfaces. The study's findings challenge the conventional view that nonlinear dynamics are essential for complex computations in neural networks, offering a new perspective on network design and computational capabilities. Future research should explore the scalability of the approach to larger networks and more complex computational tasks. Exploring different network topologies and investigating the relationship between network structure, phase delays, and computational power would further advance this field.

This study presents a novel complex-valued neural network (cv-NN) capable of performing computations based on transient spatiotemporal dynamics, described by an exact mathematical solution. The cv-NN successfully implemented logic gates, short-term memory, and a secure message passing system. Moreover, the experimental results demonstrated that biological neurons can decode the cv-NN’s computations. This work provides a significant advance in understanding computation in neural networks and offers promising avenues for developing advanced bio-hybrid computing systems.

While this study demonstrates the potential of cv-NNs, certain limitations exist. The experiments involved relatively small networks. Scaling the cv-NN to larger networks and more complex tasks requires further investigation. The current study primarily focused on a specific network topology and connectivity pattern; exploring the impact of diverse network architectures and connectivity on computational capabilities is needed. Furthermore, the current encryption system is a proof-of-concept; more rigorous security analysis is needed to fully assess its robustness against advanced attacks.