logo
ResearchBunny Logo
Multi-compartment Neuron and Population Encoding improved Spiking Neural Network for Deep Distributional Reinforcement Learning

Computer Science

Multi-compartment Neuron and Population Encoding improved Spiking Neural Network for Deep Distributional Reinforcement Learning

Y. Sun, Y. Zeng, et al.

Discover a groundbreaking approach to deep distributional reinforcement learning through a brain-inspired spiking neural network that mimics biological neuron structures. This innovative research, conducted by Yinqian Sun, Yi Zeng, Feifei Zhao, and Zhuoya Zhao, reveals extraordinary results in Atari game experiments, outperforming traditional ANN methods.

00:00
00:00
~3 min • Beginner • English
Abstract
Inspired by the information processing with binary spikes in the brain, the spiking neural networks (SNNs) exhibit significant low energy consumption and are more suitable for incorporating multi-scale biological characteristics. Spiking Neurons, as the basic information processing unit of SNNs, are often simplified in most SNNs which only consider LIF point neuron and do not take into account the multi-compartmental structural properties of biological neurons. This limits the computational and learning capabilities of SNNs. In this paper, we proposed a brain-inspired SNN-based deep distributional reinforcement learning algorithm with combination of bio-inspired multi-compartment neuron (MCN) model and population coding method. The proposed multi-compartment neuron built the structure and function of apical dendritic, basal dendritic, and somatic computing compartments to achieve the computational power close to that of biological neurons. Besides, we present an implicit fractional embedding method based on spiking neuron population encoding. We tested our model on Atari games, and the experiment results show that the performance of our model surpasses the vanilla ANN-based FQF model and ANN-SNN conversion method based Spiking-FQF models. The ablation experiments show that the proposed multi-compartment neural model and quantile fraction implicit population spike representation play an important role in realizing SNN-based deep distributional reinforcement learning.
Publisher
Not specified in provided text
Published On
Jan 18, 2023
Authors
Yinqian Sun, Yi Zeng, Feifei Zhao, Zhuoya Zhao
Tags
spiking neural networks
reinforcement learning
multi-compartment neuron
population coding
Atari games
Listen, Learn & Level Up
Over 10,000 hours of research content in 25+ fields, available in 12+ languages.
No more digging through PDFs, just hit play and absorb the world's latest research in your language, on your time.
listen to research audio papers with researchbunny