Finding the gap: neuromorphic motion-vision in dense environments

The study addresses how animals, particularly flying insects and birds, navigate safely through dense, cluttered environments while avoiding collisions, crossing gaps, and maintaining a course. Optic flow (OF), the apparent motion of the environment on the retina during translation, provides primary information for flight control, relating to both self-motion and distance to objects and, thus, to time-to-collision. Multiple mechanisms have been proposed to transform OF into collision-free movements, including left-right OF balancing, global integration of motion, foreground-background OF contrast, optimized spatial sensitivity, and learned associations between active vision and object size. The purpose here is to test whether a single mechanism—steering toward regions of low apparent motion—can account for safe travel across diverse scenarios. The authors use a closed-loop neuromorphic approach inspired by the fruit fly motion pathway (T4/T5 elementary motion processing and downstream integration) to model and evaluate collision-free navigation. They build a robot that uses an event-based camera and a spiking neural network (SNN) implementing spiking elementary motion detectors (sEMDs) and an inverse soft winner-take-all (WTA) selection of low OF directions. They evaluate obstacle avoidance, gap crossing, and corridor centering in simulations and real-world experiments, aiming to provide a parsimonious, biologically inspired mechanism explaining insect behavior and enabling efficient autonomous navigation.

The paper situates its approach within extensive work on insect flight control based on optic flow. Prior strategies include OF balancing across eyes for centering in corridors, integrating motion across the visual field, using OF contrast between foreground and background, optimizing spatial sensitivity, and learning associations between self-induced motion and object size. Studies in controlled tunnels and obstacle fields have dissected insect behaviors such as speed modulation with tunnel width or environmental clutter, gap preference (larger gaps chosen), and saccadic gaze strategies. Neuromorphic sensing (event-based cameras) and spiking processing are well-aligned with asynchronous, distributed visual processing in insects. Prior robotics work has used frame-based cameras or mixed systems; few fully neuromorphic closed-loop systems exist. The sEMD model builds on correlation-based elementary motion detectors and previous event-driven implementations. The authors highlight gaps in unifying mechanisms that handle obstacle avoidance, gap selection, and centering with one parameter set and fully spike-based sensing-to-actuation pipelines.

The system comprises a fully neuromorphic sensing and control loop. Visual input: an event-based camera (DVS PAER128 in real-world; a Gazebo event-based camera model in simulation) providing asynchronous ON/OFF events when log-intensity changes exceed a threshold. Motion sensing: arrays of spiking Elementary Motion Detectors (sEMDs) tuned to horizontal motion, each built from two macropixels (2×2 pixels), a spatio-temporal correlation (SPTC) layer to remove noise by requiring spatiotemporal coincidences within a 20 ms window, and a Time Difference Encoder (TDE) that converts time-to-travel between adjacent pixels into a burst of spikes (rate and interspike interval encode motion). The sEMDs approximate fly T4/T5 elementary motion processing and produce a 2D retinotopic OF map. Integration and dimension reduction: an ONSET stage reduces 2D to 1D horizontal map and suppresses redundancies; an Integrator (INT) population collapses each column to one neuron, yielding a 1D OF map aligned with azimuth. Decision mechanism: an inverse soft WTA network receives inhibitory input from INT (so high OF inhibits that direction) and Poisson background excitation to ensure a winner even with weak input, effectively selecting directions of minimal OF (putative gap/open space). Lateral excitation and a global inhibitory (GI) neuron enforce competition. If no winner emerges within ~220 ms of an intersaccade (forward translation), an Escape Turn (ET) neuron triggers a large U-turn via GI-mediated suppression, serving as a fallback when no gap is found. Motor control: a Motor (MOT) population translates the inverse WTA winner (or ET) into turn direction and duration using pulse-width modulation; left and right motors are mutually inhibitory to enforce exclusivity. Turning speed is constant; turn angle scales with winner’s distance from visual center. Saccadic control: during turns (rotational OF), the MOT layer suppresses visual input and inverse WTA (saccadic suppression) to prevent selecting new directions without depth information. Speed control: an Optic Flow Integrator (OFI) modulates forward intersaccade speed inversely with global OF to slow in cluttered environments (v_ints = 1 − 0.001 × f_ofi). Experimental setups: (1) SEMD characterization with real event-based recordings of square-wave gratings (20° wavelength) moving at 0.1–10 Hz under varied illumination (5–5000 lux) and contrast; simulated on NEST and SpiNNaker; compared to Drosophila T4/T5 tuning. (2) Simulation experiments (Neurorobotics Platform with Gazebo robot): corridor centering across multiple widths; cluttered arenas with obstacle density up to 38%; gap selection between two gaps of varying sizes; 128×40 pixel camera, 140° FOV, 200 Hz refresh, ≤1000 events per update, robot 20×20×10 cm, inter-ommatidial angle ~1.1°. Runs lasted 2–6 h real time, repeated 10–79 times per condition; collisions defined by overlap with obstacles. (3) Real-world corridor experiments: differential-drive robot (2.3 kg) with PAER128 DVS, SpiNN-3 running a simplified network (constant speed, no OFI, no ET), and FPGA AERnode for motor PWM; tested in narrow (~30 cm) and wide (~50 cm) corridors with high-contrast walls; control condition without visual input.

• sEMD tuning: Preferred-direction responses show a bell-shaped velocity tuning peaking at 5 Hz (≈100 deg s−1), with low null-direction response, qualitatively matching Drosophila T4/T5 tuning (flies peak ~1 Hz in walking state). Robust across illumination (5–5000 lux) and contrasts (≈35% relative contrast for 50% response).
• Corridor centering (real world): In a wide corridor (~50 cm), 9/10 runs centered well; 1 early crash; one run included a 360° turn near the end. In a narrow corridor (~30 cm), no direct wall crashes; occasional slight touches; a left bias attributed to slight camera-robot misalignment. Control without visual input: crashed into wall in 9/10 runs; the remaining run collided halfway, confirming vision drives centering.
• Corridor centering (simulation): Velocity increases approximately linearly with tunnel width. Measured speeds (a.u./s): 1.25 ± 0.13 (13.3 a.u.), 1.62 ± 0.64 (16.6 a.u.), 2.00 ± 0.83 (20 a.u.), 2.31 ± 0.93 (23.3 a.u.). The agent remains near center, with deviation increasing with width, consistent with blowfly data.
• Cluttered environments (simulation): Across obstacle densities 0–38% (0.05 objects per square a.u.), mean success rate 97% without collisions. At low densities (<5%), decisions are more random due to weak sensory drive; at higher densities, the robot follows locally low-density paths. Collisions occur mainly with high densities due to reaction time and a rear blind spot after strong turns. Speed modulation via global OF significantly reduces collisions; with fixed speed, collisions increased notably above ~24% density.
• Gap selection (simulation): When presented two gaps, the inverse WTA more often selects the larger gap; gap-crossing probability increases with gap width. Smaller arenas increase crossing probability (larger relative gap size in FOV). The agent tended to cross gaps ≥5× its body width; smaller gaps often not selected due to insufficiently low apparent motion.
• System performance and efficiency: The fully spike-based sensing-to-actuation pipeline operates in real time at 1–4 W total (DVS128: ~23 mW; SpiNN-3: ~1–3.8 W; AERnode: ~30 mW), substantially more energy-efficient than comparable GPU approaches. A single parameter set handled corridor centering, cluttered obstacle avoidance, and gap crossing.

The findings support the hypothesis that a single, parsimonious mechanism—steering toward regions of minimal apparent motion—can underlie multiple aspects of insect-like navigation: centering in corridors, meandering through clutter while avoiding obstacles, selecting larger gaps, and speed modulation in response to environmental clutter. By approximating the fly’s elementary motion processing (T4/T5) with sEMDs and integrating OF signals via an inverse soft WTA, the agent selects directions with longest time-to-contact without explicitly balancing left-right OF, offering an alternative explanation for centering behavior. The model reproduces several insect behaviors, including saccadic gaze control with intersaccadic OF-based depth extraction and speed reduction in clutter. The strong improvement in success rate (from ~85% to ~97%) when using OF-based speed control underscores the importance of global OF feedback. Differences from insects (e.g., minimum passable gap width) likely arise from the agent’s reliance solely on OF and absence of goal-directed components or brightness cues that insects may integrate. The model also proposes a plausible circuit-level switch (ET/GI-WTA) for exploratory turns when no gap is found, offering a testable hypothesis about insect neural control. Overall, the work demonstrates that a fully neuromorphic, closed-loop system can capture key principles of biological obstacle avoidance and suggests new experiments (e.g., disambiguating OF balancing vs. minimal OF steering through manipulated visual environments).

This work introduces a fully spike-based, closed-loop neuromorphic system that enables robust collision-free navigation in dense environments using a single mechanism: selecting directions of minimal apparent motion. The approach, inspired by insect motion vision (sEMDs approximating T4/T5) and implemented with an inverse soft WTA, achieves corridor centering, gap selection, and obstacle avoidance in clutter without parameter retuning and with low power consumption. The results offer a working hypothesis for insect navigation strategies and demonstrate the engineering potential of neuromorphic hardware for edge robotics. Future directions include: integrating goal-directed navigation to combine obstacle avoidance with route following; incorporating additional sensory cues (e.g., brightness cues within gaps, radar) to handle low-contrast surfaces; testing in more natural, unstructured environments; and experimental work to distinguish minimal OF steering from OF balancing in insects, potentially via virtual reality manipulations.

• Afforded field of view (140°) leaves a rear blind spot; after large turns, unseen obstacles may cause immediate collisions in simulation.
• The event-based camera struggles with low-contrast, textureless surfaces; detection of such walls may require prediction from edges or complementary sensors (e.g., radar).
• The real-world robot used a simplified network (constant speed, no OFI/ET), limiting direct comparability with the full simulation pipeline.
• The agent lacks goal-directed behavior; consequently, it exhibits search-like trajectories in clutter and is conservative in gap crossing (typically ≥5× body width), unlike bees combining goal cues and brightness information.
• Simulation camera is a periodically sampled event emulator (Gazebo) rather than a fully asynchronous physical sensor for closed-loop trials; temporal resolution capped at 5 ms per update and event cap per cycle may constrain performance.
• Slight camera-drive misalignment observed in real-world corridor tests introduced a lateral bias.
• Environmental simplifications (e.g., removing ground texture from FOV) were applied to reduce spurious events.