Automatic design of stigmergy-based behaviours for robot swarms

Stigmergy, a form of indirect communication where individuals modify their environment to influence others, is prevalent in nature, notably in social insects using pheromone trails. This mechanism has inspired algorithms in various fields, including swarm robotics. However, designing stigmergy-based collective behaviors for robot swarms has traditionally been a manual, time-consuming, and expertise-dependent process. This research addresses this challenge by proposing an automatic design method, offering a repeatable and efficient approach to creating complex swarm behaviors. The importance of this lies in overcoming the limitations of manual design, which is inconsistent in quality and difficult to generalize across different robotic platforms or missions. The success of automatic design would significantly advance swarm robotics by enabling the rapid prototyping and deployment of sophisticated collective behaviors for various tasks and environments.

Existing research in swarm robotics explores various approaches to implementing pheromone-based stigmergy. These include using smart environments with RFID tags or virtual pheromones projected onto the ground, and more recently, approaches involving robots physically depositing artificial pheromones (e.g., alcohol or wax). However, these methods often face challenges such as high costs, limited applicability, safety hazards, or the need for environment preparation. While deep reinforcement learning has shown promise in designing collision avoidance behaviors using virtual pheromones, it primarily remains confined to simulations. The scarcity of formal design methods for stigmergy-based behaviors further highlights the need for a more systematic and efficient design process. Current practice predominantly relies on manual design through trial-and-error, a process that is both time-consuming and heavily reliant on the designer's experience.

The proposed approach, Habanero, is an automatic off-line design method belonging to the AutoMoDe family. AutoMoDe reformulates the design problem as an optimization problem solved in simulation before deploying robots in the real environment. Habanero's solution space consists of control software instances created by combining pre-existing software modules (low-level behaviors and transition conditions) within a modular architecture (probabilistic finite-state machines). The Iterated F-race algorithm is used to search this solution space, optimizing the control software to maximize performance. The target platform is an e-puck robot augmented with a UV-light module for laying pheromone trails on photochromic material and an omnidirectional camera for pheromone detection. ARGOS3, a multi-robot simulator with a custom library for pheromone trail simulation, is used in the optimization process. Four benchmark missions – AGGREGATION, DECISION MAKING, RENDEZVOUS POINT, and STOP – are used to evaluate Habanero's performance. This is compared against three alternatives: EvoPheromone (a neuroevolutionary method), Human-Designers (manual design by human experts), and Random-Walk (a baseline behavior). Each method generates 10 instances of control software for each mission, then are evaluated both in simulation and with physical robots. The performance of each method is analyzed using boxplots and a Friedman rank sum test to assess statistical significance across the missions.

Habanero successfully designed effective stigmergy-based collective behaviors for all four missions. Statistical analysis shows that Habanero consistently outperformed EvoPheromone and, in many cases, also Human-Designers, particularly in real-robot experiments. In AGGREGATION, Habanero's robots avoided pheromone saturation by laying trails intermittently, leading to better aggregation than manual designs which failed in real-world application due to local pheromone traps. In DECISION MAKING, Habanero correctly selected the higher-reward region and maintained aggregation even after visual cues disappeared. In RENDEZVOUS POINT, Habanero robots efficiently used random walks to locate the target, whereas EvoPheromone’s strategy, effective in simulation, failed in real-robot tests. In STOP, Habanero robots effectively used stigmergy to halt the swarm upon detecting the signal. Notably, although Habanero's modules are mission-agnostic, the resulting interaction strategies were mission-specific, adapting to the requirements of each task. The Friedman rank sum test confirmed Habanero's superior overall performance across all missions compared to the alternatives.

Habanero's success demonstrates the feasibility of automating the design of pheromone-based collective behaviors in robot swarms. This approach can significantly simplify the development of swarm robotic systems, reducing time and expertise requirements and improving the consistency of results. The ability to generate mission-specific strategies from mission-agnostic modules highlights the power of the optimization process to adapt to diverse tasks. The superior performance of Habanero compared to both manual design and neuroevolution suggests the importance of the structured modular approach in bridging the reality gap. The study's results encourage further research in automating the design of more complex behaviors by leveraging finer control over pheromone intensity and decay and integrating direct and indirect communication methods.

Habanero provides a significant advance in swarm robotics by demonstrating the effectiveness of automatic design for creating complex, stigmergy-based behaviors. The superior performance of Habanero over manual and neuroevolutionary methods highlights the potential of this approach for streamlining swarm robotics development and deployment. Future work should explore more sophisticated pheromone control, integration of direct and indirect communication, and the application of this method to more complex tasks and environments.

The current implementation of Habanero uses a binary pheromone detection system (presence/absence). Future work should explore the use of pheromone intensity and decay rates for finer control over robot behavior. The study primarily focuses on indoor environments utilizing a photochromic material, limiting the generalizability to outdoor or unprepared environments. The benchmark missions, while providing a varied set of tasks, do not encompass the full spectrum of possible swarm robotic applications.