First-person view (FPV) drone racing is a challenging and dynamic sport requiring pilots to maneuver high-speed quadcopters through complex 3D courses using only onboard camera feed. Creating an autonomous system capable of matching, or exceeding, the skill of human champions is a significant challenge in artificial intelligence and robotics. Previous attempts at autonomous drone racing have fallen short of human-level performance, often relying on external motion-capture systems for state estimation. This research addresses this limitation by developing a system that utilizes only onboard sensors and computation to achieve champion-level performance in real-world races against experienced human pilots. The research question is whether deep reinforcement learning, combined with robust perception and control algorithms, can enable an autonomous drone to compete successfully at the highest level of FPV drone racing. The purpose of this study is to demonstrate that champion-level performance in a complex physical environment is achievable using a combination of simulation-based learning and real-world data adaptation. The importance of this work lies in its potential to advance the field of autonomous mobile robotics and inspire the development of similar hybrid learning-based solutions for other physical systems.

Deep reinforcement learning (DRL) has demonstrated impressive results in various simulated and game environments, surpassing human performance in games like Atari, chess, Go, StarCraft II, Dota 2, and Gran Turismo. However, translating this success to real-world physical competitions has been a significant hurdle, largely due to the discrepancies between simulated and real-world dynamics and sensor noise. Prior autonomous drone racing systems have made advancements, such as using deep networks for gate detection and transferring policies from simulation to reality, but they often relied on external motion-capture systems, thus negating a key challenge of true autonomous flight. This research builds upon previous advancements in drone racing, particularly addressing the gap between simulated and real-world performance by incorporating empirical noise models trained on real-world data.

The Swift system comprises two key modules: a perception system and a control policy. The perception system uses a visual-inertial estimator (VIO) and a convolutional neural network (CNN) to detect racing gates in the onboard camera images. The gate detections, combined with VIO data, provide an estimate of the drone's global pose via camera resectioning and Kalman filtering. The control policy, a two-layer perceptron, then maps this low-dimensional state representation to control commands for the drone. Model-free on-policy deep RL is used to train this control policy in simulation, maximizing a reward function that incentivizes progress towards the next gate while maintaining the gate within the camera's field of view. To bridge the gap between simulation and reality, the simulation is augmented with non-parametric empirical noise models (Gaussian processes and k-nearest-neighbor regression) estimated from real-world data collected using a motion capture system. These models capture the perception and dynamic discrepancies between the simulation and the real drone. This data-driven approach allows the policy, initially trained in simulation, to perform effectively in real-world conditions. Ablation studies and comparisons against traditional methods (trajectory planning and Model Predictive Control, MPC) are performed to validate the effectiveness of the proposed approach.

Swift competed against three world-champion drone racers: Alex Vanover, Thomas Bitmatta, and Marvin Schaepper. In head-to-head races, Swift won a majority of races against each champion (5 out of 9 against Vanover, 4 out of 7 against Bitmatta, and 6 out of 9 against Schaepper). Moreover, Swift achieved the fastest recorded race time. Analysis of lap times showed Swift to consistently achieve faster lap times than the human champions, with lower variance, indicating more consistent high-speed performance. The analysis also compared the performances segment-by-segment, highlighting Swift’s superior performance at the start and in tight turns (such as the Split-S maneuver), while human pilots exhibited faster times in some specific maneuvers. While human pilots are incredibly robust to crashes and environmental changes, Swift's performance was more consistent in controlled conditions. The improved performance of Swift was partly attributed to its use of inertial data (similar to the human vestibular system) and lower sensorimotor latency compared to human pilots.

The results demonstrate that a deep reinforcement learning-based system, using only onboard sensors and computation, can achieve champion-level performance in a complex, real-world, competitive environment. Swift's success addresses a major challenge in the field of autonomous robotics: transferring the performance achieved in simulation to the real world. The findings highlight the effectiveness of integrating real-world data into the simulation training process to account for discrepancies in perception and dynamics. The faster lap times and consistent performance of Swift, compared to even the top human pilots, showcase the potential of reinforcement learning to optimize complex control tasks beyond human capabilities, even in the face of noisy sensor data. The study’s implications extend beyond drone racing, suggesting that similar hybrid learning approaches could be applied to other high-performance robotic systems.

This research marks a significant milestone in autonomous mobile robotics and machine intelligence, demonstrating for the first time that an autonomous system can achieve world-champion-level performance in a real-world competitive sport. Swift's success showcases the potential of deep reinforcement learning coupled with robust simulation-to-reality transfer techniques. Future research could focus on improving Swift's robustness to environmental variations, such as changing lighting conditions, and enhancing its ability to recover from crashes, bringing its performance closer to that of adaptable human pilots.

While Swift outperformed human champions in many races, its performance is limited by the refresh rate of its onboard camera (30 Hz), which is slower than that used by human pilots (120 Hz). The system’s perception system also assumes consistent environmental appearance during training and might fail if this assumption is violated. Furthermore, Swift lacks the inherent robustness of human pilots in recovering from crashes and adapting to unpredictable events.