The development of flapping-wing micro aerial vehicles (FWMAVs) has made significant strides, inspired by the aerodynamic mechanisms of biological flight. Examples like RoboBee, DelFly, KUBee, and the Purdue Hummingbird showcase high agility and energy efficiency at small scales. However, challenges remain in flight control and lightweight design, especially in windy conditions. Controlling FWMAVs in such environments is difficult due to their complex dynamics, small size (limiting thrust and moment generation), and the limitations of conventional sensor systems. Current approaches often rely on inertial measurement units (IMUs), vision cameras, and proportional-integral-derivative (PID) controllers, but these suffer from drawbacks: IMUs located on the body cause delayed stabilization due to body movement, existing sensors are inadequate for flexible flapping wings, and model-based controllers struggle with the complex aerodynamics of flexible wings. In contrast, insects like dragonflies, hawk moths, and hoverflies exhibit exceptional flight agility, even in unsteady environments, partly attributed to campaniform sensilla—mechanoreceptors on their wings that detect wing strain, providing immediate proprioceptive flight control. These sensilla detect complex aerodynamic loads, including airflow stagnation, separation, vortex growth, and shedding. Recent research suggests they also detect Coriolis forces, functioning as inertial sensors. Directly measuring wing load is advantageous for flight control because the aerodynamic unsteadiness makes real-time estimation challenging for conventional systems. While attempts have been made to utilize wing deformation for aircraft control, successful implementation in flapping drones remains elusive due to the complexity of the aerodynamics. This article introduces a "fly-by-feel" wing-strain-based flight controller using reinforcement learning (RL) to overcome the challenges of modeling complex aerodynamics. The research aims to assess the feasibility and reliability of autonomously determining flight paths using only wing strain sensors.

Existing literature highlights the advancements in flapping-wing drone technology, focusing on bio-inspired designs and their limitations in flight control, particularly in windy conditions. Studies on insect flight mechanisms, particularly the role of campaniform sensilla in sensing aerodynamic loads and providing proprioceptive feedback for flight control, are reviewed. The limitations of traditional sensor systems (IMUs, vision-based systems) for controlling FWMAVs in dynamic environments are discussed. Previous attempts to incorporate wing deformation for flight control in other aircraft are examined, highlighting the challenges of applying these principles to flapping-wing drones due to the complex, unsteady aerodynamics involved. The use of reinforcement learning as a powerful tool for controlling complex systems, especially in the context of robotics and autonomous navigation, is also explored. The review establishes the novelty of the proposed approach by emphasizing the gap in existing literature: successful application of a wing-strain-based, RL-driven flight controller for flapping-wing drones.

The study employed a bio-inspired "fly-by-feel" control system using ultrasensitive, lightweight crack-based strain sensors attached to the wing bases of a commercial flapping-wing drone. These sensors, inspired by insect campaniform sensilla, collect aeroelastic information reflecting wing deformation. A 1D convolutional neural network (CNN) analyzes this deformation data to extract state information, including wind direction, speed, and drone attitude. A reinforcement learning (RL)-based controller, specifically the soft actor-critic (SAC) algorithm, processes the state information and determines the optimal control actions (motor power adjustments) to maximize a reward function. The SAC algorithm was chosen for its sample efficiency and robustness. The experimental approach involved five phases: 1. **Sensor Validation:** This phase verified the sensor's ability to provide reliable state information by systematically varying wind direction and speed, testing the CNN's accuracy in decoding this information. A temporal 1D CNN classified the wind direction and speed, with accuracy assessed using a confusion matrix and error metrics. 2. **1 DOF Control:** This experiment evaluated control in a single degree-of-freedom (DOF) environment (circular motion), focusing on the drone's ability to adapt to changing wind conditions using only strain sensor data. The RL agent learned to adjust motor power to maintain a target position. 3. **2 DOF Control:** This phase extended the control to two DOFs (rotation and pitch), testing the drone's ability to maintain optimal pitch angles for forward motion despite aerodynamic disturbances. The RL agent learned to identify and recover from falling states. 4. **Position Control in Wind:** This phase tested 3D position control in a wind tunnel with complex, asymmetric airflow. The drone, using only strain sensor data, navigated towards a target position, its performance compared to an untrained drone. 5. **Windless Flight Control:** This final experiment demonstrated the system's ability to control flight paths (zigzag, circular, and s-curve motions) in a windless environment. The RL agent was trained using a combination of human demonstrations and autonomous learning, the accuracy of the reconstructed drone trajectory (odometry) compared to ground truth data from a motion capture system. The crack-based strain sensors were fabricated using metal layers deposited on a polyimide substrate, creating nanoscale cracks that provide high sensitivity. Signal processing involved bandpass filtering and normalization of the sensor signals. The RL algorithm used a 1D CNN to process the time-series strain data and a SAC algorithm for policy optimization. The experiments were conducted in various controlled environments (wind tunnel, controlled space) using motion capture cameras to track the drone's position and orientation.

The study successfully demonstrated that wing strain data, acquired from lightweight, ultrasensitive crack-based sensors, could provide sufficient information for controlling a flapping-wing drone in various environments. Key findings include: 1. **Sensor Accuracy:** The 1D CNN accurately classified wind direction and speed (mean accuracy of 80%), and the mean absolute theta error for wind direction was 29°. This indicates that wing strain data contains significant information about the drone's surrounding environment. 2. **1 DOF Control Success:** In the single DOF environment, the RL-trained drone successfully maintained its target position despite changing wind conditions, significantly outperforming untrained drones in terms of accumulated reward. This demonstrates that strain data can be used for effective real-time control. 3. **2 DOF Control and Disturbance Recovery:** The two DOF experiment confirmed that the drone can determine its pitch angle from strain data alone and maintain balance to achieve maximum reward by keeping its pitch angle within a specific range. Moreover, the trained drone could quickly recover from falling states, highlighting the system's robustness. 4. **Windy Environment Position Control:** In the wind tunnel experiment, the RL-trained drone effectively maintained proximity to a target position in complex airflow, outperforming the untrained drone in reaching and maintaining the target, indicating the potential of wing strain sensors for position control in unsteady conditions. The tests for reaching alternative target positions show the generalization of this adaptive control system. 5. **Precise Windless Flight Control:** The final experiment showcased precise control of the drone's flight path in a windless environment using only strain sensors. The RL agent successfully learned complex maneuvers (zigzag, circular, and s-curve) demonstrating the robustness and precision of the system. The odometry results, compared against ground truth data, show high accuracy in trajectory reconstruction from strain data alone. This suggests wing strain data is a potentially reliable source of motion information.

The study's results successfully validate the hypothesis that wing strain information, combined with reinforcement learning, can enable autonomous flight control of flapping-wing drones without relying on traditional sensors like IMUs or vision systems. The system's adaptability to various airflow conditions (windy and windless environments) and its ability to control both position and attitude demonstrate its potential for wider applications. The findings offer a new paradigm for drone control, mimicking the biological sensing mechanisms of insects, which often outperform current technological solutions. The use of lightweight, bio-inspired sensors reduces the drone's weight, crucial for smaller, more agile designs. The inherent robustness of the RL-based approach allows for adaptation to unexpected aerodynamic changes. The superior performance in various tasks compared to untrained drones underscores the effectiveness of the reinforcement learning algorithm in learning an optimal policy based on strain sensor data. However, limitations exist and need future research to overcome.

This research demonstrates a novel wing-strain-based flight control system for flapping-wing drones using reinforcement learning. The system successfully controlled a drone in various environments using only strain sensors, showcasing adaptability and robustness. This bio-inspired approach opens new possibilities for lightweight, agile, and robust drone control. Future research will focus on integrating additional sensors (gyroscopes, accelerometers) to improve performance, miniaturizing the sensors further, and exploring more complex flight maneuvers like hovering and wind-assisted flight.

The study's limitations include the use of a tethered drone in some experiments, which could affect the drone's natural flight dynamics. The current system is also limited by the accuracy of the CNN's interpretation of wing strain data and the ability of the RL algorithm to optimize the control policy. Further research is needed to address these limitations and explore the potential for even more precise and robust autonomous flight control.