Soft strain sensors offer significant advantages over traditional metallic sensors, including high sensitivity, large working range, and flexibility. However, limitations such as low reproducibility, vulnerability to environmental noise, and performance degradation hinder their real-world applications. These challenges stem from the complex micro/nanofabrication processes and inherent randomness in nanomaterials used in piezoresistive and piezocapacitive sensors. Factors like uneven material distribution, sensitivity to temperature and curvature, and damage to thin conductive layers contribute to these limitations. Optical strain sensors, which detect optical responses, offer potential improvements in uniformity and repeatability, but they still require a robust interface to correct for errors caused by flexible surfaces. This study proposes a computer vision-based optical strain (CVOS) sensor to address these issues by integrating AI-based vision technology with an easily fabricated optical sensor. The sensor system combines a simple laser-processed micro-marker pattern on a soft silicone substrate, a compact optical system for capturing marker movement, and an automated response correction algorithm to improve accuracy and robustness. The integration of computer vision enables multiaxial strain mapping, allowing for simultaneous measurement of strain magnitude and orientation, a key advantage over previous sensor designs.

Existing literature extensively covers various types of soft strain sensors, focusing on materials and fabrication methods. Piezoresistive and piezocapacitive sensors utilizing carbon nanotubes, nanoparticles, nanowires, conductive polymers, and laser-induced graphene have shown promising results. However, these often suffer from reproducibility issues due to the complexity of nanofabrication techniques and the inherent randomness of nanomaterials. Studies highlight the susceptibility of these sensors to environmental noise, particularly temperature and curvature changes, affecting signal integrity and linearity. Performance degradation over time due to material degradation or damage to conductive layers is another significant concern. Recent research explores optical strain sensors as an alternative, leveraging the piezo-transmittance effect. While these show promise in sensor-to-sensor uniformity and repeatability, challenges remain in mitigating the influence of external light sources on the sensor response. The current research uses computer vision to address these limitations and improve sensor performance.

The CVOS sensor system comprises two subsystems: a sensing part and an optical system. The sensing part is a square-shaped (14mm x 14mm) white Ecoflex film with laser-etched micro-markers (average diameter 528.11 ± 23.58 µm, spacing 670.23 ± 18.2 µm). The white Ecoflex film enhances marker detection by reducing light reflection and penetration. The optical system consists of a tiny camera, a compact microscope lens, and an LED light source. A camera calibration process corrects optical distortions. The system captures images of micro-markers at a resolution of 640 x 480 pixels from a close distance (3-6 mm). A custom automated response correction algorithm processes images in real time. This algorithm includes several stages: micro-marker detection using image filtering and contour approximation, estimation of the sensor state (curvature and loading/unloading), object tracking of micro-markers, machine learning-based response correction, and multiaxial strain mapping. To address limitations of general object-tracking methods, particularly at image corners, a novel approach using virtual micro-markers is implemented to predict the position of markers outside the camera's field of view. Multiaxial strain mapping is achieved through micro-marker matching and image partitioning into quadrants, enabling determination of strain direction and magnitude in each quadrant. Curvature state detection utilizes a Voronoi diagram, classifying the sensor state as linear or bending (in-plane or out-of-plane). The algorithm's sampling rate is analyzed on both a PC and an embedded board (Raspberry Pi 4), demonstrating real-time capabilities. Numerical simulations using ANSYS are employed to predict micro-marker positions under tensile deformation. Finite Element Method (FEM) simulations are used to correct the sensor response for bending states. The gauge factor (GF) of the CVOS sensor is calculated, and sensor-to-sensor uniformity is evaluated using multiple samples. The dynamic response of the sensor to various strains, including very small strains, is characterized. Finally, the sensor’s performance is evaluated over 10,000 loading-unloading cycles. The CVOS sensor's performance is compared with that of existing strain sensors in terms of gauge factor and working range.

The CVOS sensor demonstrates superior performance compared to previously reported flexible strain sensors. It achieves a high gauge factor (503.4), low hysteresis (0.9%), and high linearity (R² > 0.99) across a wide working range (0-81% strain). The sensor exhibits excellent sensor-to-sensor uniformity (MAPE = 3.1%) due to the precise laser-based fabrication method. Furthermore, the CVOS sensor shows a high level of stability and durability, maintaining performance even after 10,000 loading-unloading cycles. The real-time multiaxial strain mapping capability allows for accurate detection of both strain magnitude and direction in multiple axes. This enables the sensor to differentiate complex body motions such as elbow flexion, wrist bending, and knee bending with high repeatability and stability. The system successfully distinguishes between elbow flexion towards the chest and towards the humerus, highlighting its ability to capture subtle differences in movement patterns. The CVOS sensor can also distinguish various shoulder movements (flexion, extension, abduction, internal rotation, external rotation, and adduction). The ability to detect complex strains, including rotational motions (supination and pronation), sets it apart from existing sensors that primarily focus on stretching motions. Comparisons between the CVOS sensor and an IMU sensor show highly correlated results in forearm rotation monitoring, illustrating its potential as a long-term alternative to IMU sensors for body motion monitoring. The CVOS sensor has a sampling rate of 83 FPS on a PC and 15 FPS on a Raspberry Pi 4, which is sufficient for monitoring human motion. Reducing the image resolution increases the sampling rate at the cost of a slightly reduced GF and a higher limit of detection. The sensor has a lower limit of detection of 0.19% which is close to the detection limit of recently developed strain sensors.

The integration of computer vision and simplified fabrication techniques enables the CVOS sensor to overcome limitations associated with traditional soft strain sensors. The easy fabrication method improves reproducibility and long-term operability. The sensor's high sensitivity, linearity, and robustness, combined with its real-time multiaxial strain mapping capability, make it suitable for diverse applications, particularly in human-machine interfaces for rehabilitation and healthcare. The ability to distinguish subtle differences in movement and rotational motions provides significant advantages over existing sensor technologies. The close correlation between the CVOS sensor and IMU sensor data in forearm rotation monitoring demonstrates its potential to replace IMU sensors in applications requiring long-term monitoring. The multiaxial strain mapping further enhances the sensor's capabilities by enabling the classification of complex movements not easily distinguishable with conventional strain sensors. This opens up new possibilities for motion analysis and control in various fields.

This study presents a novel CVOS sensor that successfully addresses the limitations of existing soft strain sensors. The integration of computer vision with a simple fabrication method yields a highly sensitive, reproducible, and durable device capable of real-time multiaxial strain mapping. This sensor has significant implications for human-machine interfaces, particularly in rehabilitation and healthcare, offering potential for feedback-based treatment and enhanced monitoring of complex body movements. Future work could focus on improving the sampling rate through hardware optimization, expanding the curvature state detection algorithm to include in-plane bending, and exploring different AI algorithms to further enhance sensor performance. Developing a softer device platform with fewer rigid components would broaden its applicability and usability. This work demonstrates a successful combination of AI and microfabrication technologies, paving the way for highly advanced motion-sensing applications.

The current curvature state detection algorithm focuses primarily on out-of-plane bending, limiting its ability to fully characterize in-plane bending. While the sampling rate on the embedded board (Raspberry Pi 4) is sufficient for many applications, higher sampling rates might be necessary for monitoring rapid movements. The use of virtual MOIs to extend the working range might introduce some prediction errors, though the study shows it maintains good accuracy. Further research could explore alternative methods for improving the accuracy of this virtual MOI generation and the algorithm itself. Although the sensor performed well in the experiments, further testing in a wider range of real-world conditions is necessary to fully validate its robustness and generalizability.