Human skin's remarkable somatosensory system efficiently perceives diverse physical stimuli with high precision, a capability that current machine sensors struggle to replicate. Machines often rely on numerous specialized sensors to detect limited stimuli, hindering the development of sophisticated human-robot interfaces and soft robots. This research addresses this limitation by developing a single sensing element capable of perceiving multiple modes of physical interaction. Existing soft sensors primarily focus on deformation, often employing conductive hyperelastic composites that measure changes in resistance or capacitance with strain. However, discerning multiple deformation modes with a single conductive sensor is challenging, and employing multiple elements increases complexity and mass. Optical-based sensors offer advantages such as immunity to electromagnetic interference and higher information density, utilizing light intensity or wavelength to detect deformation. Yet, no existing architecture distinguishes omnidirectional bending, stretch, compression, and their combinations using a single element. This is crucial for applications like soft robotics and human-robot interfaces, where omnidirectional bending is common. To mimic the human skin's capability for combined temperature and deformation sensing, the next generation of sensors must extend beyond deformation detection. In human skin, temperature sensing beyond a threshold triggers various responses; similarly, in soft machines, exceeding critical temperatures can induce material modulus changes, shape-morphing, actuation, and self-healing. Existing combined sensors often employ toxic materials or fail under cyclic loading, limiting their applications. Optical temperature and strain sensing has been demonstrated in conventional optic fibers, but typically only under uni-axial strain. ChromoSense aims to overcome these limitations by providing a unified architecture for multi-modal sensing, advancing the field of human-machine interfaces and robotics.

The existing literature highlights the challenges in creating multi-modal sensors for soft robotics and human-machine interfaces. Many studies focus on soft deformation sensors, utilizing conductive hyperelastic composites to measure strain via changes in resistance or capacitance. While effective for single-mode detection, discerning multiple deformation types (bending, stretching, compression) with a single sensor is difficult. Multi-element arrays improve modal detection but add complexity and mass. Optical sensing offers a promising alternative, with several studies demonstrating the use of light intensity and wavelength to detect various deformation modes. However, these approaches often lack the capacity to distinguish omnidirectional bending, a key requirement for many applications. Combined temperature and deformation sensors have also been explored, but limitations arise from the use of toxic materials or poor performance under cyclic loading. The existing work on optical temperature and strain sensing typically functions under uni-axial strain, further underscoring the need for a more versatile multi-modal sensor.

ChromoSense, the proposed sensor architecture, is a stretchable cylinder composed of dyed sections of optically transparent rubber connected by 3D-printed interfaces. A white LED illuminates three sections doped with red, blue, and green pigments, and a thermochromic section (changing color above a transition temperature) is incorporated in series. A clear section mixes the light, and a miniaturized spectral sensor captures the output. Deformations alter the light path length through the colored sections, modifying chromaticity and intensity. Temperature changes affect the thermochromic section's transmittance. The sensor was characterized via an opto-thermo-mechanical assay, subjecting it to various single-mode stimuli (bending in three principal directions and their bisections, tension, and compression) and multi-mode stimuli (sequential combinations of bending, stretching, compression, and heat). Bending tests involved a 5-DoF platform, enabling precise control of curvature and direction. Tension and compression tests utilized an Instron machine. Combined loading tests assessed the sensor's ability to discriminate between sequentially applied modes. Data analysis involved visualizing chromaticity in CIE 1931 color space and analyzing intensity changes. The sensor's performance was evaluated in terms of sensitivity, repeatability, and stability over multiple cycles. Three application demonstrations were included: (1) a soft exosuit to estimate wearer's 3D pose; (2) a compliant origami-inspired interface to track its orientation in real-time; and (3) a variable stiffness soft robotic manipulator controlled by temperature and deformation. In the exosuit demo, two ChromoSenses were used on the hip and shoulder to track movements, with color space threshold control triggering assisted lifting. The origami interface demo involved using a neural network to map chromaticity to orientation, updating a digital twin in real time. The soft robotic manipulator used the sensor to perform autonomous move-and-hold operations at prescribed curvatures, responding to environmental temperature changes. The ChromoSense fabrication involved casting cylindrical thirds of PDMS or urethane with dyes, bonding them together, adding a thermochromic section, and a clear mixing section. The LED and color sensor are housed in 3D-printed interfaces. Design considerations involved balancing light intensity, chromatic response distinctness, sensitivity, and material stiffness and stretch at yield. Additional notes describe the multi-DoF characterization setup, the working principle of the sensor in bending (ray path propagation and path length modulation), UV-Vis spectra of dyed polymers, thermal characterization experiments, the exosuit design, the origami interface design, and the variable stiffness manipulator design.

ChromoSense successfully distinguishes omnidirectional bending, compression, stretch, and discrete temperature changes, both individually and in sequentially applied combinations. Bending induces chromaticity shifts primarily towards colors opposite the bending direction, with intensity changes providing further discrimination. Tension and compression primarily affect intensity. Heating past the thermochromic transition point desaturates the color, providing clear distinction in multi-modal scenarios. The sensor demonstrates high stability over multiple cycles and exhibits a linear response to deformation within its operational range. Three application demonstrations showcase ChromoSense's versatility: (1) In a soft exosuit, two ChromoSenses provided dense state feedback to estimate the wearer's 3D pose, enabling color-space threshold control to trigger force-assisted lifting. Analysis of the resulting sensor trajectories offered insights into wearer biomechanics, such as phase offsets between torso and arm movements. (2) In a compliant origami interface, a single ChromoSense accurately estimated the 3D orientation in real-time, feeding a digital twin through a minimal neural network. Testing showed excellent agreement between predicted and actual Euler angles, highlighting accurate capture of user input across various frequencies and amplitudes. (3) In a variable stiffness soft robotic manipulator, the sensor facilitated autonomous move-and-hold operations at prescribed curvatures, showing closed-loop control incorporating both thermal signal processing and curvature control, a novel advancement. The sensor's ability to decouple multiple stimuli increases information density and enables adaptive responses to environmental changes. The sensor exhibits a linear relationship between applied deformation and resulting chromatic shift, offering a simpler and more robust alternative to conventional methods, avoiding complexities such as strain rate dependence, electromigration, and Mullins' effect often found in conductive composite sensors. ChromoSense's design is scalable and adaptable to various applications.

ChromoSense addresses the need for compact, multi-modal sensors for advanced robotics and human-machine interfaces. Its ability to discern various deformation modes and temperature changes using a single element represents a significant advancement over existing technologies. The sensor's high information density and straightforward output interpretation simplify system design and reduce computational demands. The application demonstrations highlight ChromoSense's potential for enhancing the capabilities of soft exosuits, haptic interfaces, and soft robotic manipulators. The results showcase the efficacy of integrating multi-modal sensing into a single, robust element, bringing machines closer to the level of proprioception found in biological systems. Future research could explore continuous temperature sensing using materials that exhibit continuous spectral shifts with temperature and address the challenge of decoupling simultaneously applied stimuli using advanced signal processing techniques. Further investigation into material properties and sensor geometry optimization could enhance the sensor's performance and expand its operational range.

ChromoSense represents a significant advancement in multi-modal sensing technology. Its ability to detect multiple stimuli using a single element, its high information density, and its ease of integration into various applications make it a promising candidate for numerous applications in robotics and human-machine interfaces. Future work should focus on refining the sensor's design to improve its sensitivity, expand its operational range, and enable the decoupling of simultaneously applied stimuli.

While ChromoSense demonstrates significant capabilities, limitations remain. The current embodiment's ability to decouple simultaneously applied stimuli is limited, and the thermochromic dye provides only binary temperature sensing. Further research is needed to address these limitations, potentially involving advanced signal processing techniques or different thermochromic materials. The sensor's baseline chromaticity is sensitive to dye concentration variations, requiring precise control during fabrication. The maximum bending angle is limited by the critical angle for total frustrated internal reflection. Future work could address this limitation through material selection or cladding.