Multi-modal deformation and temperature sensing for context-sensitive machines

The paper addresses the challenge of endowing machines—particularly soft robots and human–robot interfaces—with compact, information-dense, multi-modal sensing akin to human skin. Prior soft sensors mainly capture deformation and often require arrays of elements and complex models to distinguish multiple modes. Optical sensors offer advantages (e.g., immunity to electromagnetic interference and richer information via intensity and wavelength), but existing architectures cannot distinguish omnidirectional bending along with stretch, compression, temperature, and their combinations using a single element. The authors introduce ChromoSense, a single soft sensing element that decodes omnidirectional bending, axial stretch and compression, discrete temperature changes (across a thermochromic transition), and sequential combinations of these stimuli by leveraging chromaticity and intensity changes of light traveling through dyed elastomer sections with TFIR-based guidance. The goal is to increase proprioceptive capability and reduce hardware complexity for context-sensitive machines.

The authors review soft deformation sensors, noting many use conductive hyperelastic composites whose resistance or capacitance varies with strain; single conductive sensors struggle to distinguish multiple deformation modes, and multi-element arrays add mass, fabrication complexity, and often require data-driven interpretation. Optical soft sensors provide higher information density and EMI resilience; prior work used intensity or combined intensity and wavelength to detect various deformation modes, and heterogeneous mechanisms combining optics, fluidics, and conductives have been used with learned models. However, a single-element sensor distinguishing omnidirectional bending, stretch, compression, and their combinations has been lacking. Temperature sensing combined with deformation has been shown in systems like optical fibers (e.g., Brillouin loss for temperature and very small uni-axial strain) and flexible multimodal sensors, but typically limited to uni-axial strains or materials with drawbacks (toxicity, poor cyclic durability). This context motivates an integrated optical, dye-doped elastomer approach to multi-modal deformation and temperature sensing.

Sensor architecture: ChromoSense is a stretchable cylinder of optically transparent elastomer segmented into: (1) three longitudinal parallel thirds doped with red, green, and blue pigments that act as wavelength-selective filters and encode bending via differential path length and ray concentration; (2) a serial section doped with thermochromic microcapsules that undergo a spectral shift above a transition (~31 °C), enabling binary temperature detection and separating heated from unheated chromaticity trajectories; (3) a clear section to mix light additively before reaching a miniature spectral sensor. Light from a white LED (120° emission) is guided within the elastomer by total frustrated internal reflection (TFIR). Cladding of lower refractive index can be added for robustness; air otherwise suffices. Fabrication (Note M1): Cylindrical thirds (50 mm each) were cast in molds from PDMS (Sylgard 184, 10:1 by mass) or urethane (ClearFlex 30, 10:9.4) and doped with dyes (EPODEX Transparent Red/Blue/Green; example concentrations R: 1 drop/11 g, B: 3/11 g, G: 7/11 g). Thirds were bonded using the same polymer. A 1 mm-thick thermochromic section (Adafruit Thermochromic Pigment, Yellow) was cast in series, followed by a 50 mm clear section. Final sensor: ~101 mm length, 10 mm diameter. A white LED (OSLONSignal LUW CRBP.01) was mounted in the top interface; a color sensor (Adafruit TCS34725) in the bottom interface; both held by 3D-printed ABS parts. Design considerations (Note M2) discuss LED luminous flux, optical properties and dye concentrations, refractive index requirements (e.g., PDMS n≈1.41, ClearFlex n≈1.46, air n≈1), and thermochromic options (organic microcapsules with threshold behavior vs liquid crystals for continuous response). Characterization setup (Note M3): A custom 5-DoF platform (bottom X–Y gantry; top Z-roll-pitch) applied controlled pure bending with prescribed curvature κ, length L, and direction φ via kinematic modeling and genetic algorithm optimization to achieve desired centerline curvature while minimizing stretch/compression. Tension and compression tests used an Instron 3345 (1 kN load cell) at 40 mm/min; tension up to 25% strain for characterization, compression up to just below buckling. Combined bending+stretch used a custom fixture with maximum curvature and axial strain up to 5%. Heat was applied by forced convection at 109 °C until the thermochromic section reached steady-state heated absorbance (transition at ~31 °C). Operating principles (Notes M4–M5): In bending, ray path propagation concentrates light on the convex side, increasing intensity there; simultaneously, path length modulation through dyed thirds changes absorbance per Beer's law, shifting chromaticity toward colors opposite the bend direction. UV-Vis spectra of dyed polymers confirmed selective filtering; thermochromic transmittance increased above transition, especially above ~550 nm. Intensity tends to decrease at high curvature when critical angles for TFIR are exceeded. Bending sensitivity scales with sensor radius and varies with direction and heating. Thermal response (Note M6): A heat gun at varying apparent temperatures (e.g., 49 °C, 169 °C) produced chromaticity changes whose time derivatives correlate with heat flux; the thermochromic response is binary in steady state at the chosen dye but rate-dependent. Application setups: Soft exosuit (Note M7): ChromoSenses anchored to shoulder and hip via Velcro; actuators controlled by an SMC ITV1011 regulator driven by Arduino PWM at 20 kPa; body motions induce combined bending/stretch at shoulder and bending at torso. Origami 3-DoF interface (Note M8): Parallel platform with glass fiber rigid parts and Kapton flexures; one ChromoSense between base and end-effector. A minimal neural network (MATLAB fitnet; 1 hidden sigmoid layer; output ReLU) maps 3×1 RGB to a 9×1 rotation matrix; trained with ArUco-based ground truth to drive a real-time digital twin (Tkinter visualization). Variable-stiffness manipulator (Note M9): Two Dragon Skin 10 pneumatic actuators bonded to fabric strain limiters and conjoined by a 1 mm thermoset (Jeffamine D400:Epon 828, 10:4 by mass; cured at 38 °C for 12 h) with glass transition Tg≈35–40 °C. One ChromoSense bonded along the neutral axis. An environmental chamber provided heating/cooling; miniature regulators inflated actuators. A state machine used empirically calibrated CIE thresholds for curvature and temperature to autonomously execute move-and-hold sequences at target curvatures (±κ).

- Single-element ChromoSense decodes omnidirectional bending direction and curvature via chromaticity trajectories in CIE 1931 space; axial stretch and compression minimally affect chromaticity but modulate intensity, enabling discrimination of bending vs axial modes. - Bending: Chromaticity shifts toward colors opposite the bend direction; intensity generally decreases at high curvature due to TFIR losses. Average intensity sensitivity (unheated) across directions in curvature regime 0.02–0.03 mm⁻¹ is ~3.31 dB mm⁻²; with heat, ~5.72 dB mm⁻². Intensity spread during bending <10% of average trajectory. - Stretch: Negligible chromaticity change; intensity sensitivity 13.4 dB ε⁻¹ with only ~2% spread. Compression: Negligible chromaticity change; intensity sensitivity 2.3 dB N⁻¹ with ~14% spread due to geometric instabilities. - Cyclic durability: 500 stretch-return cycles with no chromaticity drift. Tensile failure at ~61% strain; chromaticity remains negligible until failure. - Temperature: Thermochromic transition at ~31 °C produces a large chromatic shift (yellow desaturation) that cleanly separates heated vs unheated trajectories for all modes; rate of chromatic change correlates with applied heat flux. - Multimodal (sequential) stimuli: Heated vs unheated trajectories do not overlap, enabling decoupling of temperature from deformations. Bending+stretch shows bending-like chromaticity with additional intensity decrease from stretch; bending+stretch+heat similarly separable. Stretch+heat and compression+heat show shifted origins but negligible chromaticity change compared to unheated counterparts. - Heating enhances sensitivity for stretch (from 13.4 to 15.2 dB ε⁻¹) with negligible impact on repeatability; compression shows comparable intensity change when heated due to small deformation range. - Applications: Exosuit pose estimation—hip sensor shows linear chromaticity changes under torso flexion; shoulder sensor shows quasi-elliptical chromaticity trajectories during rotation with significant intensity drops indicating combined bending+stretch; color-space thresholds trigger assistive actuation during lifting. 3-DoF origami interface—minimal NN maps chromaticity to orientation with low RMS errors: 0.0516 rad (roll α), 0.0966 (pitch β), 0.1052 (yaw γ) for slow large spirals; 0.0499 (α), 0.0784 (β), 0.0915 (γ) for faster small spirals. Variable-stiffness manipulator—single sensor enables autonomous closed-loop move-and-hold at ±κ by jointly detecting thermal state (above Tg) and curvature.

ChromoSense satisfies the need for compact, information-dense, multi-modal sensing by leveraging light intensity and chromaticity changes in a single dyed elastomeric waveguide. The distinct chromaticity trajectories for bending directions/curvatures and the minimal chromatic effect for axial strains allow clear discrimination between bending and axial modes. The thermochromic section adds a robust temperature channel that shifts the chromatic baseline to non-overlapping regions in CIE space, enabling decoupling of thermal and mechanical stimuli when applied sequentially. This architecture reduces sensor count and computational burden for high-DoF systems, as demonstrated by: (1) wearable exosuit state estimation and control with only two sensors, revealing nuanced 3D pose dynamics and enabling threshold-based assistance; (2) accurate, real-time orientation tracking of an origami interface with a single sensor and a small neural network; and (3) closed-loop control of a variable-stiffness manipulator with joint thermal and deformation feedback from a single element. The results validate that rich proprioceptive information can be compactly encoded and decoded without large sensor arrays or complex heterogeneous packages, advancing the state of multi-modal sensing for human–machine interfaces and soft robotics.

The work introduces ChromoSense, a single-element, soft, optical sensor that distinguishes omnidirectional bending, stretch, compression, discrete temperature changes, and sequential combinations via chromaticity and intensity readouts. It replaces arrays of conventional sensors for state feedback in wearables, interfaces, and soft robots, as shown in three demonstrations (exosuit assistance triggering, 3-DoF interface pose estimation, and autonomous variable-stiffness manipulation). Future research could: (1) improve simultaneous (not just sequential) stimulus decoupling using enhanced data processing or recurrent models; (2) achieve continuous temperature sensing by employing thermochromic liquid crystals or multi-threshold dye mixtures; (3) expand dynamic range (e.g., higher bending angles) via materials/claddings with larger refractive index contrast; and (4) refine fabrication (e.g., integrated 3D printing) and dye concentrations to enhance mechanical robustness and baseline chromatic stability. The ChromoSense paradigm opens a large design space for scalable, robust, multi-modal proprioception approaching biological richness.

- Simultaneous (non-sequential) decoupling of temperature and deformation is limited by the time-dependent thermochromic transition; full decoupling demonstrated at steady-state heated absorbance only. - Thermochromism used is binary above its transition, preventing continuous temperature estimation (e.g., cannot distinguish 40 °C vs 90 °C). - Mechanical limits: In tension, lower detection limit ~0.02 mm; failure at ~61% strain, with embrittlement exacerbated by higher dye concentrations. Interfaces between sections (colored/thermo/clear) can be failure initiation sites. - Bending limits: Lower detection limit on the order of 3×10⁻⁴ mm² (direction-dependent). Saturation/non-monotonic behavior occurs around ~150° due to exceeding TFIR critical angle and light escape. - Baseline chromaticity is sensitive to dye concentration balance; ±5% variations can shift CIE x,y by ±0.02, affecting linearity and centering. - Model dependence: Mapping outputs to finite curvature/strain requires models; sensor exhibits nonlinearities under certain combined stimuli. Data-driven or physics-based models must account for these effects. - Potential mitigations: Use higher n-contrast claddings/materials; reduce thermochromic volume and increase concentration to speed thermal response; adopt continuous thermochromic materials; integrated manufacturing to strengthen interfaces; lower dye loading with geometry scaling to preserve filtering effect.