A multifunctional soft robotic shape display with high-speed actuation, sensing, and control

Shape-changing interfaces generate surface geometries through actuation for applications such as information displays, human interaction, object manipulation, and aerodynamics. Many existing approaches—push-pin arrays, hinge-actuated elements, particle jamming, pneumatic cells, magnetic or thermal morphing—suffer drawbacks including surface discontinuities or heat that limit haptics, low geometric fidelity, reliance on bulky external hardware, slow reversible morphing, and lack of embedded feedback. The research question addressed is whether a scalable, soft robotic shape display can be built that provides high-fidelity, high-speed morphing with embedded sensing and closed-loop control at scale. The purpose is to integrate soft electrohydraulic actuators with magnetic-based sensing and on-board control to overcome these limitations, enabling high refresh rates, accurate deformation and force sensing, and rich multifunctionality. The study’s importance lies in advancing shape displays toward practical, self-sensing, compliant, and high-performance platforms capable of dynamic interaction with solids and liquids and complex user interactivity without external tracking systems.

Prior work on shape-morphing surfaces includes push-pin actuator arrays manipulating surfaces directly or via elastic layers, hinge-actuated elements, particle jamming, pneumatics, and magnetically or thermally driven in-surface morphing. These methods often face surface discontinuities or elevated temperatures that diminish haptic viability, limited geometric fidelity, dependence on external equipment (magnetic plates, tracking systems, pumps), slow reversible morphing that restricts refresh rates and manipulation dynamics, and a lack of embedded in-surface sensing. Electrohydraulic actuators have shown promise for high-speed, high-force deformation in haptic interfaces, and embedded sensing for such actuators has been explored via various modalities. However, prior electrohydraulic arrays generally lacked embedded feedback and closed-loop control at the scales required for high-fidelity morphing, with challenges including electromagnetic interference from high-voltage driving and signal collection in dense arrays.

System architecture: The display is composed of 100 repeated cells arranged as ten 1×10 modules forming a 10×10 array, covered by a continuous elastomeric silicone skin to create a compliant, continuous surface. Each cell integrates: (i) actuation via a folded HASEL (Hydraulically Amplified Self-healing Electrostatic) actuator (58×49×14 mm undeformed; cell pitch 60×60 mm) operating from 0–8 kV with ~12 mm sustained deformation at 8 kV; (ii) embedded magnetic sensing using a soft magnetic block and a magnetometer to measure deformation; and (iii) a local HV driving circuit with an optoelectronic half-bridge and an integrated HV sensor for closed-loop voltage regulation. Ten cells share power and computation per module; a PC and AC/DC supply provide centralized control and power. Power considerations: average 2.8 W per cell at peak load; ~1.6 A at 24 V per module. Actuation and voltage control: Each cell’s half-bridge uses two optocouplers to charge/drain the actuator from a shared 8 kV rail. PWM duty cycles modulate charge/discharge rates. An HV divider senses voltage for feedback. Open-loop electrical dynamics were identified across all 100 cells, enabling a loop-shaped voltage controller solved at 1 kHz per cell (inputs at 600 Hz), achieving ~200 Hz bandwidth. The HASEL mechanical response was characterized via motion capture and frequency domain analysis; open-loop dynamics exhibit ~12 Hz natural frequency with damping from 20–50 Hz. High-speed capabilities enable traveling waves (up to 354 cm s−1) and quiet operation; the skin permits safe manipulation of liquids. Embedded sensing and calibration: Magnetometers (sampled at 600 Hz; 50 Hz low-pass filter) measure magnetic flux changes from the moving magnetic block. A per-cell third-order polynomial maps flux to deformation; coefficients are calibrated using synchronized motion capture during voltage ramps (0–8 kV over 8 s). To compensate for mechanical coupling through the skin, estimated deformation is adjusted by a weighted sum of adjacent cells. Force sensing: using a dynamic mechanical analyzer, quasi-static force–displacement curves were obtained at voltages 4–8 kV; actuators produce ~2.5 N at sub-mm deformations. A 15th-order polynomial surface maps measured voltage and deformation to estimated force with ~50 mN resolution. Deformation control: An outer-loop deformation controller, designed via loop shaping using identified HASEL dynamics, runs at 200 Hz independently on each cell. The closed-loop deformation bandwidth is ~20 Hz, limited by HASEL dynamics and communication latency. A prefilter shapes commands. The controller rejects internal disturbances (e.g., charge retention) and external perturbations, enabling accurate static and time-varying shapes (e.g., scrolling text). Interaction and applications: Embedded sensing supports stimulus–response behaviors (e.g., mirroring touch), a self-displaying scale that sums distributed forces to estimate and display object mass, and interaction with a magnetic-tip wand for user-driven drawing by mapping sensed magnetic flux to local deformation. Ball manipulation experiments: An overhead color camera (260 Hz) provides ball position for closed-loop manipulation; velocities are derived and filtered, yielding a 52 Hz control loop. A semicircular, Gaussian-profile local deformation is commanded near the ball to induce rolling. Experiments: (i) single ball follows a square path through four waypoints; (ii) three balls are color-sorted to goals. Trajectories are recorded over 25 trials per experiment. Fabrication: HASELs and magnetic blocks were fabricated per Supplementary Methods. Magnetometers (LIS3MDL) are wired in series per 1×10 module. HV driver boards employ HV photodiodes and insulating epoxy. Modules use 3D-printed and laser-cut components; silicone skin is a 550 µm EcoFlex 00-30 layer. Software is implemented in Julia on the PC and C++ on microcontrollers.

- Achieved high-speed, high-fidelity shape morphing on a compliant, continuous 10×10 soft robotic display with tightly integrated actuation, sensing, and control. - Performance metrics: 200 Hz voltage regulation bandwidth; up to 50 Hz actuation (mechanical response damped 20–50 Hz); closed-loop deformation bandwidth ~20 Hz; deformation sensing resolution <0.1 mm (≈0.8% of 12 mm stroke); force sensing resolution ≈50 mN (≈5 g mass). - HASEL actuators deliver ≈12 mm sustained deformation at 8 kV after 60 cycles (~86% actuator strain; ~13.3% relative to 90 mm cell height). Force output ≈2.5 N for sub-mm deformations. - High-speed traveling waves generated with speeds up to 354 cm s−1; quiet operation (≈8 dB above ambient at resonance; 1–2 dB above ambient at slower operation). Continuous silicone skin enables manipulation of liquids. - Accurate self-sensing of deformation across 100 cells; frequency tracking accurate up to ~30 Hz; capability to map distributed forces and detect nonconductive objects. - Closed-loop deformation control improved geometric accuracy over open-loop voltage control, enabling high-speed dynamic content (e.g., scrolling text) and stimulus–response functions (e.g., mirroring touch), as well as an interactive self-displaying scale that estimates and displays mass in real time. - Dynamic object manipulation: precise rolling of a table tennis ball along a square trajectory (25 trials) within ~0.5 cell (≈3 cm) of the desired path; mean completion time 28.48 s; mean rolling speed 85.44 mm s−1 (1.424 cells s−1). Color sorting of three balls succeeded in all 25 trials with mean sorting time 4.66 s per ball.

The integrated soft shape display addresses key limitations of prior systems by combining high-speed electrohydraulic actuation with embedded magnetic sensing and distributed closed-loop control in a scalable 10×10 array. The continuous, compliant silicone skin and HASEL actuators enable safe, quiet, and high-frequency morphing without heat associated with SMA/LCE or noise from pneumatics, while providing sufficient force to manipulate solids and liquids. Embedded sensing yields accurate deformation and force maps at high sampling rates, enabling responsiveness to external stimuli and user interactions without external tracking systems. Closed-loop control compensates for actuator nonlinearities and disturbances (e.g., charge retention), achieving precise static and dynamic shapes. The demonstrated capabilities—traveling waves, liquid handling, interactive drawing, self-displaying scales, and dynamic ball manipulation—validate the approach and highlight the multifunctionality achievable when actuation, sensing, and control are tightly integrated at scale. Remaining challenges include sensitivity to external magnetic materials, edge and series-wiring effects on sensing accuracy, and communication/computation constraints for scaling beyond 10×10. Nonetheless, the platform sets a new benchmark for soft, sensor-rich, shape-morphing surfaces and suggests applicability to other high-DoF soft robots.

This work demonstrates a multifunctional soft robotic shape display built from 100 tightly integrated electrohydraulic actuator–sensor–controller cells, enabling high-fidelity, high-speed morphing with embedded deformation and force sensing and distributed control. The system achieves 200 Hz voltage regulation, ~20 Hz closed-loop deformation, up to 50 Hz actuation, sub-millimeter deformation sensing, and 50 mN force resolution, supporting rich applications from dynamic content display to responsive haptics and object manipulation of solids and liquids. The approach advances the scalability of soft electrohydraulic arrays with embedded feedback and opens the door to sensor-rich soft robotic devices and high-DoF systems. Future work includes scaling to larger arrays with improved communication architectures, miniaturizing cells for higher pixel density while preserving performance, enhancing sensing (e.g., multi-axis magnetometers for shear), mitigating magnetic susceptibility, and optimizing calibration and signal routing to reduce edge and impedance-related sensing errors.

- Magnetic-based sensing is susceptible to external magnetic materials; relocation requires recalibration to the local magnetic environment. - Edge cells exhibit higher sensing inaccuracies; series wiring of magnetometers within a module increases impedance and sensor noise toward the end of the chain; skin coupling introduces cross-talk that requires compensation. - Current module design faces scaling limits (pin addressing, sensor signal impedance, and power distribution); larger arrays will require new communication and computation architectures. - Miniaturization below ~25×25 mm is challenged by fabrication limits, actuator film thickness effects, and reduced magnetic field strength, which diminish sensing resolution and actuation performance. - Closed-loop deformation bandwidth is limited (~20 Hz) by actuator dynamics and communication latency; sensing accuracy decreases above ~30 Hz as deformation amplitudes approach sensor resolution. - The system currently relies on an external power supply and PC (though battery operation is feasible given power draw).