Shape displays, which actively manipulate surface geometry, are a growing field with applications in haptics, manufacturing, and aerodynamics. However, current displays often lack high-fidelity shape morphing, high-speed deformation, and embedded sensing. This research addresses these limitations by developing a multifunctional soft shape display. The need for a high-performance, multi-functional shape display stems from the limitations of existing technologies. Current methods often suffer from surface discontinuities, low-fidelity shape morphing, dependence on bulky external equipment, slow actuation speeds, and a lack of embedded sensing. These shortcomings restrict the potential applications of shape displays in various fields. The integration of high-speed soft robotic actuators and sensors with natural mechanical compliance offers a promising solution. Soft electrohydraulic actuators, known for their high-speed, high-force deformation, have shown promise in braille and haptic interfaces. Additionally, embedded state sensing in electrohydraulic actuators has been demonstrated. However, integrating and controlling these actuators and sensors in high-dimensional arrays remains a significant challenge. Previous approaches have demonstrated actuation of electrohydraulic arrays, but lacked embedded feedback sensors, limiting their capabilities. Integrating sensor arrays is difficult due to electromagnetic interference, and integrated closed-loop sensor feedback has not been demonstrated at scales sufficient for high-fidelity shape morphing. This paper introduces a novel approach to overcome these challenges.

Existing shape displays employ various actuation methods, including push-pin actuator arrays, hinge-actuated surface elements, particle jamming, pneumatic cells, and magnetic or thermal-driven morphing. Each approach has limitations: push-pin arrays can cause surface discontinuities and high temperatures; hinge-actuated elements may have low-fidelity geometries; particle jamming and pneumatic cells require external devices; magnetically driven systems might be susceptible to environmental constraints; and thermal methods generate heat. Furthermore, many lack embedded in-surface state feedback, hindering responsiveness to external stimuli. The integration of soft robotics has improved speed, compliance, and embodied intelligence in other fields such as biomedical devices and human-robot interaction. While soft electrohydraulic actuators have been effective in haptic interfaces, integrating and controlling them in high-dimensional arrays with embedded feedback remains a challenge.

This research introduces a multifunctional soft shape display composed of scalable cellular units integrating soft actuation, embedded deformation sensing, and control. Each cell utilizes a Hydraulically Amplified Self-healing Electrostatic (HASEL) actuator for high-speed, low-power shape morphing. HASEL actuators offer advantages over other soft actuators like shape memory alloys (SMAs) or pneumatics due to their speed, low power consumption, and high specific power. A magnetic-based sensing mechanism, decoupled from the HASEL electric field, provides interference-free deformation feedback. An elastic surface skin ensures low surface compliance. The 10x10 array is constructed using a hierarchical architecture: ten cells form a module sharing power and computation, and ten modules compose the display. The hierarchical approach minimizes components and complexity. An elastomeric skin covers the cells to create a continuous surface. The display is controlled by a PC and power supply, communicating with each module for global control. The HASEL actuator consists of a stack of liquid-dielectric-filled pouches with electrodes. An optoelectronic half-bridge driving circuit enables independent control of multiple actuators using a shared high-voltage power supply. Pulse-width modulation (PWM) of the LEDs controls the current and voltage across the actuator. An integrated high-voltage sensor facilitates closed-loop voltage regulation. The open-loop dynamics of the actuator and circuit were measured and used to design a closed-loop voltage regulator using a loop-shaping approach. The open-loop deformation response of the actuators was characterized using motion-captured surface deformation data and frequency domain analysis. The magnetic-based sensor uses a magnetic block and magnetometer to measure deformation. A third-order polynomial maps magnetometer readings to deformation. Calibration uses motion capture data. To measure external forces, a dynamic mechanical analyzer quasi-statically loaded actuators at different voltages, generating a force-displacement relationship and a force mapping based on the voltage and deformation. For closed-loop deformation control, a loop-shaping approach was used to design a controller that regulates desired deformation, running at 200 Hz. For object manipulation, a novel control algorithm using position and velocity feedback was developed. This algorithm generates local surface deformations to control ball rolling, using an overhead camera for ball position detection.

The researchers achieved high-speed surface deformation with a control rate of 200 Hz and actuation speed up to 50 Hz. The display demonstrated sensing of deformation with 0.1 mm resolution and force with 50 mN resolution. The magnetic-based sensing was accurate up to 30 Hz. The closed-loop deformation controller achieved a bandwidth of 20 Hz, significantly improving the accuracy of surface morphing compared to open-loop control. Traveling waves with speeds up to 354 cm/s were generated. The display successfully manipulated a ball along a planned trajectory and sorted multiple balls based on color. The display achieved precise closed-loop positional control of a ball. The mean travel time to complete a square trajectory was 28.48 s, and the ball trajectory remained within 1/2 cell diameter of the desired path. For the color sorting experiment, all balls were correctly sorted in all trials, with a mean sorting time of 4.66 s per ball. The soft, self-sensing shape morphing surface enables new functionalities like stimulus response mirroring and integration with peripheral devices via magnetic fields for user drawing.

This research demonstrates a significant advancement in the integration, sensing, and control of soft electrohydraulic arrays. The high-speed actuation, combined with precise sensing, enables a multitude of emergent capabilities. The electrohydraulic actuation offers advantages over other methods due to its speed, quiet operation, and low heat generation. The soft actuators maintain surface compliance for haptic interaction. The embedded sensors allow for novel functionalities like real-time object mass display and user-driven drawing. The limitations of the sensors are susceptibility to external magnetic fields and recalibration required for different locations. The modular design facilitates scalability, but limitations in circuit addressing and signal impedance could challenge further scaling. The high-degree-of-freedom soft robot hardware could be applied to other soft robotic systems.

This paper presents a multifunctional soft robotic shape display with integrated actuation, sensing, and control. The system's high-speed actuation, precise sensing, and advanced control algorithms enable various functionalities and applications not previously possible in shape displays. Future research could focus on scaling the system to larger sizes, improving the sensor robustness to external magnetic fields, and exploring additional applications of the technology. The design also has potential for adaptation to other high-degree-of-freedom soft robots.

The current sensor system is susceptible to interference from external magnetic materials, limiting the types of objects that can be manipulated on the display. The scalability of the system is currently limited by hardware constraints like circuit pin addressing, sensor signal impedance, and power consumption. Miniaturization of the cells could improve resolution but faces technical challenges in actuator fabrication and sensor resolution.