A soft magnetoelectric finger for robots' multidirectional tactile perception in non-visual recognition environments

The study addresses the need for robotic fingers with multidirectional tactile sensing for exploration in complex, non-visual environments. Existing fingertip tactile sensors (e.g., optical fiber-based, Biotac, magnetic MEMS skins) primarily provide unidirectional force detection, limiting their capability for complex tasks. Vision-based tactile sensors (Insight, OmniTact, GelTip) offer multidirectional perception but require complex opto-mechanical systems and algorithms. Hall-effect array skins (uSkin) can detect multidirectional forces but are rigid and require external power. Flexible magnetoelectric sensors based on Faraday induction offer self-generated signals, high sensitivity, and simple structures but prior designs often detect only unidirectional forces or use rigid copper coils. The authors propose a soft magnetoelectric finger (SMF) leveraging liquid metal coils and an internal magnet to realize self-generated, multidirectional tactile sensing suitable for robotic fingers, enabling object recognition and navigation in non-visual conditions.

Prior work on tactile sensing includes: optical-fiber sensors and Biotac multimodal sensors that mainly detect single-direction forces; vision-based tactile sensors (Insight, OmniTact, GelTip) that achieve multidirectional sensing by converting touch to images but at the cost of complex structures, optics, and algorithms; Hall-effect array-based skins (uSkin) that measure magnetic field changes for multidirectional sensing but are rigid and require external power. Self-generated magnetoelectric sensors based on electromagnetic induction have been explored: strain sensors that monitor joint deformation only; Merkel’s disks-inspired sensors integrated into robotic arms but limited to unidirectional sensing; and stretchable tentacle sensor arrays for non-visual recognition that are difficult to integrate for multidirectional fingertip sensing and often use rigid copper coils. Liquid metal (Ga-In alloy) provides soft, stretchable conductors suitable for flexible magnetoelectric sensors. The proposed SMF compares favorably by offering self-generated signals, multidirectional sensing, softness, and simple structure.

Design and fabrication:

• A flexible Ecoflex sheath (≈3 mm thick) with five molded microchannels was fabricated using 3D-printed inner/outer molds and cured at 50 °C for ~30 min. Channels were filled with EGaIn liquid metal (≥99.99%), and copper leads were attached using conductive silver glue (cured at 100 °C for 60 min). The coils were encapsulated with Ecoflex (50 °C, ~30 min).
• A 3D-printed phalangeal “bone” housed a 5 mm-side NdFeB magnet (~0.35 T at N-pole surface). A 5 mm gap between bone and sheath allows compliant deformation. The sheath was mounted on the bone forming five LM coils (Ch1–Ch5) around the magnet.

Characterization and instrumentation:

• Real-time induced voltages acquired via LinkZill; mechanical loading applied with a non-magnetic Instron E3000. Magnetic fields measured with a Teslameter and a 3D stray-field test system. Optical/SEM/EDS characterized LM–Ecoflex interfaces.

Sensing tests:

• Normal force tests: Pressing Ch1 side with displacement 3.5 mm and speed 10 mm s⁻1 three times; recorded peak voltages across channels; varied speed and displacement to assess sensitivity, detection range, and speed dependence. Stability over 10,000 cycles; vibration response 1–6 Hz. Side channels (Ch4, Ch5) tested with spherical indenter; minimum detectable forces assessed.
• Shear force tests: Defined orthogonal reference frame and force application points (e.g., P2, P3, P23). Applied shear forces in multiple directions (e.g., 4→3, 4→1, 4→5). Identified characteristic “downward-upward” vs “upward-downward” voltage peak signatures across channels to disambiguate shear direction.

Simulation:

• Abaqus computed deformation of the compliant sheath under loading. Pre- and post-deformation geometries imported into Ansys Maxwell to compute magnetic field distribution and flux through each coil. NdFeB parameters: Hc = −954,929.6 A m⁻1, Br ≈ 1.2 T. Simulated flux changes converted to predicted induced voltages via Faraday’s law and validated against experiments; 3D stray field measurements corroborated simulations.

Object recognition and machine learning:

• Six equal-sized materials (sponge cube, porous plastics, paperboard, foamed plastics, Ecoflex rubber, cured resin). Measured stress–strain curves; estimated Young’s moduli (0–20% strain linear fit). SMF contacted at 10 mm s⁻1 and 3.5 mm displacement for 360 pulses per class; Ch1 voltages collected.
• Data split into 120 strips per class; feature engineering included max, min, skewness, kurtosis, RMS, std, variance, peak-to-peak, etc. Train/test split 80/20 (576/144 samples). Evaluated five ML algorithms; LightGBM achieved best performance.

Black box exploration:

• Integrated SMF on a robotic arm to explore a dark box with walls W1–W4 containing hemispherical obstacles and floor regions R1–R4 (pits/planes/obstacles). Step 1: Perimeter scan along walls at 10 mm s⁻1; used shear signal signatures to localize obstacles within partitioned wall regions. Step 2: Vertical probing over R1–R4 with incremental descent (2–15 mm) to classify pit/plane/obstacle based on contact detection depth. Step 3: Press obstacles (3× at 10 mm s⁻1, 3.5 mm) and classify material using trained LightGBM, reporting class probabilities.

• Self-generated, multidirectional tactile sensing: five LM coils around an internal magnet produce distinct induced voltages enabling discrimination of normal and shear forces and their directions.
• Vertical pressing on Ch1 (3.5 mm, 10 mm s⁻1): average peak voltages ≈ Ch1 −9.51 µV; Ch2 1.33 µV; Ch3 0.55 µV; Ch4 1.52 µV; Ch5 0.82 µV.
• Faraday-based modeling matched experiments: Ch1 flux change 3.14×10⁻6 Wb over ~0.35 s predicts ~8.97 µV vs measured ~9.51 µV.
• Force detection range: ~0.04 N (≈8.0 kPa) to 15.0 N (≈98.68 kPa). Minimum detectable force: Ch1 ~0.04 N; Ch4 ~0.18 N; Ch5 ~0.43 N.
• Sensitivity (Ch1 at 10 mm s⁻1): 3.87 µV N⁻1 (≤2.5 N); 0.83 µV N⁻1 (>2.5 N). Voltage increases with compression speed and displacement; response requires dynamic (non-static) contact.
• Durability: stable responses after 10,000 press–release cycles.
• Vibration response: reliable in 1–6 Hz range (peak increases with frequency for same displacement due to recovery dynamics).
• Coil turns: three-turn LM coils increased signal ~70% vs single-turn; simulations aligned with measurements.
• Shear force discrimination: characteristic peak polarities across channels uniquely identify shear directions (e.g., 4→3 shows Ch4 down–up, Ch3 up–down).
• Object recognition: LightGBM achieved 97.46% average accuracy across six materials; notable confusion primarily between porous plastics and Ecoflex rubber in tests.
• Black box exploration: correctly localized wall obstacles (e.g., W1 region 1, W2 region 2, W3 region 1, W4 none). Floor probing classified R1 as pit, R4 as 1 mm plane; identified R2 porous plastic (prob. 0.609) and R3 cured resin (prob. 0.958).

The SMF addresses the challenge of achieving multidirectional tactile perception with a soft, finger-like sensor that generates its own signals without external power for sensing. By combining a deformable LM coil array with an internal magnet, dynamic contact deformations translate into distinct induced voltages that encode both normal and shear forces. Simulations and measurements confirm that deformation-induced flux changes explain signal magnitudes and polarities. These capabilities enable practical tasks in non-visual environments, demonstrated by accurate object classification and spatial exploration/localization within a black box using only tactile cues. The approach thus advances tactile sensing for robotics by offering a simple, soft, and integrable sensor that provides rich multidirectional information suitable for navigation and recognition.

This work presents a soft magnetoelectric finger that provides self-generated, multidirectional tactile sensing via LM coil arrays and an internal magnet. It discriminates force direction (normal and shear), detects small forces, endures extensive cycling, and enables material recognition (97.46% accuracy) and non-visual exploration when integrated with a robotic arm. Future research directions include: increasing the number of LM coils for finer directional resolution; improving precision, stability, and cost-effectiveness (e.g., optimizing multi-turn coil fitting on 3D surfaces); expanding recognition beyond Young’s modulus differences by fusing modalities (e.g., vision); and advancing data processing and machine learning for more robust force estimation under variable speeds.

• Accurate force quantification currently requires a known compression speed; speed variations affect induced voltage, limiting static force measurement and complicating use without speed control.
• Limited number of sensing units (five coils) constrains directional resolution; more coils would improve multidirectional perception.
• Needs improvements in precision, stability, and cost, including better fitting/placement of multi-turn LM coils on irregular 3D surfaces and robust encapsulation.
• Object recognition primarily based on differences in Young’s modulus; distinguishing objects with similar stiffness may require multimodal sensing (e.g., visual fusion).
• As a self-generating sensor reliant on motion-induced flux changes, it detects dynamic contacts better than static loads.