Challenges of controlling the rotation of virtual objects with variable grip using force-feedback gloves

The paper addresses how to reproduce realistic, dexterous object manipulation in VR, where current systems typically glue an object’s pose to the hand, limiting intrinsic finger-based adjustments. The authors focus on using grip strength as a control dimension: with gentle pressure, a real object can slip/rotate within the hand, while firm pressure fixes its orientation—affording additional rotational degrees of freedom without clutching. Prior controller-based implementations of variable grip suggested improved realism and satisfaction but introduced mapping and learning challenges. The research question is whether force-feedback gloves that directly render resistance to individual fingers can enable more natural, high-fidelity control of an object’s rotational freedom via variable grip strength. The authors implement a glove-based version of variable grip using the SenseGlove DK1 and evaluate it in a pick-and-place task measuring accuracy, time, grasp attempts, task load, perceived control, and presence.

The work builds on literature in VR haptics and object manipulation. Kinesthetic force-feedback devices for the hand/fingers have a long history (e.g., PHANToM; haptic gloves and exoskeletons) and are commonly used for translation, shape exploration, and weight simulation. Applying pressure as input has been explored with squeeze-based interactions on mobile devices and MR pinching, and for basic object manipulation. Perception of object elasticity has been studied via deformable objects and variable-stiffness proxies, including with force-feedback gloves. Prior variable-grip techniques on controller-based systems mapped trigger/handle pressure to grip strength, enabling an object to rotate freely under low grip and lock under high grip. These studies reported increased realism and satisfaction, with minor speed penalties. However, mapping button pressure to virtual grip introduced mental demand. The authors highlight the importance of tactile cues: friction and skin shear at fingertips are critical to maintaining secure grip and informing grip force, suggesting that kinesthetic-only gloves may be insufficient for precise grip modulation without additional cutaneous feedback.

System and apparatus: The prototype uses the SenseGlove DK1 force-feedback glove (300 g; dorsal exoskeleton with cable brakes) that restricts finger flexion by applying brake force (up to ~40 N at fingertip; ~200 Hz update; 100 force steps). Finger joint angles are sensed at 120 Hz with ~0.35° resolution. Hand position is tracked via HTC Vive Tracker; display via Valve Index HMD. The virtual environment is a photo-realistic workshop built in Unity 2020.3.23f1 with SteamVR Plugin v2.7.3 and SenseGlove Unity Plugin v2.3.1. Interaction technique and implementation: When grasping, the SenseGlove system attaches the object rigidly to the hand. To enable variable rotation, a visible copy of the object is spawned at grasp; the original (invisible while grasped) continues to provide consistent force feedback to the fingers in its original orientation relative to the hand. The copy follows the hand’s position but can rotate relative to the original depending on grip. A Unity Configurable Joint connects original and copy; the Slerp Drive Position Spring parameter controls rotational compliance. Lower spring values allow the copy to deviate more under gravity about an anchor between thumb and index fingertips. After pilot tests revealed insufficient precision with continuous grip control and risk of accidental drops at very low grip, the design used binary grip states: above 80% finger pressure (firm) the object is rotation-locked to the hand; below that threshold (loose) it is free to rotate with slight resistance. A visual bar above the hand indicates pressure, turning from green to red beyond the threshold. Pressure from thumb and index finger controls the grip state (all fingers still receive force feedback). Study design: Within-subjects comparison of two conditions: (1) Fixed grip (baseline, rigid attachment and rotation fixed to hand), and (2) Variable grip (binary grip; loose allows gravity-driven rotation; firm locks). Each participant performed two tasks per condition; condition order and task order were counterbalanced. Controller-based conditions from prior work were omitted to keep session length feasible. Tasks: Placement tolerance: < 3 cm position error and < 20° orientation error. Task A: move six identical cans from identical start poses to an identical target pose (repeated measures). Task B: move 12 items—four cans, four books, four milk cartons—from various predefined starting poses to a single target pose; order of items randomized per participant but kept identical across the two conditions. A reference behind the target area indicated desired orientation. Total planned manipulations: 756 ((6+12)*2*21). After outlier/system-error removal, 654 valid cases remained; 287 paired cases had valid data in both conditions. Procedure: After consent and demographics, participants were introduced to the HMD and glove; only the right glove was used, and only right-handed users were included to avoid bimanual interactions. Tutorials preceded each condition until the participant demonstrated proficiency. Participants then performed both tasks and completed in-VR questionnaires (Presence Questionnaire; raw NASA TLX; five custom 7-point items on precision, speed vs real world, grip awareness, developing a grip sense, and expected move/rotate behavior). A short post-study interview followed. Average session length: ~45 minutes. Sample: N=21 (6 female, 15 male), age 15–61 (mean 27.3). All right-handed. VR experience was low: 9 never used VR; 2 used VR at least monthly; 4 felt moderately/very experienced with VR. Ten had used VR controllers before; 8 used them for object manipulation. Only 4 had used hand tracking (3 for object handling). Eighteen participants used a glove for object manipulation for the first time. Data analysis: Nonparametric Wilcoxon signed-rank tests (two-sided; alpha .05; Bonferroni-Holm corrected) due to non-normality by Shapiro–Wilk (p = [0.001…0.061]). Effect sizes as matched pairs rank biserial (r_pb). Reliability of custom items: Cronbach’s alpha α = .71. Cleaned data and analyses available on OSF (https://osf.io/d64va).

Performance favored fixed grip. Translational accuracy: fixed grip median 8 mm (SD 6 mm) vs variable grip 9 mm (SD 7 mm); 1 mm difference, significant with small effect (Z = -2.4, p = .015, r_pb = .17). Rotational accuracy: fixed grip Mdn 4.2° (SD 5°) vs variable 6.1° (SD 5.6°); 1.9° difference, significant with small effect (Z = -3.2, p < .005, r_pb = .22). Completion time: fixed grip Mdn 3.8 s (SD 2.3) vs variable 5.2 s (SD 3.6); 1.4 s slower with variable, significant with large effect (Z = 3.4, p < .005, r_pb = .57). Grasp attempts: fixed grip Mdn 1 (SD 1.1) vs variable 2 (SD 1.5); significant with medium effect (Z = 4.6, p < .005, r_pb = .43). Questionnaires: Presence Questionnaire overall and most subscores showed no differences (p > .805), except interface quality, which favored fixed grip (Mdn_f = 12 ± 2.4 vs Mdn_v = 10 ± 2), large effect (Z = 2.9, p = .018, r_pb = .77). NASA TLX raw score trended lower workload for fixed grip (Mdn_f = 41.7 ± 9 vs Mdn_v = 43.3 ± 6.4) but not significant after correction (p = .092). TLX subitems showed higher demands for variable grip: mental demand (Z = 3.1, p = .014, r_pb = .89), performance (Z = 3.0, p = .018, r_pb = .76), and frustration (Z = -2.9, p = .02, r_pb = .78). One custom item (“I could move and rotate the items as I expected”) differed significantly between conditions with a large effect (Z = 2.9, p = .018, r ≈ .79). Overall, participants took longer, needed more grasps, and reported higher task load and lower perceived control with variable grip.

Contrary to expectations from prior controller-based studies, variable grip with a force-feedback glove yielded poorer performance, higher mental workload, and less perceived control. The authors attribute this to several factors: (1) Limitations of the SenseGlove DK1 prototype—kinesthetic resistance depends strongly on fingertip–surface contact angle, causing inconsistent or absent feedback at oblique contacts; pressure is inferred from brake resistance, not directly sensed, reducing precision. (2) Lack of cutaneous cues (e.g., skin shear, friction, slip) critical for regulating grip force and preventing drops; kinesthetic-only feedback may be insufficient for intuitive grip modulation. (3) Novice sample with steep learning curve for glove-based manipulation; many participants defaulted to safer fixed-grip strategies. (4) Accidental drops in variable grip increased handling time. (5) The visual grip indicator bar was distracting and did not compensate for missing tactile information. Despite these outcomes, the authors argue that mapping finger pressure to grip strength is not inherently undesirable; rather, higher-fidelity hardware (accurate actuation and pressure sensing), better force-vector estimation per finger, and added cutaneous feedback may enable the anticipated benefits.

The authors present a glove-based system that maps finger pressure to rotational freedom of a held object: loose grip allows gravity-driven reorientation, while firm grip locks orientation to the hand. In a pick-and-place study with SenseGlove DK1, variable grip resulted in slower performance, more attempts, higher workload, and reduced perceived control than fixed grip. These results likely stem from hardware and feedback limitations and novice users, not from the concept of variable grip itself. Future research should integrate cutaneous cues (shear, friction, slip), improve physics and force rendering fidelity, use direct fingertip pressure sensing and per-finger force vector estimation, avoid interpenetration, and support continuous grip. The technique may also be beneficial in tasks without gravity when fine rotational adjustment is desired (e.g., surgical/assembly training) and in MR with improved optical hand tracking and physics-based grasp simulations.

- Hardware constraints of SenseGlove DK1: kinesthetic resistance highly dependent on contact angle; inconsistent or absent feedback at oblique contacts; only flexion inhibition (cannot actively move fingers); pressure only approximated from brake resistance (no direct fingertip pressure sensing). - No cutaneous feedback (e.g., skin shear, friction, slip), which is crucial for regulating grip force and preventing drops; likely contributes to higher workload and accidental releases. - Interaction design compromises: binary grip threshold at 80% (vs intended continuous control); reliance on thumb and index pressure only; visual grip indicator potentially distracting; object–hand interpenetration allowed to increase rotational freedom, reducing realism. - Novice participant sample with minimal prior glove/hand-tracking experience; steep learning curve; only right-handed, single-glove use (no bimanual interactions). - Did not include controller-based comparison conditions from prior work to limit session length, reducing cross-technology comparability. - Some objects treated as rigid for feedback calculations (no elasticity), potentially reducing realism for deformable items (e.g., milk carton).