“Human hands rarely drop tools because our brain continuously adjusts grip force based on tactile and kinesthetic feedback,” explains Professor Fang. “We wanted to give prosthetic hands the same ability.” The team designed a multimodal controller called the Tactile, Kinesthetic, and EMG Bionic Gripping Controller (TKE‑BGC). It integrates three streams of information: surface electromyography (EMG) signals from the user’s residual muscles, tactile contact forces, and joint angles (kinesthetic data).
To train the controller, able‑bodied participants wore a data glove with embedded tactile sensors and bend sensors, plus three EMG electrodes. They performed two tool tasks: hammering nails and sawing wooden strips. These demonstrations provided a rich dataset of how a natural hand responds to rigid impacts and varying loads. The TKE‑BGC model uses a Transformer encoder to fuse the multimodal inputs and a multilayer perceptron to predict the next joint angle adjustments in real time. “The key innovation is that tactile and joint information act as a query to ‘weight’ the EMG features via cross‑attention,” says co‑author Boao Li. “The mimics the biological sensorimotor loop—physical feedback continuously modulates the motor command.”
The team compared TKE‑BGC against two conventional strategies: fixed force (FiF) and force follows (FoF). FiF applies a constant preload; FoF adjusts grip force based on real‑time load measurements but suffers from delays and over‑compensation. In offline tests using the demonstration data, TKE‑BGC produced joint motion predictions with much lower root‑mean‑square error, especially during the impact‑rich hammering task. “Fixed force cannot adapt at all, and force follows responds too late,” notes Professor Fang.
Online experiments involved six able‑bodied participants (wearing an extended limb) and three transradial amputees. They performed four tool tasks: two “seen” (hammering, sawing) and two “unseen” (peeler operation, desktop organization). For each task, participants used all three control methods in random order. The results were striking. TKE‑BGC significantly reduced the number of tool drops in every task and shortened completion times. In hammering, for example, FiF and FoF caused multiple drops, while TKE‑BGC kept the tool secure. Equally important, the average contact force with TKE‑BGC closely matched that of a natural human hand, whereas FiF and FoF exerted either too little or excessive force.
The benefits extended to user workload. EMG amplitude analysis showed that TKE‑BGC required much lower muscle activation than FiF and FoF (average EMG amplitude 0.0023 versus 0.0124 for FoF). Integrated EMG, which reflects overall muscle fatigue, was also lowest for the proposed method. “Amputees reported that they felt less exhausted and could complete tasks more naturally,” says co‑author Shuhui Wu. Subjective USE questionnaire scores were highest for TKE‑BGC, particularly in ease of use and satisfaction. Some users noted that FoF often pushed against the external force rather than anticipating it, making control feel unnatural.
Crucially, TKE‑BGC generalized well across subjects and tasks. Despite being trained on data from a single able‑bodied participant, it performed consistently for all six other able‑bodied users and the three amputees. It also transferred to the unseen peeler and desktop tasks without retraining, demonstrating strong task generalization. An ablation study confirmed that all three modalities contribute: removing tactile feedback caused the largest drop in prediction accuracy, underscoring the critical role of touch in dynamic manipulation.
The current tactile sensing is sparse (only nine force points), whereas human skin provides dense, multidimensional feedback. The team plans to integrate high‑density tactile sensors and process local tactile images with neural networks. They also aim to collect multi‑source demonstration data to capture personalized manipulation styles and to introduce optimization‑based mapping for adaptive joint posture alignment.
“This work moves prosthetic hand control from static grasping to real‑world tool handling,” concludes Professor Fang. “By transferring human manipulation skills through a multimodal, attention‑based controller, we can help amputees not only with daily living but also with vocational rehabilitation and returning to work.”
Authors of the paper include Boao Li, Shuhui Wu, Ting You, Shixian Wang, Ziming Chen, Ye Liu, Di Guo, Fuchun Sun, Guangyuan Xu, Du Jiang, Gongfa Li, and Bin Fang.
This work was supported by the Brain Science and Brain-like Intelligence Technology–National Science and Technology Major Project (grant no. 2025ZD0215600) and National Natural Science Foundation of China under grant nos. 62573063 and 62536001.
The paper “Dynamic Manipulation Skill Learning for Tactile Myoelectric Prosthetic Hands in Tool Handling” was published in the journal Cyborg and Bionic Systems on May. 13, 2026, at DOI: 10.34133/cbsystems.0572.
Cyborg and Bionic Systems