Research on the prediction of lower limb muscle forces based on multimodal sensor fusion and its application methods in exoskeleton control
DOI:
CSTR:
Author:
Affiliation:

1.Tianjin Key Laboratory for Advanced Mechatronic System Design and Intelligent Control, School of Mechanical Engineering, Tianjin University of Technology, Tianjin 300384, China; 2.National Demonstration Center for Experimental Mechanical and Electrical Engineering Education, Tianjin University of Technology, Tianjin 300384, China; 3.State Key Laboratory of Robotics and Intelligent Systems, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, China; 4.College of Electrical Engineering, Liaoning University of Technology, Jinzhou 121001, China

Clc Number:

TH-39TP242

Fund Project:

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    Accurate real-time perception of human muscle force is a core prerequisite for achieving "assistance-on-demand" control in lower-limb exoskeletons. To address the vulnerability of traditional surface electromyography (sEMG)based motion intention recognition to failure under challenging real-world conditions such as muscle fatigue, electrode displacement, and skin perspiration, this paper proposes a non-invasive lower-limb muscle force prediction and control method based on the multimodal fusion of depth vision and inertial measurement unit (IMU). First, a multimodal visual and kinematic dataset was constructed involving five healthy subjects across five typical daily terrains, including level ground and sloped walking at various gradients. The internal muscle forces of the quadriceps and hamstrings were calculated offline using an sEMG-driven Hill-type musculoskeletal model to serve as the supervised ground truth. Second, a dual-branch spatiotemporal feature fusion deep network was designed, which extracts spatial-geometric features from depth image sequences and temporal dynamics from inertial motion data through parallel channels. By incorporating an adaptive attention mechanism for the intelligent, heterogeneous weighted fusion of these features, the proposed framework completely eliminates the reliance on sEMG sensors during the practical application phase, thereby bypassing signal degradation issues. To bridge the gap from perception to closed-loop control, an adaptive impedance control framework driven by real-time predicted muscle force was established, continuously mapping variations in muscle force to the assistance stiffness of the exoskeleton. Experimental results demonstrate that the proposed method significantly outperforms unimodal approaches, yielding a robust cross-subject average coefficient of determination (R2) exceeding 0.81 and a root-mean-square error (RMSE) of less than 6 N. Furthermore, the predicted force curves maintain highly consistent profiles under simulated muscle fatigue interference, exhibiting superior robustness against physiological disturbances. Finally, platform validation on a commercial lower-limb exoskeleton system achieved high temporal phase synchronization and low lateral drift, providing a promising and viable new paradigm for the highly robust, assistance-on-demand control of exoskeletons in complex, dynamic environments.

    Reference
    Related
    Cited by
Get Citation
Related Videos

Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:
  • Revised:
  • Adopted:
  • Online: July 24,2026
  • Published:
Article QR Code