基于辅助喂食全过程意图识别的多模态安全交互方法研究
DOI:
CSTR:
作者:
作者单位:

作者简介:

通讯作者:

中图分类号:

TN911. 72 TH711 TH9

基金项目:

国家自然科学基金项目( 92248304)、辽宁省人工智能重点研发计划项目( 2023JH26 / 10200018)、辽宁省教育厅基本科研项目(LJ212410142073)资助


A multimodal safe interaction method based on intention recognition in the whole process of assisted feeding
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    为提高无法自主进食的弱机能人群面对被喂食过程中的安全性与灵活性,提出一种面向辅助喂食行为的多模态安全人 机交互框架及其全过程意图识别算法。 首先,结合用户特征及安全性与灵活性需求,提出一种结合视、触、力、位与语言融合的 多模态人机交互框架并构建辅助喂食系统。 其次,针对进食全过程包括对进食意图、菜品选择意图、动态喂食点估计与递送位 姿计算、咀嚼意图等,提出一种基于视觉为主导的喂食意图全过程识别方法。 选取有效刻画进食过程面部动态变化的特征点并 设计融合口部纵横比与下颌纵横比识别算法,通过视线矢量分析用户菜品选择意图并基于面部实时位姿设计动态喂食点,从而 形成全过程动态意图准确识别。 同时,在辅助喂食虚拟映射系统中,结合大语言模型对交互全过程中的模糊意图、临时变更意 图进行提问,形成反馈机制进而提升交互安全性。 最终通过仿真与综合性实验验证提出的方法,通过多模态交互框架可以有效 提升辅助喂食过程的灵活性,同时结合大语言模型形成模糊意图与变更意图有效反馈,最终提升交互过程的安全性,该方法为 弱机能人群的日常生活辅助喂食行为提供了新型护理方法。

    Abstract:

    To improve the safety and flexibility of the feeding process for individuals with limited mobility who are unable to feed themselves, this study proposes a multimodal human-robot interaction framework for assisted feeding and a whole-process intention recognition algorithm. First, based on user characteristics and the requirements for safety and flexibility, a multimodal interaction framework integrating vision, touch, force, position, and language fusion is proposed, and an assisted feeding system is developed. Second, a vision-driven whole-process intention recognition method is introduced to address the entire feeding process, including feeding intention, dish selection intention, dynamic feeding point estimation, delivery pose calculation, and chewing intention. Key facial feature points that effectively capture dynamic changes during feeding are selected, and an algorithm combining the aspect ratio of the mouth and the mandible is designed. The user′s dish selection intention is analyzed through gaze vector estimation, and dynamic feeding points are determined based on real-time facial pose tracking, enabling accurate recognition of dynamic intentions throughout the process. Furthermore, in the virtual mapping system for assisted feeding, a feedback mechanism is established by leveraging a large language model to clarify ambiguous intentions and adapt to temporary changes during the interaction, thereby enhancing safety. Finally, the proposed method is validated through simulations and comprehensive experiments. The results demonstrate that the multimodal interaction framework significantly improves the flexibility of the assisted feeding process, while the integration of the large language model provides effective feedback for ambiguous and changing intentions, ultimately enhancing the safety of the interaction. This approach offers a novel care solution for assisting feeding behaviors in the daily lives of individuals with limited mobility

    参考文献
    相似文献
    引证文献
引用本文

吕相谊,赵东辉,丁嘉辉,杨俊友,王硕玉.基于辅助喂食全过程意图识别的多模态安全交互方法研究[J].仪器仪表学报,2025,46(1):351-362

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:
  • 最后修改日期:
  • 录用日期:
  • 在线发布日期: 2025-04-08
  • 出版日期:
文章二维码