Abstract:Human beings can accomplish decisionmaking and dexterous manipulation with their complex sensory perception and motor control capabilities. The fusion of visual and auditory feedback together with haptic feedback, enables close resemblance of manipulation in the virtual scenarios with that in the physical world. Further, it is challenging to coordinate the human intervention and the autonomy of the robot in the virtual scenario. Here, a virtual scenario is built up with Unity3D, taking advantages of its powerful rendering capabilities. Then, bidirectional communication between Unity3D and the omega.7 haptic platform is realized. After that, virtual range sensors based on collision detection are designed and implemented for environment perception of the avatar in the virtual scenario. Further, a humanrobot haptic shared control strategy is designed and realized, enabling the avatar to move under the shared control of the dexterous operation by the human and the autonomous algorithms. The control authority can shift smoothly between the human operation and robot autonomy. Thus, a dexterous haptic interaction solution with Unity3D and the omega.7 haptic interface is established, which can find promising applications in rehabilitation, teleoperation, military missions and also the entertainment industry.