With the development of sensor and communication technologies, body sensor networks(BSNs) have become an indispensable part of smart medical services by monitoring the real-time state of users. Due to introducing of smart medical robots, BSNs are not related to users, but also responsible for data acquisition and multi-sensor fusion in medical human–robot interaction scenarios. In this paper, a hybrid body sensor network architecture based on multi-sensor fusion(HBMF) is designed to support the most advanced smart medical services, which combines various sensor, communication, robot, and data processing technologies. The infrastructure and system functions are described in detail and compared with other architectures. Especially, A multi-sensor fusion method based on interpretable neural network(MFIN) for BSNs in medical human–robot interaction scenario is designed and analyzed to improve the performance of fusion decision-making. Compared with the current multi-sensor fusion methods, our design guarantees both the flexibility and reliability of the service in the medical human–robot interaction scenario.