Basic Functions
The basic functions of our robot are explained. These techniques are combined to achieve a complex robot’s action.
Object Recognition
YolactEdge[1] is used for object recognition and the result is used for object grasping point estimation. This is an important process for the robot to find an object and grasp it. We, Hibikino-Musashi@Home focus on this area.
Human Detection
Human attributions are recognized by using CSRA[2] and the pointing position is estimated by using LightWightHumanPose[3]. These functions are used to explain the attributes of humans and to bring a pointed object to a human.
Voice Recognition
Noise is reduced by noisereduce[4] and the voice is processed by using Vosk[5]. Voice recognition is important to communicate the robot with humans. A robust recognition system that recognizes spoken words in a noisy crowd has been developed.
Environment Recognition
The placable area and travelable area are estimated by semantic segmentation. These techniques are used to place an object in an empty space on a shelf and travel by avoiding obstacles on a floor.
Dataset Generation and Sim2Real[6]
A large amount of training data should be prepared to train the object recognition system. Hibikino-Musashi@Home develops dataset generation software using a 3D simulator to train the object recognition system.
First, an object is scanned using a 3D scanner to get 3D data about the object. Then, the scanned 3D data is taken in the simulator to generate a dataset. The objects are placed randomly in a room simulated in a 3D environment. By changing the background image randomly, the recognition system with high accuracy can be trained.
By using this method, 100 thousand datasets with mask and label automatically within one hour.
Open-source Simulator
Hibikino-Musashi@Home published the development system as an open source.
https://github.com/Hibikino-Musashi-Home/hma_wrs_sim_ws
The following link is the ROS workspace for the tidy-up robot developed by Hibikino-Musashi@Home. This workspace uses an OSS HSR simulator[7]. A developer who does not have a physical HSR can start development easily using libraries developed by Hibikino-Musashi@Home.
Research Activities
Brain-inspired Artificial Intelligence[8]
Brain-inspired Artificial Intelligence, which adopts brain functions to obtain a family’s preference or behavior through the small amount of robot experience has been developed. It can acquire knowledge that differs in each home environment and is difficult to acquire by using deep learning. The goal is to implement the model into the digital or analog circuit to develop a highly efficient and low power consumption system for robots.
Analog Chip and Soft Hand[9]
This object recognition system uses tactile information taken by sensors made of soft materials and process tactile information on an analog chip. The tactile sensor is made of silicone rubber with liquid metal. It can hold an object gently without damage. The circuit is implemented on an analog chip with 300 TOPS/W of high performance and power efficiency, which is only a few millimeters in size.
The combination of the dedicated robot sensor can improve recognition accuracy, and this approach can develop a long battery life robot system.
References
[1] H. Liu et al., ICRA, 2021.
[2] K. Zhu et al., ICCV 2021.
[4] vosk-api, https://github.com/alphacep/vosk-api.
[5] T. Sainburg et al., PLoS computational biology, 2020.
[6] T. Ono et al., Advanced Robotics, 2022.
[7] hsrb_robocup_dspl_docker, https://github.com/hsr-project/hsrb_robocup_dspl_docker
[8] H. Nakagawa et al., IEEE Access, 2022.
[9] M.Yamaguchi et al., IJCNN 2019.
