HSR is a Home Service Robot developed by TOYOTA MOTOR CORPORATION.
This robot is provided as an open platform, and other universities and research institutions are also working on development. Hibikino-Musashi@Home has ported know-how gained through Exi@ development and software assets developed up to now to HSR, and is developing highly functional robots.
We have successfully transplanted “state management”, “voice interaction”, “image processing” algorithms related to “Speech Recognition & Audio Detection” and “Object Recognition & Manipulation” which are RoboCup@Home competitions from Exi@.
In Hibikino-Musashi@Home, we customize HSR by installing PCs and microphones externally. By installing one notebook PC on the back of HSR and connecting it with HSR in hsrb mode via wired Ethernet, we are controlling HSR from the notebook PC.
In addition, by installing “Omni-directional microphone” for speech recognition and “Egg-shaped microphone array” for sound source localisation, it is configured to be able to recognise ambient sound stably.
For detailed information on hardware, please see below.
|Name||Human Support Robot (HSR)|
|Height (min/max)||1,005/1,350mm (top of the head height)|
|Moving base||Sensors||•Laser Range Sensor
|Gripper||Sensors||•Gripping force sensor
|Head||Sensors||•RGB-D sensor x1
•Stereo camera x1
•Wide-angle camera x1
•Microphone array x1
•12V-0.5A output x1
Like Exi@, we are developing software based on the Robot Operating System (ROS) for HSR as well.
In Hibikino-Musashi@Home, we realize rapid system development by porting software group developed for Exi@, including “state management”, “voice interaction” and ”image processing” algorithms, to HSR.
Speech Recognition / Sound Source Localization
In HSR, we customised the robot by two types of omnidirectional microphones and egg-shaped microphone array to be mounted on HSR. HSR voice recognition / sound source localization operates as follows.
- The omnidirectional microphone captures the voice of the questioner and uses the speech recognition engine Julius to recognize the speaker’s speech.
- The voice of the questioner is captured by an egg-shaped microphone array, and sound source localization is performed by applying the MUSIC method using HARK which is a robot auditory software.
For other details of the software, please see the following specification table.
|State Management||smach (ROS)|
|Voice Interaction||Speech Recognition
|Intel RealSense SDK 2016 R2|
|Morphological Analysis Dependency Structure Analysis
|Dependency Structure Analysis
|Speech Synthesis||Open JTalk|
|Image Processing||Object Detection||Point Cloud Library (PCL)|
|Object Recognition||Caffe + GoogLeNet (Transfer Learning, relearning for the final layer)
GoogLeNet：Christian Szegedy, et. al.: \Going Deeper with Convolutions”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.1-9, 2015.
|Self Navigation||SLAM||slam_gmapping (ROS)|
|Path Planning||move_base (ROS)|
Advantages of HSR
In HSR, software and algorithms for controlling robots such as SLAM and arm control are incorporated in advance.
Therefore, we combine the stable operation of HSR with the high-performance software assets developed on Exi@ to realize a high-performance and stable Home Service Robot.