Human Support Robot (HSR)

Outline

HSR is a Home Service Robot developed by TOYOTA MOTOR CORPORATION.
This robot is provided as an open platform, and other universities and research institutions are also working on development. Hibikino-Musashi@Home has ported know-how gained through Exi@ development and software assets developed up to now to HSR, and is developing highly functional robots.

We have successfully transplanted “state management”, “voice interaction”, “image processing” algorithms related to “Speech Recognition & Audio Detection” and “Object Recognition & Manipulation” which are RoboCup@Home competitions from Exi@.

Hardware

In Hibikino-Musashi@Home, we customize HSR by installing PCs and microphones externally. By installing one notebook PC on the back of HSR and connecting it with HSR in hsrb mode via wired Ethernet, we are controlling HSR from the notebook PC.

In addition, by installing “Omni-directional microphone” for speech recognition and “Egg-shaped microphone array” for sound source localisation, it is configured to be able to recognise ambient sound stably.

For detailed information on hardware, please see below.

Name Human Support Robot (HSR)
Size Footprint φ430mm
Height (min/max) 1,005/1,350mm (top of the head height)
Weight About 37kg
Moving base Sensors •Laser Range Sensor
•IMU
Arm length 600mm
Payload (recommended/max) 0.5/1.2kg
Gripper Sensors •Gripping force sensor
•Wide-angle camera
Head Sensors •RGB-D sensor x1
•Stereo camera x1
•Wide-angle camera x1
•Microphone array x1
Body Expandability •USB x3
•VGA x1
•LAN x1
•Serial x1
•12V-0.5A output x1

Software

Like Exi@, we are developing software based on the Robot Operating System (ROS) for HSR as well.

In Hibikino-Musashi@Home, we realize rapid system development by porting software group developed for Exi@, including “state management”, “voice interaction” and ”image processing” algorithms, to HSR.

Speech Recognition / Sound Source Localization

In HSR, we customised the robot by two types of omnidirectional microphones and egg-shaped microphone array to be mounted on HSR. HSR voice recognition / sound source localization operates as follows.

  1. The omnidirectional microphone captures the voice of the questioner and uses the speech recognition engine Julius to recognize the speaker’s speech.
  2. The voice of the questioner is captured by an egg-shaped microphone array, and sound source localization is performed by applying the MUSIC method using HARK which is a robot auditory software.

For other details of the software, please see the following specification table.

System OS Ubuntu14.04
Middleware ROS Indigo
State Management smach (ROS)
Voice Interaction Speech Recognition
(English)
Intel RealSense SDK 2016 R2
Morphological Analysis Dependency Structure Analysis
(English)
SyntaxNet
Speech Recognition
(Japanese)
Julius
Morphological Analysis
(Japanese)
MeCab
Dependency Structure Analysis
(Japanese)
CaboCha
Speech Synthesis Open JTalk
Sound Location HARK
Image Processing Object Detection Point Cloud Library (PCL)
Object Recognition Caffe + GoogLeNet (Transfer Learning, relearning for the final layer)
GoogLeNet:Christian Szegedy, et. al.: \Going Deeper with Convolutions”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.1-9, 2015.
Self Navigation SLAM slam_gmapping (ROS)
Path Planning move_base (ROS)

Advantages of HSR

In HSR, software and algorithms for controlling robots such as SLAM and arm control are incorporated in advance.

Therefore, we combine the stable operation of HSR with the high-performance software assets developed on Exi@ to realize a high-performance and stable Home Service Robot.