Exi@

Outline

Exi@ is a robot developed by Hibikino-Musashi@Home. To have Exi@ do the same job as a human, it has various sensors and actuators including an arm for carrying objects and a microphone for enabling voice interaction.

Hardware

Exi@ has hardware corresponding to human functionalities so that it can do the same work as a human being. 

Also, to incorporate a better system, our robots have evolved over and over.

For detailed information on hardware, please see below.

Name Exi@
Base RoboPlus EXIA
Manipulators Exact Dynamics iARM
Camera ASUS Xtion Camera
* Attach to pan tilt / lift mechanism
LRF Hokuyo UTM-30LX laser range finder
Microphone SANKEN CS-3e Shotgun microphone
Batteries Lead-acid battery 12V
Lead-acid battery 24V
Computer ThinkPad PC Core-i5 4850U processor and 12Gb RAM x2
FPGA board Xilinx ZedBoard (FPGA+ARM)
Height About 1500mm
Weight About 80kg
Base About 600mm x 600mm

Software

Exi@’s robot system is based on the Robot Operating System (ROS). The ROS is the de facto standard middleware for robots. We focus on developing new technologies such as voice interaction and image processing, using SLAM and AR marker recognition parts using open source packages, thereby reducing bugs and improving development efficiency.

Exi@ has two computers and one FPGA. The software configuration is as follows.

 

Exi@ has a number of distinctive software.

Object Detection / Recognition

Point Cloud Library (PCL) is used for object detection. We extract only the object by removing the plane using the plane detection algorithm, then apply Deep Learning to the extracted object and recognize the object. In the Deep Learning part, we use GoogLeNet [2], which is a 22 layered Convolutional Neural Network learned according to the data set of Image Net Large Scale Visual Recognition Challenge 2012 [1]. Based on the GoogLeNet, we employ transfer learning and its final layer is fine-tuned by the data set dedicated for RoboCup@Home.

Robot Arm Visual Feedback

For more accurate object grasping, we introduce a visual feedback function of the arm to Exi@. Using the marker attached to the arm of Exi@ (iARM), it is done in the following way.

  1. Obtain the coordinates of the object by PCL and Deep Learning.
  2. Move the arm to the vicinity of the object.
  3. By using ar_track_alvar, a marker of the end effector is detected by the RGB-D camera and feedback control is performed to minimize the error with respect to the target coordinates.

For other details of the installed software, please refer to the following specification table.

System OS Ubuntu14.04
Middleware ROS Indigo
State Management SMACH (ROS)
Voice Interaction Speech Recognition
(English)
Intel RealSense SDK 2016 R2
Morphological Analysis Dependency Structure Analysis
(English)
SyntaxNet
Speech Recognition
(Japanese)
Julius
Morphological Analysis
(Japanese)
MeCab
Dependency Structure Analysis
(Japanese)
CaboCha
Speech Synthesis Open JTalk
Sound Location HARK
Image Processing Object Detection Point Cloud Library (PCL)
Object Recognition Caffe + GoogLeNet (Transfer Learning, relearning for the final layer)
GoogLeNet:Christian Szegedy, et. al., “Going Deeper with Convolutions,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015.
Human Detection / Tracking Depth image+particle filter (FPGA implementation)
Robotic arm visual feedback AR Mark Tracking ar_track_alivar (ROS)
Self Navigation SLAM slam_gmapping (ROS)
Path Planning move_base (ROS)


[1] “ImageNet Large Scale Visual Recognition Challenge 2012,”                              http://image-net.org/challenges/LSVRC/2012/

[2] Christian Szegedy, et. al. “Going Deeper with Convolutions,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015.

Advantages of our Exi@

We use a Field Programmable Gate Array (FPGA) for our robot system. FPGAs are hardware that can reconfigure its internal circuit. Since FPGAs can process multiple operations in parallel, it has the advantage of being able to handle time-consuming operations at high speed, compared to the CPU.

In Hibikino-Musashi@Home, we develop an interface “COMTA” that enables us to easily access FPGAs from the ROS, and is installed on Exi@. We are aiming to realize a real-time and low-power intelligent processing system by offloading a part of heavy processing to FPGAs. In addition, we improve processing capacity per power consumption by executing intelligent processing with FPGAs.

We are also developing Exi@ as a robot platform for imprementing research results.
For instance, we have developed a robot control system that uses brain waves and a robot application that measures human health status with a contactless device.

[3] (In Japanese): 辻湧弥, 湯澤慶太,夏目季代久, “脳波SSVEPを用いたブレインロボットインターフェースシステムの構築,” 第3回インテリジェントホームロボティクス研究会, 11月, 2015.

[4] Japanese Patent, Application Number 2013-557382, 特願2013-557382号, 「生体情報処理装置、生体情報処理システム、生体情報の圧縮方法、及び、生体情報の圧縮処理プログラム」, 佐藤寧, 2013年8月.