Exi@ is a robot developed by Hibikino-Musashi@Home. To have Exi@ do the same job as a human, it has various sensors and actuators including an arm for carrying objects and a microphone for enabling voice interaction.
Also, to incorporate a better system, our robots have evolved over and over.
For detailed information on hardware, please see below.
|Manipulators||Exact Dynamics iARM|
|Camera||ASUS Xtion Camera
* Attach to pan tilt / lift mechanism
|LRF||Hokuyo UTM-30LX laser range finder|
|Microphone||SANKEN CS-3e Shotgun microphone|
|Batteries||Lead-acid battery 12V
Lead-acid battery 24V
|Computer||ThinkPad PC Core-i5 4850U processor and 12Gb RAM x2|
|FPGA board||Xilinx ZedBoard (FPGA+ARM)|
|Base||About 600mm x 600mm|
Exi@’s robot system is based on the Robot Operating System (ROS). The ROS is the de facto standard middleware for robots. We focus on developing new technologies such as voice interaction and image processing, using SLAM and AR marker recognition parts using open source packages, thereby reducing bugs and improving development efficiency.
Exi@ has two computers and one FPGA. The software configuration is as follows.
Exi@ has a number of distinctive software.
Object Detection / Recognition
Point Cloud Library (PCL) is used for object detection. We extract only the object by removing the plane using the plane detection algorithm, then apply Deep Learning to the extracted object and recognize the object. In the Deep Learning part, we use GoogLeNet , which is a 22 layered Convolutional Neural Network learned according to the data set of Image Net Large Scale Visual Recognition Challenge 2012 . Based on the GoogLeNet, we employ transfer learning and its final layer is fine-tuned by the data set dedicated for RoboCup@Home.
Robot Arm Visual Feedback
For more accurate object grasping, we introduce a visual feedback function of the arm to Exi@. Using the marker attached to the arm of Exi@ (iARM), it is done in the following way.
- Obtain the coordinates of the object by PCL and Deep Learning.
- Move the arm to the vicinity of the object.
- By using ar_track_alvar, a marker of the end effector is detected by the RGB-D camera and feedback control is performed to minimize the error with respect to the target coordinates.
For other details of the installed software, please refer to the following specification table.
|State Management||SMACH (ROS)|
|Voice Interaction||Speech Recognition
|Intel RealSense SDK 2016 R2|
|Morphological Analysis Dependency Structure Analysis
|Dependency Structure Analysis
|Speech Synthesis||Open JTalk|
|Image Processing||Object Detection||Point Cloud Library (PCL)|
|Object Recognition||Caffe + GoogLeNet (Transfer Learning, relearning for the final layer)
GoogLeNet：Christian Szegedy, et. al., “Going Deeper with Convolutions,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015.
|Human Detection / Tracking||Depth image＋particle filter (FPGA implementation)|
|Robotic arm visual feedback||AR Mark Tracking||ar_track_alivar (ROS)|
|Self Navigation||SLAM||slam_gmapping (ROS)|
|Path Planning||move_base (ROS)|
 “ImageNet Large Scale Visual Recognition Challenge 2012,” http://image-net.org/challenges/LSVRC/2012/
 Christian Szegedy, et. al. “Going Deeper with Convolutions,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015.
Advantages of our Exi@
We use a Field Programmable Gate Array (FPGA) for our robot system. FPGAs are hardware that can reconfigure its internal circuit. Since FPGAs can process multiple operations in parallel, it has the advantage of being able to handle time-consuming operations at high speed, compared to the CPU.
In Hibikino-Musashi@Home, we develop an interface “COMTA” that enables us to easily access FPGAs from the ROS, and is installed on Exi@. We are aiming to realize a real-time and low-power intelligent processing system by offloading a part of heavy processing to FPGAs. In addition, we improve processing capacity per power consumption by executing intelligent processing with FPGAs.
We are also developing Exi@ as a robot platform for imprementing research results.
For instance, we have developed a robot control system that uses brain waves and a robot application that measures human health status with a contactless device.
 (In Japanese): 辻湧弥, 湯澤慶太，夏目季代久, “脳波SSVEPを用いたブレインロボットインターフェースシステムの構築,” 第3回インテリジェントホームロボティクス研究会, 11月, 2015.
 Japanese Patent, Application Number 2013-557382, 特願2013-557382号, 「生体情報処理装置、生体情報処理システム、生体情報の圧縮方法、及び、生体情報の圧縮処理プログラム」, 佐藤寧, 2013年8月.