{"id":1130,"date":"2022-11-05T02:01:02","date_gmt":"2022-11-04T17:01:02","guid":{"rendered":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/?page_id=1130"},"modified":"2022-11-13T02:08:21","modified_gmt":"2022-11-12T17:08:21","slug":"techniques_en","status":"publish","type":"page","link":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/techniques_en\/","title":{"rendered":"Our Techniques"},"content":{"rendered":"\n

Basic Functions<\/h2>\n\n\n\n

The basic functions of our robot are explained. These techniques are combined to achieve a complex robot’s action.<\/p>\n\n\n\n

Object Recognition<\/h3>\n\n\n\n

YolactEdge[1] is used for object recognition and the result is used for object grasping point estimation. This is an important process for the robot to find an object and grasp it. We, Hibikino-Musashi@Home focus on this area.<\/p>\n\n\n\n

\n
\n
\"\"
Object recognition by YolactEdge<\/figcaption><\/figure>\n<\/div>\n\n\n\n
\n
\"\"
Object grasping point estimation<\/figcaption><\/figure>\n<\/div>\n<\/div>\n\n\n\n

Human Detection<\/h3>\n\n\n\n

Human attributions are recognized by using CSRA[2] and the pointing position is estimated by using LightWightHumanPose[3]. These functions are used to explain the attributes of humans and to bring a pointed object to a human.<\/p>\n\n\n\n

\n
\n
\"\"
Human attribute recognition<\/figcaption><\/figure>\n<\/div>\n\n\n\n
\n
\"\"
Pointing point estimation<\/figcaption><\/figure>\n<\/div>\n<\/div>\n\n\n\n

Voice Recognition<\/h3>\n\n\n\n

Noise is reduced by noisereduce[4] and the voice is processed by using Vosk[5]. Voice recognition is important to communicate the robot with humans. A robust recognition system that recognizes spoken words in a noisy crowd has been developed.<\/p>\n\n\n\n

\n
\n
\"\"
Voice recognition<\/figcaption><\/figure>\n<\/div>\n\n\n\n
\n
\"\"
Noise reduction<\/figcaption><\/figure>\n<\/div>\n<\/div>\n\n\n\n

Environment Recognition<\/h3>\n\n\n\n

The placable area and travelable area are estimated by semantic segmentation. These techniques are used to place an object in an empty space on a shelf and travel by avoiding obstacles on a floor.<\/p>\n\n\n\n

\n
\n
\"\"
Placable area estimation<\/figcaption><\/figure>\n<\/div>\n\n\n\n
\n
\"\"
Travelable area estimation<\/figcaption><\/figure>\n<\/div>\n<\/div>\n\n\n\n

Dataset Generation and Sim2Real[6]<\/h2>\n\n\n\n

A large amount of training data should be prepared to train the object recognition system. Hibikino-Musashi@Home develops dataset generation software using a 3D simulator to train the object recognition system.<\/p>\n\n\n\n

First, an object is scanned using a 3D scanner to get 3D data about the object. Then, the scanned 3D data is taken in the simulator to generate a dataset. The objects are placed randomly in a room simulated in a 3D environment. By changing the background image randomly, the recognition system with high accuracy can be trained.<\/p>\n\n\n\n

By using this method, 100 thousand datasets with mask and label automatically within one hour.<\/p>\n\n\n\n

\"\"\/
3D model scanning<\/figcaption><\/figure>\n\n\n\n

Open-source Simulator<\/h2>\n\n\n\n

Hibikino-Musashi@Home published the development system as an open source.<\/p>\n\n\n\n

https:\/\/github.com\/Hibikino-Musashi-Home\/hma_wrs_sim_ws<\/a><\/p>\n\n\n\n

The following link is the ROS workspace for the tidy-up robot developed by Hibikino-Musashi@Home. This workspace uses an OSS HSR simulator[7]. A developer who does not have a physical HSR can start development easily using libraries developed by Hibikino-Musashi@Home.<\/p>\n\n\n\n

\"This
Open-source HSR simulator<\/figcaption><\/figure>\n\n\n\n

Research Activities<\/h2>\n\n\n\n

Brain-inspired Artificial Intelligence[8]<\/h3>\n\n\n\n

Brain-inspired Artificial Intelligence, which adopts brain functions to obtain a family’s preference or behavior through the small amount of robot experience has been developed. It can acquire knowledge that differs in each home environment and is difficult to acquire by using deep learning. The goal is to implement the model into the digital or analog circuit to develop a highly efficient and low power consumption system for robots.<\/p>\n\n\n\n

\"This
Brain-inspired AI model<\/figcaption><\/figure>\n\n\n\n

Analog Chip and Soft Hand[9]<\/h3>\n\n\n\n


This object recognition system uses tactile information taken by sensors made of soft materials and process tactile information on an analog chip. The tactile sensor is made of silicone rubber with liquid metal. It can hold an object gently without damage. The circuit is implemented on an analog chip with 300 TOPS\/W of high performance and power efficiency, which is only a few millimeters in size.<\/p>\n\n\n\n

The combination of the dedicated robot sensor can improve recognition accuracy, and this approach can develop a long battery life robot system.<\/p>\n\n\n\n

\"\"
Analog chip and soft hand<\/figcaption><\/figure>\n\n\n\n

References<\/h2>\n\n\n\n

[1] H. Liu et al., ICRA, 2021.<\/a>
[2] K. Zhu et al., ICCV 2021.<\/a>
[4] vosk-api, https:\/\/github.com\/alphacep\/vosk-api.<\/a>
[5] T. Sainburg et al., PLoS computational biology, 2020.<\/a>
[6] T. Ono et al., Advanced Robotics, 2022.<\/a>
[7] hsrb_robocup_dspl_docker, https:\/\/github.com\/hsr-project\/hsrb_robocup_dspl_docker<\/a>
[8] H. Nakagawa et al., IEEE Access, 2022.<\/a>
[9] M.Yamaguchi et al., IJCNN 2019.<\/a><\/p>\n\n\n\n

\"\"<\/a><\/figure>\n","protected":false},"excerpt":{"rendered":"

Basic Functions The basic functions of our robot are explained. These techniques are combined to achieve a complex robot’s action. Object Recognition YolactEdge[1] is used for object recognition and the result is used for object grasping point estimation. This is an important process for the robot to find an object and grasp it. We, Hibikino-Musashi@Home […]<\/p>\n","protected":false},"author":4,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_uag_custom_page_level_css":"","site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"class_list":["post-1130","page","type-page","status-publish","hentry"],"yoast_head":"\nOur Techniques | HMA<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/techniques_en\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Our Techniques | HMA\" \/>\n<meta property=\"og:description\" content=\"Basic Functions The basic functions of our robot are explained. These techniques are combined to achieve a complex robot’s action. Object Recognition YolactEdge[1] is used for object recognition and the result is used for object grasping point estimation. This is an important process for the robot to find an object and grasp it. We, Hibikino-Musashi@Home […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/techniques_en\/\" \/>\n<meta property=\"og:site_name\" content=\"Hibikino-Musashi@Home\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/HibikinoMusashiAthome\/\" \/>\n<meta property=\"article:modified_time\" content=\"2022-11-12T17:08:21+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wordpress\/wp-content\/uploads\/2022\/11\/Picture1.jpg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@HMA_wakamatsu\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.brain.kyutech.ac.jp\\\/~hma\\\/techniques_en\\\/\",\"url\":\"https:\\\/\\\/www.brain.kyutech.ac.jp\\\/~hma\\\/techniques_en\\\/\",\"name\":\"Our Techniques | HMA\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.brain.kyutech.ac.jp\\\/~hma\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.brain.kyutech.ac.jp\\\/~hma\\\/techniques_en\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.brain.kyutech.ac.jp\\\/~hma\\\/techniques_en\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.brain.kyutech.ac.jp\\\/~hma\\\/wordpress\\\/wp-content\\\/uploads\\\/2022\\\/11\\\/Picture1.jpg\",\"datePublished\":\"2022-11-04T17:01:02+00:00\",\"dateModified\":\"2022-11-12T17:08:21+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.brain.kyutech.ac.jp\\\/~hma\\\/techniques_en\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.brain.kyutech.ac.jp\\\/~hma\\\/techniques_en\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.brain.kyutech.ac.jp\\\/~hma\\\/techniques_en\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.brain.kyutech.ac.jp\\\/~hma\\\/wordpress\\\/wp-content\\\/uploads\\\/2022\\\/11\\\/Picture1.jpg\",\"contentUrl\":\"https:\\\/\\\/www.brain.kyutech.ac.jp\\\/~hma\\\/wordpress\\\/wp-content\\\/uploads\\\/2022\\\/11\\\/Picture1.jpg\",\"width\":399,\"height\":300},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.brain.kyutech.ac.jp\\\/~hma\\\/techniques_en\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.brain.kyutech.ac.jp\\\/~hma\\\/ja\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Our Techniques\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.brain.kyutech.ac.jp\\\/~hma\\\/#website\",\"url\":\"https:\\\/\\\/www.brain.kyutech.ac.jp\\\/~hma\\\/\",\"name\":\"Hibikino-Musashi@Home\",\"description\":\"Hibikino-Musashi@Home (HMA)\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.brain.kyutech.ac.jp\\\/~hma\\\/#organization\"},\"alternateName\":\"HMA\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.brain.kyutech.ac.jp\\\/~hma\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.brain.kyutech.ac.jp\\\/~hma\\\/#organization\",\"name\":\"Hibikino-Musashi@Home\",\"alternateName\":\"HMA\",\"url\":\"https:\\\/\\\/www.brain.kyutech.ac.jp\\\/~hma\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.brain.kyutech.ac.jp\\\/~hma\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.brain.kyutech.ac.jp\\\/~hma\\\/wordpress\\\/wp-content\\\/uploads\\\/2021\\\/07\\\/cropped-HMA_Logo_2_512.png\",\"contentUrl\":\"https:\\\/\\\/www.brain.kyutech.ac.jp\\\/~hma\\\/wordpress\\\/wp-content\\\/uploads\\\/2021\\\/07\\\/cropped-HMA_Logo_2_512.png\",\"width\":512,\"height\":512,\"caption\":\"Hibikino-Musashi@Home\"},\"image\":{\"@id\":\"https:\\\/\\\/www.brain.kyutech.ac.jp\\\/~hma\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/HibikinoMusashiAthome\\\/\",\"https:\\\/\\\/x.com\\\/HMA_wakamatsu\",\"https:\\\/\\\/www.instagram.com\\\/hma_wakamatsu\\\/\",\"https:\\\/\\\/www.youtube.com\\\/@hma_wakamatsu\",\"https:\\\/\\\/github.com\\\/Hibikino-Musashi-Home\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Our Techniques | HMA","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/techniques_en\/","og_locale":"en_US","og_type":"article","og_title":"Our Techniques | HMA","og_description":"Basic Functions The basic functions of our robot are explained. These techniques are combined to achieve a complex robot’s action. Object Recognition YolactEdge[1] is used for object recognition and the result is used for object grasping point estimation. This is an important process for the robot to find an object and grasp it. We, Hibikino-Musashi@Home […]","og_url":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/techniques_en\/","og_site_name":"Hibikino-Musashi@Home","article_publisher":"https:\/\/www.facebook.com\/HibikinoMusashiAthome\/","article_modified_time":"2022-11-12T17:08:21+00:00","og_image":[{"url":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wordpress\/wp-content\/uploads\/2022\/11\/Picture1.jpg","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_site":"@HMA_wakamatsu","schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/techniques_en\/","url":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/techniques_en\/","name":"Our Techniques | HMA","isPartOf":{"@id":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/techniques_en\/#primaryimage"},"image":{"@id":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/techniques_en\/#primaryimage"},"thumbnailUrl":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wordpress\/wp-content\/uploads\/2022\/11\/Picture1.jpg","datePublished":"2022-11-04T17:01:02+00:00","dateModified":"2022-11-12T17:08:21+00:00","breadcrumb":{"@id":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/techniques_en\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.brain.kyutech.ac.jp\/~hma\/techniques_en\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/techniques_en\/#primaryimage","url":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wordpress\/wp-content\/uploads\/2022\/11\/Picture1.jpg","contentUrl":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wordpress\/wp-content\/uploads\/2022\/11\/Picture1.jpg","width":399,"height":300},{"@type":"BreadcrumbList","@id":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/techniques_en\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/ja\/"},{"@type":"ListItem","position":2,"name":"Our Techniques"}]},{"@type":"WebSite","@id":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/#website","url":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/","name":"Hibikino-Musashi@Home","description":"Hibikino-Musashi@Home (HMA)","publisher":{"@id":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/#organization"},"alternateName":"HMA","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/#organization","name":"Hibikino-Musashi@Home","alternateName":"HMA","url":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/#\/schema\/logo\/image\/","url":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wordpress\/wp-content\/uploads\/2021\/07\/cropped-HMA_Logo_2_512.png","contentUrl":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wordpress\/wp-content\/uploads\/2021\/07\/cropped-HMA_Logo_2_512.png","width":512,"height":512,"caption":"Hibikino-Musashi@Home"},"image":{"@id":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/HibikinoMusashiAthome\/","https:\/\/x.com\/HMA_wakamatsu","https:\/\/www.instagram.com\/hma_wakamatsu\/","https:\/\/www.youtube.com\/@hma_wakamatsu","https:\/\/github.com\/Hibikino-Musashi-Home"]}]}},"uagb_featured_image_src":{"full":false,"thumbnail":false,"medium":false,"medium_large":false,"large":false,"1536x1536":false,"2048x2048":false},"uagb_author_info":{"display_name":"HMA","author_link":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/author\/hma_editor\/"},"uagb_comment_info":0,"uagb_excerpt":"Basic Functions The basic functions of our robot are explained. These techniques are combined to achieve a complex robot’s action. Object Recognition YolactEdge[1] is used for object recognition and the result is used for object grasping point estimation. This is an important process for the robot to find an object and grasp it. We, Hibikino-Musashi@Home…","_links":{"self":[{"href":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wp-json\/wp\/v2\/pages\/1130","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wp-json\/wp\/v2\/comments?post=1130"}],"version-history":[{"count":1,"href":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wp-json\/wp\/v2\/pages\/1130\/revisions"}],"predecessor-version":[{"id":2157,"href":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wp-json\/wp\/v2\/pages\/1130\/revisions\/2157"}],"wp:attachment":[{"href":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wp-json\/wp\/v2\/media?parent=1130"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}