{"id":2158,"date":"2024-12-06T23:25:33","date_gmt":"2024-12-06T14:25:33","guid":{"rendered":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/?page_id=2158"},"modified":"2024-12-06T23:27:31","modified_gmt":"2024-12-06T14:27:31","slug":"tech2024_en","status":"publish","type":"page","link":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/tech2024_en\/","title":{"rendered":"Technologies (2024)"},"content":{"rendered":"\n

Basic Functions<\/h2>\n\n\n\n

The basic functions of our robot are explained. These techniques are combined to achieve a complex robot’s action.<\/p>\n\n\n\n

Vision<\/h3>\n\n\n\n

We use YOLOv8 and Grounding DINO + Nano SAM methods depending on the situation.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Navigation<\/h3>\n\n\n\n

We modify the navigation system by Pumas.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Tidy-up Simulator<\/h3>\n\n\n\n

The tidy-up task on the HSR simulator is available on GitHub.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

https:\/\/github.com\/Hibikino-Musashi-Home\/hma_wrs_sim_ws<\/a><\/p>\n\n\n\n

Research Activities<\/h2>\n\n\n\n

Dataset Generation<\/h3>\n\n\n\n

A dataset generator is a powerful tool for creating training datasets for the YOLO system.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Reservoir Computing<\/h3>\n\n\n\n

Reservoir computing is the lightweight model to handle the time-series information. We conduct research on this method for sound recognition and hand-wave detection systems.<\/p>\n\n\n\n

\n
\n
\"\"<\/figure>\n<\/div>\n\n\n\n
\n
\"\"<\/figure>\n<\/div>\n<\/div>\n\n\n\n

<\/p>\n\n\n\n

Towards Realization of General-Purpose Service Robots<\/h3>\n\n\n\n

We conduct research on brain-inspired systems for home service robots. The hippocampus is the brain part that handles episodic memories. The system handles episodic memories acquired by the experience of the robot.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Our GPSR system is powered by large language model. We also implement the function to understand the spatial relationship of the objects. <\/p>\n\n\n\n

\n
\n
\"\"<\/figure>\n<\/div>\n\n\n\n
\n
\"\"<\/figure>\n<\/div>\n<\/div>\n\n\n\n

References<\/h2>\n\n\n\n

[1] G. Jocher et al., https:\/\/github.com\/ultralytics\/ultralytics, 2023.
[2] S. Liu et al, arXiv:2303.05499<\/em>, 2023.
[3] M. Negrete et al., SPIIRAS Proceedings<\/em>, 2018.
[4] C. Lugaresiet al., arXiv:1906.08172<\/em>, 2019.
[5] H. Jaeger et al., Res. Center Inf. Technol., GMD Rep<\/em>, 2001.
[6] A. Radford et al., arXiv:2212.04356<\/em>, 2022.
[7] Open AI, https:\/\/openai.com\/index\/hello-gpt-4o, 2024.
[8] Open AI, https:\/\/openai.com\/index\/gpt-4v-system-card\/, 2024<\/p>\n\n\n\n

Publications<\/h2>\n\n\n\n

[9]\u00a0\u00a0 T. Ono et al., Advanced Robotics<\/em>, 2022.
[10] K. Yoshioka et al., FPT’23<\/em>, 2023.
[11] S. Matsumoto et al., ISCAS 2024.
[12] H. Yamaguchi et al., ICAROB<\/em>, 2024.
[13] Y. Shishido et al., LSI and System Workshop<\/em>, 2023. [Poster Award<\/strong>]
[14] A. Mizutani et al., IJCNN<\/em>, 2024.
[15] Y. Shishido et al, IJCNN<\/em>, 2024.
[16] K. Yamao et al., ICRA Late Breaking Results Poster<\/em>, 2024.\u00a0<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

TDP2024: https:\/\/arxiv.org\/abs\/2410.06192<\/a><\/p>\n\n\n\n

<\/p>\n","protected":false},"excerpt":{"rendered":"

Basic Functions The basic functions of our robot are explained. These techniques are combined to achieve a complex robot’s action. Vision We use YOLOv8 and Grounding DINO + Nano SAM methods depending on the situation. Navigation We modify the navigation system by Pumas. Tidy-up Simulator The tidy-up task on the HSR simulator is available on […]<\/p>\n","protected":false},"author":4,"featured_media":2159,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_uag_custom_page_level_css":"","site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"set","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"class_list":["post-2158","page","type-page","status-publish","has-post-thumbnail","hentry"],"yoast_head":"\nTechnologies (2024) | HMA<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/tech2024_en\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Technologies (2024) | HMA\" \/>\n<meta property=\"og:description\" content=\"Basic Functions The basic functions of our robot are explained. These techniques are combined to achieve a complex robot’s action. Vision We use YOLOv8 and Grounding DINO + Nano SAM methods depending on the situation. Navigation We modify the navigation system by Pumas. Tidy-up Simulator The tidy-up task on the HSR simulator is available on […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/tech2024_en\/\" \/>\n<meta property=\"og:site_name\" content=\"Hibikino-Musashi@Home\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/HibikinoMusashiAthome\/\" \/>\n<meta property=\"article:modified_time\" content=\"2024-12-06T14:27:31+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wordpress\/wp-content\/uploads\/2024\/12\/rc24_poster_am4.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1080\" \/>\n\t<meta property=\"og:image:height\" content=\"1527\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@HMA_wakamatsu\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/tech2024_en\/\",\"url\":\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/tech2024_en\/\",\"name\":\"Technologies (2024) | HMA\",\"isPartOf\":{\"@id\":\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/tech2024_en\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/tech2024_en\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wordpress\/wp-content\/uploads\/2024\/12\/rc24_poster_am4.jpg\",\"datePublished\":\"2024-12-06T14:25:33+00:00\",\"dateModified\":\"2024-12-06T14:27:31+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/tech2024_en\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/tech2024_en\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/tech2024_en\/#primaryimage\",\"url\":\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wordpress\/wp-content\/uploads\/2024\/12\/rc24_poster_am4.jpg\",\"contentUrl\":\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wordpress\/wp-content\/uploads\/2024\/12\/rc24_poster_am4.jpg\",\"width\":1080,\"height\":1527},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/tech2024_en\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/ja\/hibikino-musashihome\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Technologies (2024)\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/#website\",\"url\":\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/\",\"name\":\"Hibikino-Musashi@Home\",\"description\":\"Hibikino-Musashi@Home (HMA)\",\"publisher\":{\"@id\":\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/#organization\"},\"alternateName\":\"HMA\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/#organization\",\"name\":\"Hibikino-Musashi@Home\",\"alternateName\":\"HMA\",\"url\":\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wordpress\/wp-content\/uploads\/2021\/07\/cropped-HMA_Logo_2_512.png\",\"contentUrl\":\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wordpress\/wp-content\/uploads\/2021\/07\/cropped-HMA_Logo_2_512.png\",\"width\":512,\"height\":512,\"caption\":\"Hibikino-Musashi@Home\"},\"image\":{\"@id\":\"https:\/\/www.brain.kyutech.ac.jp\/~hma\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/HibikinoMusashiAthome\/\",\"https:\/\/x.com\/HMA_wakamatsu\",\"https:\/\/www.instagram.com\/hma_wakamatsu\/\",\"https:\/\/www.youtube.com\/@hma_wakamatsu\",\"https:\/\/github.com\/Hibikino-Musashi-Home\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Technologies (2024) | HMA","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/tech2024_en\/","og_locale":"en_US","og_type":"article","og_title":"Technologies (2024) | HMA","og_description":"Basic Functions The basic functions of our robot are explained. These techniques are combined to achieve a complex robot’s action. Vision We use YOLOv8 and Grounding DINO + Nano SAM methods depending on the situation. Navigation We modify the navigation system by Pumas. Tidy-up Simulator The tidy-up task on the HSR simulator is available on […]","og_url":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/tech2024_en\/","og_site_name":"Hibikino-Musashi@Home","article_publisher":"https:\/\/www.facebook.com\/HibikinoMusashiAthome\/","article_modified_time":"2024-12-06T14:27:31+00:00","og_image":[{"width":1080,"height":1527,"url":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wordpress\/wp-content\/uploads\/2024\/12\/rc24_poster_am4.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_site":"@HMA_wakamatsu","schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/tech2024_en\/","url":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/tech2024_en\/","name":"Technologies (2024) | HMA","isPartOf":{"@id":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/tech2024_en\/#primaryimage"},"image":{"@id":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/tech2024_en\/#primaryimage"},"thumbnailUrl":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wordpress\/wp-content\/uploads\/2024\/12\/rc24_poster_am4.jpg","datePublished":"2024-12-06T14:25:33+00:00","dateModified":"2024-12-06T14:27:31+00:00","breadcrumb":{"@id":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/tech2024_en\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.brain.kyutech.ac.jp\/~hma\/tech2024_en\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/tech2024_en\/#primaryimage","url":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wordpress\/wp-content\/uploads\/2024\/12\/rc24_poster_am4.jpg","contentUrl":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wordpress\/wp-content\/uploads\/2024\/12\/rc24_poster_am4.jpg","width":1080,"height":1527},{"@type":"BreadcrumbList","@id":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/tech2024_en\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/ja\/hibikino-musashihome\/"},{"@type":"ListItem","position":2,"name":"Technologies (2024)"}]},{"@type":"WebSite","@id":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/#website","url":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/","name":"Hibikino-Musashi@Home","description":"Hibikino-Musashi@Home (HMA)","publisher":{"@id":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/#organization"},"alternateName":"HMA","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/#organization","name":"Hibikino-Musashi@Home","alternateName":"HMA","url":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/#\/schema\/logo\/image\/","url":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wordpress\/wp-content\/uploads\/2021\/07\/cropped-HMA_Logo_2_512.png","contentUrl":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wordpress\/wp-content\/uploads\/2021\/07\/cropped-HMA_Logo_2_512.png","width":512,"height":512,"caption":"Hibikino-Musashi@Home"},"image":{"@id":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/HibikinoMusashiAthome\/","https:\/\/x.com\/HMA_wakamatsu","https:\/\/www.instagram.com\/hma_wakamatsu\/","https:\/\/www.youtube.com\/@hma_wakamatsu","https:\/\/github.com\/Hibikino-Musashi-Home"]}]}},"uagb_featured_image_src":{"full":["https:\/\/www.brain.kyutech.ac.jp\/~hma\/wordpress\/wp-content\/uploads\/2024\/12\/rc24_poster_am4.jpg",1080,1527,false],"thumbnail":["https:\/\/www.brain.kyutech.ac.jp\/~hma\/wordpress\/wp-content\/uploads\/2024\/12\/rc24_poster_am4-150x150.jpg",150,150,true],"medium":["https:\/\/www.brain.kyutech.ac.jp\/~hma\/wordpress\/wp-content\/uploads\/2024\/12\/rc24_poster_am4-212x300.jpg",212,300,true],"medium_large":["https:\/\/www.brain.kyutech.ac.jp\/~hma\/wordpress\/wp-content\/uploads\/2024\/12\/rc24_poster_am4-768x1086.jpg",768,1086,true],"large":["https:\/\/www.brain.kyutech.ac.jp\/~hma\/wordpress\/wp-content\/uploads\/2024\/12\/rc24_poster_am4-724x1024.jpg",724,1024,true],"1536x1536":["https:\/\/www.brain.kyutech.ac.jp\/~hma\/wordpress\/wp-content\/uploads\/2024\/12\/rc24_poster_am4.jpg",1080,1527,false],"2048x2048":["https:\/\/www.brain.kyutech.ac.jp\/~hma\/wordpress\/wp-content\/uploads\/2024\/12\/rc24_poster_am4.jpg",1080,1527,false]},"uagb_author_info":{"display_name":"HMA","author_link":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/author\/hma_editor\/"},"uagb_comment_info":0,"uagb_excerpt":"Basic Functions The basic functions of our robot are explained. These techniques are combined to achieve a complex robot’s action. Vision We use YOLOv8 and Grounding DINO + Nano SAM methods depending on the situation. Navigation We modify the navigation system by Pumas. Tidy-up Simulator The tidy-up task on the HSR simulator is available on…","_links":{"self":[{"href":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wp-json\/wp\/v2\/pages\/2158","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wp-json\/wp\/v2\/comments?post=2158"}],"version-history":[{"count":3,"href":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wp-json\/wp\/v2\/pages\/2158\/revisions"}],"predecessor-version":[{"id":2177,"href":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wp-json\/wp\/v2\/pages\/2158\/revisions\/2177"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wp-json\/wp\/v2\/media\/2159"}],"wp:attachment":[{"href":"https:\/\/www.brain.kyutech.ac.jp\/~hma\/wp-json\/wp\/v2\/media?parent=2158"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}