{"id":24339,"date":"2018-06-06T16:14:16","date_gmt":"2018-06-06T20:14:16","guid":{"rendered":"https:\/\/www.tun.com\/blog\/?p=24339"},"modified":"2022-03-16T11:01:35","modified_gmt":"2022-03-16T15:01:35","slug":"chores-robot-agents","status":"publish","type":"post","link":"https:\/\/www.tun.com\/blog\/chores-robot-agents\/","title":{"rendered":"No More Chores: Robot \u2018Agents\u2019 Will Do Them All"},"content":{"rendered":"<p><span style=\"font-weight: 400;\">Have you ever fantasized about having a robot sidekick? How about a machine for all your daily, time-consuming household chores, so you can sit back and relax? That could well be in our future if a team of researchers headed by MIT and the University of Toronto has anything to do with it.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The researchers have developed a <\/span><a href=\"https:\/\/www.csail.mit.edu\/news\/teaching-chores-artificial-agent\"><span style=\"font-weight: 400;\">virtual system<\/span><\/a><span style=\"font-weight: 400;\"> that can teach robots to perform detailed household chores, potentially turning that fantasy into reality. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">The full paper is available <\/span><a href=\"http:\/\/virtual-home.org\/paper\/virtualhome.pdf\"><span style=\"font-weight: 400;\">here<\/span><\/a><span style=\"font-weight: 400;\">. <\/span><\/p>\n<h2><b>The system <\/b><\/h2>\n<p><span style=\"font-weight: 400;\">Inspired by the Sims video game, the system, called <\/span><a href=\"http:\/\/virtual-home.org\/\"><span style=\"font-weight: 400;\">VirtualHome<\/span><\/a><span style=\"font-weight: 400;\">, is a 3D house environment capable of generating videos and simulating different household tasks that can be performed by a virtual \u201cagent.\u201d<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The researchers used nearly 3,000 programs of various tasks to train the system. These tasks are broken down into specific subtasks for the virtual agents to understand. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">For example, a task like \u201cmaking coffee\u201d would also include the step \u201cgrabbing a cup.\u201d <\/span><\/p>\n<p><span style=\"font-weight: 400;\">\u201cDescribing actions as computer programs gives clear, unambiguous descriptions of all the steps needed to complete a task,\u201d said <\/span><a href=\"http:\/\/people.csail.mit.edu\/xavierpuig\/\"><span style=\"font-weight: 400;\">Xavier Puig<\/span><\/a><span style=\"font-weight: 400;\">, a doctoral student in the C<\/span><a href=\"https:\/\/www.csail.mit.edu\/\"><span style=\"font-weight: 400;\">omputer Science and Artificial Intelligence Laboratory (CSAIL)<\/span><\/a><span style=\"font-weight: 400;\"> at MIT. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">\u201cWe use these programs to instruct virtual characters to execute tasks in a simulated household environment and show how these tasks can be reconstructed from either written descriptions or even video demonstrations.\u201d<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The virtual agent can complete 1,000 of these actions throughout eight different scenes, including a living room, kitchen, dining room, bedroom, and home office. <\/span><\/p>\n<p><iframe title=\"VirtualHome: Teaching Chores to Robots\" width=\"500\" height=\"375\" src=\"https:\/\/www.youtube.com\/embed\/qAMJvKpyPP4?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<h2><b>Why virtual?<\/b><\/h2>\n<p><span style=\"font-weight: 400;\">In order to complete tasks, robots need explicit, clearly defined instructions &#8212; they can\u2019t just infer and reason like humans. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">For instance, if you tell a human to \u201cswitch on the TV and watch it from the sofa,\u201d they can infer that they have to grab the remote control, and then physically sit or lie on the sofa. A robot however, would need those latter two instructions. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">So, the researchers developed VirtualHome as a way to compile a large database of household tasks described with the detailed language that robots require. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">To accomplish this, the team first collected verbal descriptions of household tasks and translated them into simple code. For instance, the directions for the previous example would be coded as: walk to the TV, switch on the TV, walk to the sofa, sit on the sofa, and watch television. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">Once these programs were developed, the team put them through the VirtualHome 3D simulator and translated them into videos. The virtual agent would then watch and execute the tasks described by the programs. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">\u201cOur model successfully demonstrated that the agents could learn to reconstruct a program, and therefore perform a task, given either a description: \u2018pour milk into glass\u2019 or a video demonstration of the activity,\u201d said Puig. <\/span><\/p>\n<h2><b>Next step: Teaching robots <\/b><\/h2>\n<p><span style=\"font-weight: 400;\">Since VirtualHome includes a large database of detailed instructions and videos that can teach a virtual agent, the team now hopes to use this information to teach robots how to perform the tasks.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">\u201cEventually, we hope that a robot would watch these videos, and by using a positive-feedback learning system, could recreate the task in a person\u2019s home,\u201d said Puig. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">Furthermore, the team wants to someday train robots using actual videos instead of the simulations. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">\u201cOne next step is to develop VirtualHome so that it can train robots with actual live-action videos, rather than the animated simulation videos that we used,\u201d Puig said. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">\u201cThis would allow the system to teach robots to do things just by having them watch a video somebody posted on YouTube.\u201d<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Ultimately, the researchers believe, VirtualHome will someday teach robots to assist us inside our home.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">\u201cYou could imagine a robot being able to anticipate what will happen in a home and acting as an assistive technology by anticipating wants and needs, especially for the elderly,\u201d said Puig. <\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Have you ever fantasized about having a robot sidekick? How about a machine for all your daily, time-consuming household chores, so you can sit back and relax? That could well be in our future if a team of researchers headed by MIT and the University of Toronto has anything to do with it. The researchers [&hellip;]<\/p>\n","protected":false},"author":58,"featured_media":24346,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_uag_custom_page_level_css":"","_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[232,376,632,230,229],"tags":[],"class_list":["post-24339","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology","category-massachusetts-institute-of-technology","category-robotics","category-news","category-lead-stories"],"aioseo_notices":[],"uagb_featured_image_src":{"full":["https:\/\/www.tun.com\/blog\/wp-content\/uploads\/2018\/06\/robotic-assistant-mit-csail-00_0-1.png",830,533,false],"thumbnail":["https:\/\/www.tun.com\/blog\/wp-content\/uploads\/2018\/06\/robotic-assistant-mit-csail-00_0-1-224x144.png",224,144,true],"medium":["https:\/\/www.tun.com\/blog\/wp-content\/uploads\/2018\/06\/robotic-assistant-mit-csail-00_0-1-300x193.png",300,193,true],"medium_large":["https:\/\/www.tun.com\/blog\/wp-content\/uploads\/2018\/06\/robotic-assistant-mit-csail-00_0-1.png",830,533,false],"large":["https:\/\/www.tun.com\/blog\/wp-content\/uploads\/2018\/06\/robotic-assistant-mit-csail-00_0-1.png",830,533,false],"1536x1536":["https:\/\/www.tun.com\/blog\/wp-content\/uploads\/2018\/06\/robotic-assistant-mit-csail-00_0-1.png",830,533,false],"2048x2048":["https:\/\/www.tun.com\/blog\/wp-content\/uploads\/2018\/06\/robotic-assistant-mit-csail-00_0-1.png",830,533,false]},"uagb_author_info":{"display_name":"Natalie Colarossi","author_link":"https:\/\/www.tun.com\/blog\/author\/natalie-colarossi\/"},"uagb_comment_info":0,"uagb_excerpt":"Have you ever fantasized about having a robot sidekick? How about a machine for all your daily, time-consuming household chores, so you can sit back and relax? That could well be in our future if a team of researchers headed by MIT and the University of Toronto has anything to do with it. The researchers&hellip;","featured_media_src_url":"https:\/\/www.tun.com\/blog\/wp-content\/uploads\/2018\/06\/robotic-assistant-mit-csail-00_0-1.png","_links":{"self":[{"href":"https:\/\/www.tun.com\/blog\/wp-json\/wp\/v2\/posts\/24339","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.tun.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.tun.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.tun.com\/blog\/wp-json\/wp\/v2\/users\/58"}],"replies":[{"embeddable":true,"href":"https:\/\/www.tun.com\/blog\/wp-json\/wp\/v2\/comments?post=24339"}],"version-history":[{"count":0,"href":"https:\/\/www.tun.com\/blog\/wp-json\/wp\/v2\/posts\/24339\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.tun.com\/blog\/wp-json\/wp\/v2\/media\/24346"}],"wp:attachment":[{"href":"https:\/\/www.tun.com\/blog\/wp-json\/wp\/v2\/media?parent=24339"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.tun.com\/blog\/wp-json\/wp\/v2\/categories?post=24339"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.tun.com\/blog\/wp-json\/wp\/v2\/tags?post=24339"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}