{"id":24603,"date":"2018-06-19T16:17:31","date_gmt":"2018-06-19T20:17:31","guid":{"rendered":"https:\/\/www.tun.com\/blog\/?p=24603"},"modified":"2022-03-16T10:55:19","modified_gmt":"2022-03-16T14:55:19","slug":"computer-program-can-see-5-minutes-into-the-future","status":"publish","type":"post","link":"https:\/\/www.tun.com\/blog\/computer-program-can-see-5-minutes-into-the-future\/","title":{"rendered":"Computer Program Can See 5 Minutes Into the Future"},"content":{"rendered":"<p><span style=\"font-weight: 400;\">Computer scientists from the University of Bonn in Germany have developed a <\/span><a href=\"https:\/\/www.uni-bonn.de\/news\/154-2018\"><span style=\"font-weight: 400;\">self-learning computer program<\/span><\/a><span style=\"font-weight: 400;\"> that can look several minutes into the future. \u00a0\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">This development will enable robots to anticipate the actions of humans, allowing machines to work side by side with people. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">\u201cIf the action a robot should do depends on the action of a human, the robot has to anticipate the future actions of the human,\u201d said <\/span><a href=\"http:\/\/pages.iai.uni-bonn.de\/gall_juergen\/\"><span style=\"font-weight: 400;\">J\u00fcrgen Gall<\/span><\/a><span style=\"font-weight: 400;\">, a professor in the Institute of Computer Science at Bonn and lead researcher of the study. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">Using this software, robots can provide humans with necessary parts, tools or ingredients at the right time. \u00a0\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">\u201cNobody wants to wait for a robot,\u201d said Gall.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Gall and his team will present their paper at the 2018 annual <\/span><a href=\"http:\/\/cvpr2018.thecvf.com\/program\/main_conference\"><span style=\"font-weight: 400;\">Conference on Computer Vision and Pattern Recognition<\/span><\/a><span style=\"font-weight: 400;\">, June 19-21, in Salt Lake City.<\/span><span style=\"font-weight: 400;\"><br \/>\n<\/span><\/p>\n<h2><b>Working with humans<\/b><\/h2>\n<p><span style=\"font-weight: 400;\">The ultimate goal of this ongoing study is to teach robots to predict the timing and span of activities, minutes or hours before they happen. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">This development could easily apply to robots that clean and work in the kitchen. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">A kitchen robot could pass ingredients to a cook as soon as they are needed, preheat the oven, and let a cook know if he or she missed a step.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">A vacuum-cleaning robot could know to avoid the kitchen around dinner time and move into a less busy room. \u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">\u201cRobots have to anticipate the future actions of the humans, otherwise they will not able to do the right action at the right time, and human-robot collaborations remain frustrating for the humans,\u201d said Gall. <\/span><\/p>\n<h2><b>Training the computer program<\/b><\/h2>\n<p><span style=\"font-weight: 400;\">To train the program, the researchers showed 40 six-minute-long videos of people preparing different salads, which the program \u201cwatched\u201d for four hours. <\/span><\/p>\n<p><iframe title=\"When will you do what? - Anticipating Temporal Occurrences of Activities (CVPR 2018)\" width=\"500\" height=\"375\" src=\"https:\/\/www.youtube.com\/embed\/xMNYRcVH_oI?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"font-weight: 400;\">In each video, there were about 20 different actions with precise details about the starting time and duration of every action.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">From observing the videos, the program learned the order in which different chefs add their ingredients. The program can then use this knowledge to predict future actions in new, similar situations.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">\u201cIf the network is trained, it anticipates the future actions in previously unseen videos,\u201d said Gall. <\/span><\/p>\n<h2><b>Testing the program<\/b><\/h2>\n<p><span style=\"font-weight: 400;\">To test the effectiveness of the learning process, the researchers showed the program videos it had never seen before.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The researchers informed the program what happened in the first 20 or 30 percent of one of the new videos and asked it to predict what would happen in the rest of the video.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">&#8220;Accuracy was over 40 percent for short forecast periods, but then dropped the more the algorithm had to look into the future,&#8221; Gall said in a statement. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">When the program was asked to predict activities that occurred more than three minutes in the future, the accuracy rate dropped to 15 percent. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">However, the researchers did not let the program off easily. They only considered it correct if it predicted both the activity and timing. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">While this success rate may not yet seem impressive, it serves as the basis for technology that could soon have greater intuition than humans. <\/span><\/p>\n<h2><b>What\u2019s next?<\/b><\/h2>\n<p><span style=\"font-weight: 400;\">The researchers want to make it clear that this study is only the first step into a new field. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">They are working on improving the program\u2019s ability to predict what happens in the first quarter of a video, because it performed much worse if the researchers didn\u2019t give it a hint. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">They also wish to limit the annotations used in each explanatory video. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">\u201cWe want that the approach directly learns from videos without additional annotations,\u201d said Gall. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">\u201cWhile we have already achieved a major progress in this direction, there is still a performance gap compared to a network that is trained on videos where the start and endpoint of each action are annotated.\u201d<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Additionally, the researchers recognize that the future is not entirely deterministic and holds multiple potential outcomes. \u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">In an effort to make the machine as knowledgeable as possible, the researchers are working to expand their variety of training videos. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">Altogether, there is still a lot of ground to cover, but this research serves as a stepping stone for developing a new type of robot that can work alongside humans without having to be told what to do. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">Maybe someday soon a robot helper could wake you up with eggs and bacon on Saturday morning, without you having to program it to.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Computer scientists from the University of Bonn in Germany have developed a self-learning computer program that can look several minutes into the future. \u00a0\u00a0 This development will enable robots to anticipate the actions of humans, allowing machines to work side by side with people. \u201cIf the action a robot should do depends on the action [&hellip;]<\/p>\n","protected":false},"author":32,"featured_media":24611,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_uag_custom_page_level_css":"","_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[626,232,230,229],"tags":[],"class_list":["post-24603","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-artificial-intelligence","category-technology","category-news","category-lead-stories"],"aioseo_notices":[],"uagb_featured_image_src":{"full":["https:\/\/www.tun.com\/blog\/wp-content\/uploads\/2018\/06\/Bonn-Future.jpg",830,533,false],"thumbnail":["https:\/\/www.tun.com\/blog\/wp-content\/uploads\/2018\/06\/Bonn-Future-224x144.jpg",224,144,true],"medium":["https:\/\/www.tun.com\/blog\/wp-content\/uploads\/2018\/06\/Bonn-Future-300x193.jpg",300,193,true],"medium_large":["https:\/\/www.tun.com\/blog\/wp-content\/uploads\/2018\/06\/Bonn-Future.jpg",830,533,false],"large":["https:\/\/www.tun.com\/blog\/wp-content\/uploads\/2018\/06\/Bonn-Future.jpg",830,533,false],"1536x1536":["https:\/\/www.tun.com\/blog\/wp-content\/uploads\/2018\/06\/Bonn-Future.jpg",830,533,false],"2048x2048":["https:\/\/www.tun.com\/blog\/wp-content\/uploads\/2018\/06\/Bonn-Future.jpg",830,533,false]},"uagb_author_info":{"display_name":"Jackson Schroeder","author_link":"https:\/\/www.tun.com\/blog\/author\/jackson-schroeder\/"},"uagb_comment_info":0,"uagb_excerpt":"Computer scientists from the University of Bonn in Germany have developed a self-learning computer program that can look several minutes into the future. \u00a0\u00a0 This development will enable robots to anticipate the actions of humans, allowing machines to work side by side with people. \u201cIf the action a robot should do depends on the action&hellip;","featured_media_src_url":"https:\/\/www.tun.com\/blog\/wp-content\/uploads\/2018\/06\/Bonn-Future.jpg","_links":{"self":[{"href":"https:\/\/www.tun.com\/blog\/wp-json\/wp\/v2\/posts\/24603","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.tun.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.tun.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.tun.com\/blog\/wp-json\/wp\/v2\/users\/32"}],"replies":[{"embeddable":true,"href":"https:\/\/www.tun.com\/blog\/wp-json\/wp\/v2\/comments?post=24603"}],"version-history":[{"count":0,"href":"https:\/\/www.tun.com\/blog\/wp-json\/wp\/v2\/posts\/24603\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.tun.com\/blog\/wp-json\/wp\/v2\/media\/24611"}],"wp:attachment":[{"href":"https:\/\/www.tun.com\/blog\/wp-json\/wp\/v2\/media?parent=24603"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.tun.com\/blog\/wp-json\/wp\/v2\/categories?post=24603"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.tun.com\/blog\/wp-json\/wp\/v2\/tags?post=24603"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}