{"id":26870,"date":"2025-07-08T19:30:53","date_gmt":"2025-07-08T19:30:53","guid":{"rendered":"https:\/\/www.tun.com\/home\/?p=26870"},"modified":"2025-07-08T19:30:54","modified_gmt":"2025-07-08T19:30:54","slug":"how-ai-transitions-from-word-positions-to-word-meanings-a-new-study-sheds-light","status":"publish","type":"post","link":"https:\/\/www.tun.com\/home\/how-ai-transitions-from-word-positions-to-word-meanings-a-new-study-sheds-light\/","title":{"rendered":"How AI Transitions From Word Positions to Word Meanings: A New Study Sheds Light"},"content":{"rendered":"\n<div class=\"wp-block-group\"><div class=\"wp-block-group__inner-container is-layout-constrained wp-block-group-is-layout-constrained\">\n<div class=\"wp-block-uagb-blockquote uagb-block-e7eb3fc3 uagb-blockquote__skin-border uagb-blockquote__stack-img-none\"><blockquote class=\"uagb-blockquote\"><div class=\"uagb-blockquote__content\">A new study uncovers how AI systems transition from relying on word positions to understanding word meanings with increased data, offering significant insights into the technology behind tools like ChatGPT and Gemini.<\/div><footer><div class=\"uagb-blockquote__author-wrap uagb-blockquote__author-at-left\"><\/div><\/footer><\/blockquote><\/div>\n\n\n\n<div class=\"wp-block-group is-content-justification-space-between is-nowrap is-layout-flex wp-container-core-group-is-layout-0dfbf163 wp-block-group-is-layout-flex\"><div style=\"font-size:16px;\" class=\"has-text-align-left wp-block-post-author\"><div class=\"wp-block-post-author__content\"><p class=\"wp-block-post-author__name\">The University Network<\/p><\/div><\/div>\n\n\n<div class=\"wp-block-uagb-social-share uagb-social-share__outer-wrap uagb-social-share__layout-horizontal uagb-block-ee584a31\">\n<div class=\"wp-block-uagb-social-share-child uagb-ss-repeater uagb-ss__wrapper uagb-block-ec619ce7\"><span class=\"uagb-ss__link\" data-href=\"https:\/\/www.facebook.com\/sharer.php?u=\" tabindex=\"0\" role=\"button\" aria-label=\"facebook\"><span class=\"uagb-ss__source-wrap\"><span class=\"uagb-ss__source-icon\"><svg xmlns=\"https:\/\/www.w3.org\/2000\/svg\" viewBox=\"0 0 512 512\"><path d=\"M504 256C504 119 393 8 256 8S8 119 8 256c0 123.8 90.69 226.4 209.3 245V327.7h-63V256h63v-54.64c0-62.15 37-96.48 93.67-96.48 27.14 0 55.52 4.84 55.52 4.84v61h-31.28c-30.8 0-40.41 19.12-40.41 38.73V256h68.78l-11 71.69h-57.78V501C413.3 482.4 504 379.8 504 256z\"><\/path><\/svg><\/span><\/span><\/span><\/div>\n\n\n\n<div class=\"wp-block-uagb-social-share-child uagb-ss-repeater uagb-ss__wrapper uagb-block-32d99934\"><span class=\"uagb-ss__link\" data-href=\"https:\/\/twitter.com\/share?url=\" tabindex=\"0\" role=\"button\" aria-label=\"twitter\"><span class=\"uagb-ss__source-wrap\"><span class=\"uagb-ss__source-icon\"><svg xmlns=\"https:\/\/www.w3.org\/2000\/svg\" viewBox=\"0 0 512 512\"><path d=\"M389.2 48h70.6L305.6 224.2 487 464H345L233.7 318.6 106.5 464H35.8L200.7 275.5 26.8 48H172.4L272.9 180.9 389.2 48zM364.4 421.8h39.1L151.1 88h-42L364.4 421.8z\"><\/path><\/svg><\/span><\/span><\/span><\/div>\n\n\n\n<div class=\"wp-block-uagb-social-share-child uagb-ss-repeater uagb-ss__wrapper uagb-block-1d136f14\"><span class=\"uagb-ss__link\" data-href=\"https:\/\/www.linkedin.com\/shareArticle?url=\" tabindex=\"0\" role=\"button\" aria-label=\"linkedin\"><span class=\"uagb-ss__source-wrap\"><span class=\"uagb-ss__source-icon\"><svg xmlns=\"https:\/\/www.w3.org\/2000\/svg\" viewBox=\"0 0 448 512\"><path d=\"M416 32H31.9C14.3 32 0 46.5 0 64.3v383.4C0 465.5 14.3 480 31.9 480H416c17.6 0 32-14.5 32-32.3V64.3c0-17.8-14.4-32.3-32-32.3zM135.4 416H69V202.2h66.5V416zm-33.2-243c-21.3 0-38.5-17.3-38.5-38.5S80.9 96 102.2 96c21.2 0 38.5 17.3 38.5 38.5 0 21.3-17.2 38.5-38.5 38.5zm282.1 243h-66.4V312c0-24.8-.5-56.7-34.5-56.7-34.6 0-39.9 27-39.9 54.9V416h-66.4V202.2h63.7v29.2h.9c8.9-16.8 30.6-34.5 62.9-34.5 67.2 0 79.7 44.3 79.7 101.9V416z\"><\/path><\/svg><\/span><\/span><\/span><\/div>\n<\/div>\n<\/div>\n<\/div><\/div>\n\n\n\n<p>The language capabilities of modern artificial intelligence systems are nothing short of remarkable, enabling natural conversations with tools like ChatGPT and Gemini, almost on par with human interaction. However, the internal workings that drive these sophisticated interactions remain largely enigmatic.<\/p>\n\n\n\n<p>A new study published in the Journal of Statistical Mechanics: Theory and Experiment (JSTAT) provides vital insights into this enigma. <\/p>\n\n\n\n<p>The research reveals that AI systems initially rely on the position of words in a sentence when trained with small data sets. But as these systems are fed more data, they transition abruptly to interpreting words based on their meanings, a shift analogous to a phase transition in physical systems.<\/p>\n\n\n\n<p>\u201cTo assess relationships between words, the network can use two strategies, one of which is to exploit the positions of words,\u201d lead author Hugo Cui, a postdoctoral researcher at Harvard University, said in a news release. &#8220;This is the first strategy that spontaneously emerges when the network is trained. However, in our study, we observed that if training continues and the network receives enough data, at a certain point \u2014 once a threshold is crossed \u2014 the strategy abruptly shifts: the network starts relying on meaning instead.\u201d<\/p>\n\n\n\n<p>The study focuses on the self-attention mechanism, a fundamental component of transformer language models such as ChatGPT and Gemini. These models are designed to process sequences of data, excelling at understanding word relationships within a sequence. <\/p>\n\n\n\n<p>Initially, these AI systems infer relationships based on word positions \u2014 identifying subjects, verbs and objects. But as training progresses, meaning takes precedence.\u00a0<\/p>\n\n\n\n<p>\u201cWhen we designed this work, we simply wanted to study which strategies, or mix of strategies, the networks would adopt,&#8221; Cui added. &#8220;But what we found was somewhat surprising: below a certain threshold, the network relied exclusively on position, while above it, only on meaning.\u201d<\/p>\n\n\n\n<p>This shift, described by Cui as a phase transition, mirrors concepts from statistical physics. In physics, phase transitions describe changes in states of matter \u2014 like water turning into vapor. Similarly, AI neural networks, composed of numerous interconnected nodes, exhibit a collective behavior that can be understood through statistical methods.<\/p>\n\n\n\n<p>\u201cUnderstanding from a theoretical viewpoint that the strategy shift happens in this manner is important,\u201d Cui explained. \u201cOur networks are simplified compared to the complex models people interact with daily, but they can give us hints to begin to understand the conditions that cause a model to stabilize on one strategy or another. This theoretical knowledge could hopefully be used in the future to make the use of neural networks more efficient, and safer.\u201d<\/p>\n\n\n\n<div style=\"height:13px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p><strong>Source:<\/strong> <a href=\"https:\/\/www.eurekalert.org\/news-releases\/1089568\" target=\"_blank\" rel=\"noopener\" title=\"\">Sissa Medialab<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>The language capabilities of modern artificial intelligence systems are nothing short of remarkable, enabling natural conversations with tools like ChatGPT and Gemini, almost on par with human interaction. However, the internal workings that drive these sophisticated interactions remain largely enigmatic. A new study published in the Journal of Statistical Mechanics: Theory and Experiment (JSTAT) provides [&hellip;]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"single-no-separators","format":"standard","meta":{"_acf_changed":false,"_uag_custom_page_level_css":"","_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[8],"tags":[78],"class_list":["post-26870","post","type-post","status-publish","format-standard","hentry","category-ai","tag-harvard-university"],"acf":[],"aioseo_notices":[],"uagb_featured_image_src":{"full":false,"thumbnail":false,"medium":false,"medium_large":false,"large":false,"1536x1536":false,"2048x2048":false},"uagb_author_info":{"display_name":"The University Network","author_link":"https:\/\/www.tun.com\/home\/author\/funky_junkie\/"},"uagb_comment_info":0,"uagb_excerpt":"The language capabilities of modern artificial intelligence systems are nothing short of remarkable, enabling natural conversations with tools like ChatGPT and Gemini, almost on par with human interaction. However, the internal workings that drive these sophisticated interactions remain largely enigmatic. A new study published in the Journal of Statistical Mechanics: Theory and Experiment (JSTAT) provides&hellip;","_links":{"self":[{"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/posts\/26870","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/comments?post=26870"}],"version-history":[{"count":5,"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/posts\/26870\/revisions"}],"predecessor-version":[{"id":26877,"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/posts\/26870\/revisions\/26877"}],"wp:attachment":[{"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/media?parent=26870"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/categories?post=26870"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/tags?post=26870"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}