{"id":26145,"date":"2025-06-19T20:59:51","date_gmt":"2025-06-19T20:59:51","guid":{"rendered":"https:\/\/www.tun.com\/home\/?p=26145"},"modified":"2025-06-19T20:59:52","modified_gmt":"2025-06-19T20:59:52","slug":"mit-engineers-uncover-bias-in-large-language-models","status":"publish","type":"post","link":"https:\/\/www.tun.com\/home\/mit-engineers-uncover-bias-in-large-language-models\/","title":{"rendered":"MIT Engineers Uncover Bias in Large Language Models"},"content":{"rendered":"\n<div class=\"wp-block-group\"><div class=\"wp-block-group__inner-container is-layout-constrained wp-block-group-is-layout-constrained\">\n<div class=\"wp-block-uagb-blockquote uagb-block-e7eb3fc3 uagb-blockquote__skin-border uagb-blockquote__stack-img-none\"><blockquote class=\"uagb-blockquote\"><div class=\"uagb-blockquote__content\">MIT scientists have discovered why large language models favor beginnings and endings in texts, proposing new frameworks to eliminate this bias, significantly improving AI performance across multiple fields.<\/div><footer><div class=\"uagb-blockquote__author-wrap uagb-blockquote__author-at-left\"><\/div><\/footer><\/blockquote><\/div>\n\n\n\n<div class=\"wp-block-group is-content-justification-space-between is-nowrap is-layout-flex wp-container-core-group-is-layout-0dfbf163 wp-block-group-is-layout-flex\"><div style=\"font-size:16px;\" class=\"has-text-align-left wp-block-post-author\"><div class=\"wp-block-post-author__content\"><p class=\"wp-block-post-author__name\">The University Network<\/p><\/div><\/div>\n\n\n<div class=\"wp-block-uagb-social-share uagb-social-share__outer-wrap uagb-social-share__layout-horizontal uagb-block-ee584a31\">\n<div class=\"wp-block-uagb-social-share-child uagb-ss-repeater uagb-ss__wrapper uagb-block-ec619ce7\"><span class=\"uagb-ss__link\" data-href=\"https:\/\/www.facebook.com\/sharer.php?u=\" tabindex=\"0\" role=\"button\" aria-label=\"facebook\"><span class=\"uagb-ss__source-wrap\"><span class=\"uagb-ss__source-icon\"><svg xmlns=\"https:\/\/www.w3.org\/2000\/svg\" viewBox=\"0 0 512 512\"><path d=\"M504 256C504 119 393 8 256 8S8 119 8 256c0 123.8 90.69 226.4 209.3 245V327.7h-63V256h63v-54.64c0-62.15 37-96.48 93.67-96.48 27.14 0 55.52 4.84 55.52 4.84v61h-31.28c-30.8 0-40.41 19.12-40.41 38.73V256h68.78l-11 71.69h-57.78V501C413.3 482.4 504 379.8 504 256z\"><\/path><\/svg><\/span><\/span><\/span><\/div>\n\n\n\n<div class=\"wp-block-uagb-social-share-child uagb-ss-repeater uagb-ss__wrapper uagb-block-32d99934\"><span class=\"uagb-ss__link\" data-href=\"https:\/\/twitter.com\/share?url=\" tabindex=\"0\" role=\"button\" aria-label=\"twitter\"><span class=\"uagb-ss__source-wrap\"><span class=\"uagb-ss__source-icon\"><svg xmlns=\"https:\/\/www.w3.org\/2000\/svg\" viewBox=\"0 0 512 512\"><path d=\"M389.2 48h70.6L305.6 224.2 487 464H345L233.7 318.6 106.5 464H35.8L200.7 275.5 26.8 48H172.4L272.9 180.9 389.2 48zM364.4 421.8h39.1L151.1 88h-42L364.4 421.8z\"><\/path><\/svg><\/span><\/span><\/span><\/div>\n\n\n\n<div class=\"wp-block-uagb-social-share-child uagb-ss-repeater uagb-ss__wrapper uagb-block-1d136f14\"><span class=\"uagb-ss__link\" data-href=\"https:\/\/www.linkedin.com\/shareArticle?url=\" tabindex=\"0\" role=\"button\" aria-label=\"linkedin\"><span class=\"uagb-ss__source-wrap\"><span class=\"uagb-ss__source-icon\"><svg xmlns=\"https:\/\/www.w3.org\/2000\/svg\" viewBox=\"0 0 448 512\"><path d=\"M416 32H31.9C14.3 32 0 46.5 0 64.3v383.4C0 465.5 14.3 480 31.9 480H416c17.6 0 32-14.5 32-32.3V64.3c0-17.8-14.4-32.3-32-32.3zM135.4 416H69V202.2h66.5V416zm-33.2-243c-21.3 0-38.5-17.3-38.5-38.5S80.9 96 102.2 96c21.2 0 38.5 17.3 38.5 38.5 0 21.3-17.2 38.5-38.5 38.5zm282.1 243h-66.4V312c0-24.8-.5-56.7-34.5-56.7-34.6 0-39.9 27-39.9 54.9V416h-66.4V202.2h63.7v29.2h.9c8.9-16.8 30.6-34.5 62.9-34.5 67.2 0 79.7 44.3 79.7 101.9V416z\"><\/path><\/svg><\/span><\/span><\/span><\/div>\n<\/div>\n<\/div>\n<\/div><\/div>\n\n\n\n<p>MIT researchers have uncovered a critical flaw in large language models (LLMs) that biases them towards information at the beginning and end of documents while overlooking the middle. This tendency, known as \u201cposition bias,\u201d was identified through an innovative theoretical framework aimed at enhancing the reliability and accuracy of these models.<\/p>\n\n\n\n<p>In practical terms, this bias might mean that a virtual assistant sifting through a lengthy legal document could miss crucial information if it&#8217;s buried in the middle. The bias results from how these models process input data, which could lead to inconsistencies in various applications.<\/p>\n\n\n\n<p>\u201cThese models are black boxes, so as an LLM user, you probably don\u2019t know that position bias can cause your model to be inconsistent. You just feed it your documents in whatever order you want and expect it to work. But by understanding the underlying mechanism of these black-box models better, we can improve them by addressing these limitations,\u201d first author  Xinyi Wu, a graduate student in the MIT Institute for Data, Systems and Society (IDSS) and the Laboratory for Information and Decision Systems (LIDS), said in a news release.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Decoding the Bias<\/h2>\n\n\n\n<p>LLMs, like GPT-4 and Claude, rely on a neural network architecture known as a transformer. These transformers use a mechanism called attention, which helps the model understand the context by focusing on related words within a sequence. <\/p>\n\n\n\n<p>However, the researchers found that design choices in how these models handle data, such as the use of attention masks and positional encodings, contribute to position bias.<\/p>\n\n\n\n<p>The <a href=\"https:\/\/arxiv.org\/pdf\/2502.01951\" target=\"_blank\" rel=\"noopener\" title=\"\">study<\/a> employed a graph-based framework to analyze these design elements. Through their research, the team discovered that causal masking, a method used to limit attention to preceding words, inherently biases the model towards initial words even if those words are less important.<\/p>\n\n\n\n<p>\u201cGraphs are a flexible language to describe the dependent relationship among words within the attention mechanism and trace them across multiple layers,\u201d added Wu.<\/p>\n\n\n\n<p>These biases can be detrimental, especially in applications outside natural language generation, like information retrieval or ranking.<\/p>\n\n\n\n<p>&#8220;While it is often true that earlier words and later words in a sentence are more important, if an LLM is used on a task that is not natural language generation, like ranking or information retrieval, these biases can be extremely harmful,&#8221; Wu added.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Experimental Insights and Future Directions<\/h2>\n\n\n\n<p>To further explore this phenomenon, the researchers conducted experiments by varying the position of the correct answer in text sequences. <\/p>\n\n\n\n<p>They observed a \u201clost-in-the-middle\u201d trend, where the model performed best when the target information was at the beginning or end of a sequence, faltering in the middle.<\/p>\n\n\n\n<p>Their findings suggest that tweaking the design, such as using alternate masking techniques or minimizing extra layers in the attention mechanism, can mitigate this bias and enhance model accuracy.<\/p>\n\n\n\n<p>\u201cBy doing a combination of theory and experiments, we were able to look at the consequences of model design choices that weren\u2019t clear at the time. If you want to use a model in high-stakes applications, you must know when it will work, when it won\u2019t, and why,\u201d added co-senior author Ali Jadbabaie, a professor and head of the Department of Civil and Environmental Engineering, a core faculty member of IDSS and a principal investigator in LIDS.\u00a0<\/p>\n\n\n\n<p>Next, the team aims to delve deeper into the effects of positional encodings and explore how position bias might be advantageous in certain applications.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Impact and Significance<\/h2>\n\n\n\n<p>This breakthrough has wide-ranging implications. Improved LLMs could result in more reliable chatbots, fairer medical AI systems and more attentive code assistants. <\/p>\n\n\n\n<p>By addressing position bias, this research helps make LLMs more robust and reliable across various domains.<\/p>\n\n\n\n<p>The research will be presented at the International Conference on Machine Learning.<\/p>\n\n\n\n<div style=\"height:12px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p><strong>Source:<\/strong> <a href=\"https:\/\/news.mit.edu\/2025\/unpacking-large-language-model-bias-0617\" target=\"_blank\" rel=\"noopener\" title=\"\">Massachusetts Institute of Technology<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>MIT researchers have uncovered a critical flaw in large language models (LLMs) that biases them towards information at the beginning and end of documents while overlooking the middle. This tendency, known as \u201cposition bias,\u201d was identified through an innovative theoretical framework aimed at enhancing the reliability and accuracy of these models. In practical terms, this [&hellip;]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"single-no-separators","format":"standard","meta":{"_acf_changed":false,"_uag_custom_page_level_css":"","_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[8],"tags":[103],"class_list":["post-26145","post","type-post","status-publish","format-standard","hentry","category-ai","tag-mit"],"acf":[],"aioseo_notices":[],"uagb_featured_image_src":{"full":false,"thumbnail":false,"medium":false,"medium_large":false,"large":false,"1536x1536":false,"2048x2048":false},"uagb_author_info":{"display_name":"The University Network","author_link":"https:\/\/www.tun.com\/home\/author\/funky_junkie\/"},"uagb_comment_info":0,"uagb_excerpt":"MIT researchers have uncovered a critical flaw in large language models (LLMs) that biases them towards information at the beginning and end of documents while overlooking the middle. This tendency, known as \u201cposition bias,\u201d was identified through an innovative theoretical framework aimed at enhancing the reliability and accuracy of these models. In practical terms, this&hellip;","_links":{"self":[{"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/posts\/26145","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/comments?post=26145"}],"version-history":[{"count":9,"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/posts\/26145\/revisions"}],"predecessor-version":[{"id":26157,"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/posts\/26145\/revisions\/26157"}],"wp:attachment":[{"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/media?parent=26145"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/categories?post=26145"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/tags?post=26145"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}