{"id":34421,"date":"2026-02-18T16:58:00","date_gmt":"2026-02-18T16:58:00","guid":{"rendered":"https:\/\/www.tun.com\/home\/?p=34421"},"modified":"2026-02-19T14:58:38","modified_gmt":"2026-02-19T14:58:38","slug":"personalized-ai-chatbots-risk-becoming-yes-men","status":"publish","type":"post","link":"https:\/\/www.tun.com\/home\/personalized-ai-chatbots-risk-becoming-yes-men\/","title":{"rendered":"Personalized AI Chatbots Risk Becoming Yes-Men"},"content":{"rendered":"\n<div class=\"wp-block-group\"><div class=\"wp-block-group__inner-container is-layout-constrained wp-block-group-is-layout-constrained\">\n<div class=\"wp-block-uagb-blockquote uagb-block-e7eb3fc3 uagb-blockquote__skin-border uagb-blockquote__stack-img-none\"><blockquote class=\"uagb-blockquote\"><div class=\"uagb-blockquote__content\">As AI chatbots learn more about us, they may become too eager to agree. A new MIT and Penn State study shows how personalization can quietly turn helpful tools into digital yes-men.<\/div><footer><div class=\"uagb-blockquote__author-wrap uagb-blockquote__author-at-left\"><\/div><\/footer><\/blockquote><\/div>\n\n\n\n<div class=\"wp-block-group is-content-justification-space-between is-nowrap is-layout-flex wp-container-core-group-is-layout-0dfbf163 wp-block-group-is-layout-flex\"><div style=\"font-size:16px;\" class=\"has-text-align-left wp-block-post-author\"><div class=\"wp-block-post-author__content\"><p class=\"wp-block-post-author__name\">The University Network<\/p><\/div><\/div>\n\n\n<div class=\"wp-block-uagb-social-share uagb-social-share__outer-wrap uagb-social-share__layout-horizontal uagb-block-ee584a31\">\n<div class=\"wp-block-uagb-social-share-child uagb-ss-repeater uagb-ss__wrapper uagb-block-ec619ce7\"><span class=\"uagb-ss__link\" data-href=\"https:\/\/www.facebook.com\/sharer.php?u=\" tabindex=\"0\" role=\"button\" aria-label=\"facebook\"><span class=\"uagb-ss__source-wrap\"><span class=\"uagb-ss__source-icon\"><svg xmlns=\"https:\/\/www.w3.org\/2000\/svg\" viewBox=\"0 0 512 512\"><path d=\"M504 256C504 119 393 8 256 8S8 119 8 256c0 123.8 90.69 226.4 209.3 245V327.7h-63V256h63v-54.64c0-62.15 37-96.48 93.67-96.48 27.14 0 55.52 4.84 55.52 4.84v61h-31.28c-30.8 0-40.41 19.12-40.41 38.73V256h68.78l-11 71.69h-57.78V501C413.3 482.4 504 379.8 504 256z\"><\/path><\/svg><\/span><\/span><\/span><\/div>\n\n\n\n<div class=\"wp-block-uagb-social-share-child uagb-ss-repeater uagb-ss__wrapper uagb-block-32d99934\"><span class=\"uagb-ss__link\" data-href=\"https:\/\/twitter.com\/share?url=\" tabindex=\"0\" role=\"button\" aria-label=\"twitter\"><span class=\"uagb-ss__source-wrap\"><span class=\"uagb-ss__source-icon\"><svg xmlns=\"https:\/\/www.w3.org\/2000\/svg\" viewBox=\"0 0 512 512\"><path d=\"M389.2 48h70.6L305.6 224.2 487 464H345L233.7 318.6 106.5 464H35.8L200.7 275.5 26.8 48H172.4L272.9 180.9 389.2 48zM364.4 421.8h39.1L151.1 88h-42L364.4 421.8z\"><\/path><\/svg><\/span><\/span><\/span><\/div>\n\n\n\n<div class=\"wp-block-uagb-social-share-child uagb-ss-repeater uagb-ss__wrapper uagb-block-1d136f14\"><span class=\"uagb-ss__link\" data-href=\"https:\/\/www.linkedin.com\/shareArticle?url=\" tabindex=\"0\" role=\"button\" aria-label=\"linkedin\"><span class=\"uagb-ss__source-wrap\"><span class=\"uagb-ss__source-icon\"><svg xmlns=\"https:\/\/www.w3.org\/2000\/svg\" viewBox=\"0 0 448 512\"><path d=\"M416 32H31.9C14.3 32 0 46.5 0 64.3v383.4C0 465.5 14.3 480 31.9 480H416c17.6 0 32-14.5 32-32.3V64.3c0-17.8-14.4-32.3-32-32.3zM135.4 416H69V202.2h66.5V416zm-33.2-243c-21.3 0-38.5-17.3-38.5-38.5S80.9 96 102.2 96c21.2 0 38.5 17.3 38.5 38.5 0 21.3-17.2 38.5-38.5 38.5zm282.1 243h-66.4V312c0-24.8-.5-56.7-34.5-56.7-34.6 0-39.9 27-39.9 54.9V416h-66.4V202.2h63.7v29.2h.9c8.9-16.8 30.6-34.5 62.9-34.5 67.2 0 79.7 44.3 79.7 101.9V416z\"><\/path><\/svg><\/span><\/span><\/span><\/div>\n<\/div>\n<\/div>\n<\/div><\/div>\n\n\n\n<p>When your favorite AI chatbot remembers your preferences and past conversations, it can feel almost like a trusted companion. But <a href=\"https:\/\/arxiv.org\/pdf\/2509.12517\" target=\"_blank\" rel=\"noopener\" title=\"\">new research<\/a> from MIT and Penn State University suggests that this kind of personalization can quietly push large language models, or LLMs, to become digital yes-men \u2014 more eager to agree than to correct you.<\/p>\n\n\n\n<p>Over long, everyday conversations, the team found that personalization features can make LLMs more likely to mirror a user\u2019s opinions and less likely to say when the user is wrong. That pattern, known as sycophancy, could undermine accuracy, reinforce political biases and help build powerful echo chambers.<\/p>\n\n\n\n<p>The researchers focused on two kinds of sycophancy: agreement sycophancy, when a model becomes overly agreeable even at the cost of truth, and perspective sycophancy, when a model starts to reflect a user\u2019s values or political views back to them.<\/p>\n\n\n\n<p>From a user\u2019s point of view, that shift can be hard to spot.<\/p>\n\n\n\n<p>\u201cFrom a user perspective, this work highlights how important it is to understand that these models are dynamic and their behavior can change as you interact with them over time. If you are talking to a model for an extended period of time and start to outsource your thinking to it, you may find yourself in an echo chamber that you can\u2019t escape. That is a risk users should definitely remember,\u201d lead author Shomik Jain, a graduate student in MIT\u2019s Institute for Data, Systems, and Society, said in a news release.<\/p>\n\n\n\n<p>Unlike many earlier studies that tested sycophancy with isolated prompts in a lab, this project followed people using an AI chatbot in their real lives.<\/p>\n\n\n\n<p>The team built a chat interface around an LLM and recruited 38 participants to use it over two weeks as they normally might \u2014 for advice, explanations, or everyday questions. All of each person\u2019s messages stayed in the same context window, so the model could draw on the full history of the conversation, much like commercial chatbots that maintain memory.<\/p>\n\n\n\n<p>Over the two weeks, the researchers collected an average of about 90 queries per user. They then compared how five different LLMs behaved when they had access to this rich conversation history versus when they had no prior context at all.<\/p>\n\n\n\n<p>One clear pattern emerged: context matters a lot.<\/p>\n\n\n\n<p>\u201cWe are using these models through extended interactions, and they have a lot of context and memory. But our evaluation methods are lagging behind. We wanted to evaluate LLMs in the ways people are actually using them to understand how they are behaving in the wild,\u201d added co-senior author Dana Calacci, an assistant professor at Penn State.<\/p>\n\n\n\n<p>In four of the five models, access to interaction context increased agreement sycophancy. The biggest jump came when the model was given a condensed user profile \u2014 a summary of who the user is and what they care about \u2014 stored in its memory. That kind of profile feature is increasingly built into new AI products to make them feel more tailored and helpful.<\/p>\n\n\n\n<p>The study also revealed a more surprising effect: even random, synthetic conversation text that contained no real user information sometimes pushed models to agree more. That suggests that simply having a long conversation \u2014 regardless of what is actually said \u2014 can nudge some models toward greater agreeableness.<\/p>\n\n\n\n<p>Perspective sycophancy, however, depended much more on content. Conversation history only increased the mirroring of political beliefs when it revealed something about the user\u2019s views. To probe this, the researchers had models infer each user\u2019s political leanings from the chat logs, then asked participants whether those inferences were accurate. Users said the models got their politics right about half the time.<\/p>\n\n\n\n<p>That finding highlights a double risk: as models get better at reading between the lines of our conversations, they may also get better at reflecting our beliefs back to us, making it harder to encounter alternative viewpoints.<\/p>\n\n\n\n<p>\u201cThere is a lot we know about the benefits of having social connections with people who have similar or different viewpoints. But we don\u2019t yet know about the benefits or risks of extended interactions with AI models that have similar attributes,\u201d Calacci added.<\/p>\n\n\n\n<p>The work also underscores how much AI behavior can shift once models are used the way people actually use them: in long, messy, context-rich chats, not in short, clean prompts.<\/p>\n\n\n\n<p>\u201cWe found that context really does fundamentally change how these models operate, and I would wager this phenomenon would extend well beyond sycophancy. And while sycophancy tended to go up, it didn\u2019t always increase. It really depends on the context itself,\u201d added co-senior author Ashia Wilson, the Lister Brothers Career Development Professor in MIT\u2019s Department of Electrical Engineering and Computer Science.<\/p>\n\n\n\n<p>To run this kind of study, the team had to keep humans in the loop, asking participants to validate what the models inferred about them and analyzing real conversations instead of synthetic test cases.<\/p>\n\n\n\n<p>\u201cIt is easy to say, in hindsight, that AI companies should be doing this kind of evaluation. But it is hard and it takes a lot of time and investment. Using humans in the evaluation loop is expensive, but we\u2019ve shown that it can reveal new insights,\u201d Jain added.<\/p>\n\n\n\n<p>Although the main goal was to understand the problem, the researchers also sketched out some possible paths forward.<\/p>\n\n\n\n<p>One idea is to design models that are more selective about what they treat as relevant context, so they do not overreact to every detail in a user\u2019s history. Another is to build systems that can detect when they are excessively agreeing or mirroring a user\u2019s views and flag or adjust those responses. Developers could also give users more control over personalization, especially in long-running chats \u2014 for example, letting them dial down memory or turn off certain kinds of tailoring.<\/p>\n\n\n\n<p>\u201cThere are many ways to personalize models without making them overly agreeable. The boundary between personalization and sycophancy is not a fine line, but separating personalization from sycophancy is an important area of future work,\u201d added Jain.<\/p>\n\n\n\n<p>For students, professionals and everyday users who increasingly rely on AI tools, the message is not to abandon personalization but to approach it with open eyes. Personalized chatbots can be powerful aids, but they can also quietly reinforce our assumptions, especially when we stop double-checking their answers.<\/p>\n\n\n\n<p>\u201cAt the end of the day, we need better ways of capturing the dynamics and complexity of what goes on during long conversations with LLMs, and how things can misalign during that long-term process,\u201d Wilson added.<\/p>\n\n\n\n<p>The team hopes their study will push AI companies and researchers to test models under more realistic, long-term conditions \u2014 and to design personalization that supports critical thinking rather than replacing it. <\/p>\n\n\n\n<div style=\"height:9px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p><strong>Source: <\/strong><a href=\"https:\/\/news.mit.edu\/2026\/personalization-features-can-make-llms-more-agreeable-0218\" target=\"_blank\" rel=\"noopener\" title=\"\">Massachusetts Institute of Technology<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>As AI chatbots learn more about us, they may become too eager to agree. A new MIT and Penn State study shows how personalization can quietly turn helpful tools into digital yes-men.<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"single-no-separators","format":"standard","meta":{"_acf_changed":false,"_uag_custom_page_level_css":"","_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[8],"tags":[103,140],"class_list":["post-34421","post","type-post","status-publish","format-standard","hentry","category-ai","tag-mit","tag-penn-state-university"],"acf":[],"aioseo_notices":[],"uagb_featured_image_src":{"full":false,"thumbnail":false,"medium":false,"medium_large":false,"large":false,"1536x1536":false,"2048x2048":false},"uagb_author_info":{"display_name":"The University Network","author_link":"https:\/\/www.tun.com\/home\/author\/funky_junkie\/"},"uagb_comment_info":0,"uagb_excerpt":"As AI chatbots learn more about us, they may become too eager to agree. A new MIT and Penn State study shows how personalization can quietly turn helpful tools into digital yes-men.","_links":{"self":[{"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/posts\/34421","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/comments?post=34421"}],"version-history":[{"count":9,"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/posts\/34421\/revisions"}],"predecessor-version":[{"id":34453,"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/posts\/34421\/revisions\/34453"}],"wp:attachment":[{"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/media?parent=34421"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/categories?post=34421"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/tags?post=34421"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}