{"id":39406,"date":"2026-08-17T14:51:00","date_gmt":"2026-08-17T14:51:00","guid":{"rendered":"https:\/\/www.tun.com\/home\/?p=39406"},"modified":"2026-08-18T13:41:42","modified_gmt":"2026-08-18T13:41:42","slug":"hugging-face-reproduced-2200-icml-papers-heres-what-they-found","status":"publish","type":"post","link":"https:\/\/www.tun.com\/home\/hugging-face-reproduced-2200-icml-papers-heres-what-they-found\/","title":{"rendered":"Hugging Face Reproduced 2,200 ICML Papers. Here&#8217;s What They Found"},"content":{"rendered":"\n<div class=\"wp-block-group\"><div class=\"wp-block-group__inner-container is-layout-constrained wp-block-group-is-layout-constrained\">\n<div class=\"wp-block-uagb-blockquote uagb-block-e7eb3fc3 uagb-blockquote__skin-border uagb-blockquote__stack-img-none\"><blockquote class=\"uagb-blockquote\"><div class=\"uagb-blockquote__content\">Hugging Face organized a community hackathon in which over 1,200 participants used coding agents to reproduce papers from ICML 2026 \u2014 one of AI&#8217;s biggest conferences. The results: 23% of examined papers had at least one claim falsified or contested, and several accepted papers contained confirmed errors.<\/div><footer><div class=\"uagb-blockquote__author-wrap uagb-blockquote__author-at-left\"><\/div><\/footer><\/blockquote><\/div>\n\n\n\n<div class=\"wp-block-group is-content-justification-space-between is-nowrap is-layout-flex wp-container-core-group-is-layout-b0ffac9c wp-block-group-is-layout-flex\"><div style=\"font-size:16px\" class=\"has-text-align-left wp-block-post-author\"><div class=\"wp-block-post-author__content\"><p class=\"wp-block-post-author__name\">The University Network<\/p><\/div><\/div>\n\n\n<div class=\"wp-block-uagb-social-share uagb-social-share__outer-wrap uagb-social-share__layout-horizontal uagb-block-ee584a31\">\n<div class=\"wp-block-uagb-social-share-child uagb-ss-repeater uagb-ss__wrapper uagb-block-ec619ce7\"><span class=\"uagb-ss__link\" data-href=\"https:\/\/www.facebook.com\/sharer.php?u=\" tabindex=\"0\" role=\"button\" aria-label=\"facebook\"><span class=\"uagb-ss__source-wrap\"><span class=\"uagb-ss__source-icon\"><svg xmlns=\"https:\/\/www.w3.org\/2000\/svg\" viewBox=\"0 0 512 512\"><path d=\"M504 256C504 119 393 8 256 8S8 119 8 256c0 123.8 90.69 226.4 209.3 245V327.7h-63V256h63v-54.64c0-62.15 37-96.48 93.67-96.48 27.14 0 55.52 4.84 55.52 4.84v61h-31.28c-30.8 0-40.41 19.12-40.41 38.73V256h68.78l-11 71.69h-57.78V501C413.3 482.4 504 379.8 504 256z\"><\/path><\/svg><\/span><\/span><\/span><\/div>\n\n\n\n<div class=\"wp-block-uagb-social-share-child uagb-ss-repeater uagb-ss__wrapper uagb-block-32d99934\"><span class=\"uagb-ss__link\" data-href=\"https:\/\/twitter.com\/share?url=\" tabindex=\"0\" role=\"button\" aria-label=\"twitter\"><span class=\"uagb-ss__source-wrap\"><span class=\"uagb-ss__source-icon\"><svg xmlns=\"https:\/\/www.w3.org\/2000\/svg\" viewBox=\"0 0 512 512\"><path d=\"M389.2 48h70.6L305.6 224.2 487 464H345L233.7 318.6 106.5 464H35.8L200.7 275.5 26.8 48H172.4L272.9 180.9 389.2 48zM364.4 421.8h39.1L151.1 88h-42L364.4 421.8z\"><\/path><\/svg><\/span><\/span><\/span><\/div>\n\n\n\n<div class=\"wp-block-uagb-social-share-child uagb-ss-repeater uagb-ss__wrapper uagb-block-1d136f14\"><span class=\"uagb-ss__link\" data-href=\"https:\/\/www.linkedin.com\/shareArticle?url=\" tabindex=\"0\" role=\"button\" aria-label=\"linkedin\"><span class=\"uagb-ss__source-wrap\"><span class=\"uagb-ss__source-icon\"><svg xmlns=\"https:\/\/www.w3.org\/2000\/svg\" viewBox=\"0 0 448 512\"><path d=\"M416 32H31.9C14.3 32 0 46.5 0 64.3v383.4C0 465.5 14.3 480 31.9 480H416c17.6 0 32-14.5 32-32.3V64.3c0-17.8-14.4-32.3-32-32.3zM135.4 416H69V202.2h66.5V416zm-33.2-243c-21.3 0-38.5-17.3-38.5-38.5S80.9 96 102.2 96c21.2 0 38.5 17.3 38.5 38.5 0 21.3-17.2 38.5-38.5 38.5zm282.1 243h-66.4V312c0-24.8-.5-56.7-34.5-56.7-34.6 0-39.9 27-39.9 54.9V416h-66.4V202.2h63.7v29.2h.9c8.9-16.8 30.6-34.5 62.9-34.5 67.2 0 79.7 44.3 79.7 101.9V416z\"><\/path><\/svg><\/span><\/span><\/span><\/div>\n<\/div>\n<\/div>\n<\/div><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">In July 2026, Hugging Face asked a simple, uncomfortable question: if you actually tried to reproduce every paper from a major AI conference, what would you find? Over 19 days, 1,221 community members provided an answer \u2014 and it wasn&#8217;t entirely reassuring.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The ICML 2026 Open Reproductions challenge ran from July 15 to Aug. 2, pairing human participants with their own coding agents \u2014 tools like Claude Code, Codex, Cursor and OpenResearch&#8217;s orx \u2014 to verify the specific scientific claims in accepted papers from the International Conference on Machine Learning. By the time the challenge closed, participants had published 6,816 Trackio logbooks covering 2,226 papers, roughly 34% of the entire conference. An automated judge running on GLM-5.2, an open-weights model, then evaluated each logbook and issued per-claim verdicts: verified, falsified, toy-scale or inconclusive.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What the Numbers Actually Say<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The headline results are mixed. Fifty-one percent of examined papers had at least one claim independently verified \u2014 266 were fully reproduced with every extracted claim confirmed, and 632 more were partially reproduced with nothing falsified. That adds up to 3,978 individual claims confirmed through real experiments across 35,908 total judgments.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">But 23% of examined papers had at least one claim falsified or contested. Forty-nine papers had all of their claims falsified with nothing verifiable. Perhaps most striking: 242 papers received opposite verdicts from different independent reproduction teams, which Hugging Face frames not as a methodological flaw but as evidence that reproducibility is inherently adversarial \u2014 the same paper can look solid or shaky depending on how carefully you look and for how long.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Several of the confirmed falsifications are documented in detail. One accepted spotlight paper \u2014 notable because its reviewer explicitly wrote, &#8220;My low confidence score is because I did not check all the proofs carefully&#8221; \u2014 contained a flaw in a core mathematical proof about a paging algorithm&#8217;s robustness guarantee. A participant&#8217;s logbook measured an additive term growing logarithmically rather than staying constant; Hugging Face&#8217;s own re-implementation extended the analysis to k = 1,024 and confirmed the discrepancy at roughly nine sigma. In a separate paper on attention mechanisms, three independent teams found counterexamples to a claimed theorem \u2014 but only after hundreds of simulation steps, explaining why shorter-horizon checks had missed them entirely. A third paper&#8217;s theory section analyzed reverse KL divergence while the released code, which produced all the paper&#8217;s actual results, computed forward KL instead. The authors confirmed the issue and uploaded a corrected version to arXiv.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Why the Conference System Is Under Pressure<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The scale of ICML 2026 itself is part of the story. The conference received 23,918 submissions and accepted 6,352 papers \u2014 roughly double the previous year. Reviewers are volunteers, and their capacity hasn&#8217;t scaled with submission volume. The same AI tools making it faster to run experiments and write papers are also, at least in theory, available to help check them. Hugging Face&#8217;s argument is that coding agents can attempt a paper reproduction in an afternoon that would cost a human reviewer a weekend \u2014 and can do it in parallel, thousands of times over.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This effort is structurally different from earlier reproducibility initiatives. Conference reproducibility checklists, like those introduced at NeurIPS and ICML in recent years, rely on authors self-reporting. The Hugging Face challenge placed the burden on the broader community, using agents to independently verify claims after publication. No comparable effort \u2014 not Papers With Code, not OpenReview, not the ML Reproducibility Challenge \u2014 has attempted agent-powered, claim-level verification across an entire major conference.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What This Means If You&#8217;re a Student or Early-Career Researcher<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The barrier to entry for this hackathon was low: a free Hugging Face account, a coding agent (Claude Code and Codex are accessible to students at low or no cost), and $20 in HF compute credits that Hugging Face provided to each participant. Every contributor with at least one verified logbook received a participation certificate \u2014 a concrete portfolio artifact. The 274 published full agent-trace datasets and thousands of logbooks now function as a free, searchable library showing exactly how to implement and test the claims in specific papers \u2014 useful for thesis work, capstone projects or self-directed ML study.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">There&#8217;s also a skills lesson embedded in how the challenge went wrong. Agents working alone made characteristic mistakes: misreading units, terminating experiments before scale-dependent effects appeared, and occasionally constructing confident falsifications on top of simple conversion errors. In one documented case, a participant claimed a 2x slowdown that turned out to be a units mismatch \u2014 per-trajectory versus per-batch-of-50 \u2014 and the participant&#8217;s own data actually confirmed the paper&#8217;s claimed 8x speedup. The reproductions that held up best came from human-agent pairs where the human was actively steering, catching errors and making qualitative judgment calls the agent couldn&#8217;t.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">That gap \u2014 between an agent that can run experiments and a person who can tell when the experiment is asking the wrong question \u2014 is where practical skill in AI research lives right now. Learning to supervise, correct and critically evaluate agent-run experiments isn&#8217;t just good scientific practice. It&#8217;s increasingly what research teams are looking for.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><em>Read the full Hugging Face writeup <a href=\"https:\/\/huggingface.co\/blog\/icml-2026-open-reproductions\">here<\/a>.<\/em><\/p>\n\n\n\n<div style=\"height:3px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p class=\"source-attribution wp-block-paragraph\"><strong>Source: <\/strong><a href=\"https:\/\/huggingface.co\/blog\/icml-2026-open-reproductions\" target=\"_blank\" rel=\"nofollow noopener\">Hugging Face<\/a><\/p>\n\n\n\n<details class=\"research-citations\">\n<summary>Additional research sources<\/summary>\n<ul>\n<li><a href=\"https:\/\/buttondown.com\/pollak\/archive\/ai-intelligence-briefing-august-16-2026\/\" rel=\"nofollow noopener\" target=\"_blank\">https:\/\/buttondown.com\/pollak\/archive\/ai-intelligence-briefing-august-16-2026\/<\/a><\/li>\n<li><a href=\"https:\/\/www.ucl.ac.uk\/engineering\/news\/papers-accepted-icml-2026\" rel=\"nofollow noopener\" target=\"_blank\">https:\/\/www.ucl.ac.uk\/engineering\/news\/papers-accepted-icml-2026<\/a><\/li>\n<li><a href=\"https:\/\/csconfstats.xoveexu.com\/conferences\/icml\/2026\/\" rel=\"nofollow noopener\" target=\"_blank\">https:\/\/csconfstats.xoveexu.com\/conferences\/icml\/2026\/<\/a><\/li>\n<li><a href=\"https:\/\/howaiworks.ai\/blog\/icml-2026-agent-reproduction-challenge\" rel=\"nofollow noopener\" target=\"_blank\">https:\/\/howaiworks.ai\/blog\/icml-2026-agent-reproduction-challenge<\/a><\/li>\n<li><a href=\"https:\/\/huggingface.co\/ICML-2026-agent-repro\" rel=\"nofollow noopener\" target=\"_blank\">https:\/\/huggingface.co\/ICML-2026-agent-repro<\/a><\/li>\n<\/ul>\n<\/details>\n","protected":false},"excerpt":{"rendered":"<p>Hugging Face organized a community hackathon in which over 1,200 participants used coding agents to reproduce papers from ICML 2026 \u2014 one of AI&#8217;s biggest conferences. The results: 23% of examined papers had at least one claim falsified or contested, and several accepted papers contained confirmed errors.<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"single-no-separators","format":"standard","meta":{"_acf_changed":false,"_uag_custom_page_level_css":"","_monsterinsights_skip_tracking":false,"footnotes":""},"categories":[8],"tags":[3807,724,737,3346,666,3806,733,3808,3805,3804],"class_list":["post-39406","post","type-post","status-publish","format-standard","hentry","category-ai","tag-alphaxiv","tag-anthropic","tag-coding-agents","tag-hugging-face","tag-machine-learning","tag-open-science","tag-openai","tag-openresearch","tag-peer-review","tag-reproducibility"],"acf":[],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.3 - aioseo.com -->\n\t<meta name=\"description\" content=\"Hugging Face organized a community hackathon in which over 1,200 participants used coding agents to reproduce papers from ICML 2026 \u2014 one of AI&#039;s biggest conferences. The results: 23% of examined papers had at least one claim falsified or contested, and several accepted papers contained confirmed errors.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"The University Network\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/www.tun.com\/home\/hugging-face-reproduced-2200-icml-papers-heres-what-they-found\/\" \/>\n\t\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.3\" \/>\n\t\t<meta property=\"og:locale\" content=\"en_US\" \/>\n\t\t<meta property=\"og:site_name\" content=\"TUN - The University Network\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"Hugging Face Reproduced 2,200 ICML Papers. Here\u2019s What They Found - TUN\" \/>\n\t\t<meta property=\"og:description\" content=\"Hugging Face organized a community hackathon in which over 1,200 participants used coding agents to reproduce papers from ICML 2026 \u2014 one of AI&#039;s biggest conferences. The results: 23% of examined papers had at least one claim falsified or contested, and several accepted papers contained confirmed errors.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/www.tun.com\/home\/hugging-face-reproduced-2200-icml-papers-heres-what-they-found\/\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2026-08-17T14:51:00+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2026-08-18T13:41:42+00:00\" \/>\n\t\t<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n\t\t<meta name=\"twitter:title\" content=\"Hugging Face Reproduced 2,200 ICML Papers. Here\u2019s What They Found - TUN\" \/>\n\t\t<meta name=\"twitter:description\" content=\"Hugging Face organized a community hackathon in which over 1,200 participants used coding agents to reproduce papers from ICML 2026 \u2014 one of AI&#039;s biggest conferences. The results: 23% of examined papers had at least one claim falsified or contested, and several accepted papers contained confirmed errors.\" \/>\n\t\t<script type=\"application\/ld+json\" class=\"aioseo-schema\">\n\t\t\t{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"BlogPosting\",\"@id\":\"https:\\\/\\\/www.tun.com\\\/home\\\/hugging-face-reproduced-2200-icml-papers-heres-what-they-found\\\/#blogposting\",\"name\":\"Hugging Face Reproduced 2,200 ICML Papers. Here\\u2019s What They Found - TUN\",\"headline\":\"Hugging Face Reproduced 2,200 ICML Papers. Here&#8217;s What They Found\",\"author\":{\"@id\":\"https:\\\/\\\/www.tun.com\\\/home\\\/author\\\/funky_junkie\\\/#author\"},\"publisher\":{\"@id\":\"https:\\\/\\\/www.tun.com\\\/home\\\/#organization\"},\"datePublished\":\"2026-08-17T14:51:00+00:00\",\"dateModified\":\"2026-08-18T13:41:42+00:00\",\"inLanguage\":\"en-US\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.tun.com\\\/home\\\/hugging-face-reproduced-2200-icml-papers-heres-what-they-found\\\/#webpage\"},\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.tun.com\\\/home\\\/hugging-face-reproduced-2200-icml-papers-heres-what-they-found\\\/#webpage\"},\"articleSection\":\"AI, AlphaXiv, Anthropic, coding agents, Hugging Face, machine learning, open science, OpenAI, OpenResearch, peer review, reproducibility\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.tun.com\\\/home\\\/hugging-face-reproduced-2200-icml-papers-heres-what-they-found\\\/#breadcrumblist\",\"itemListElement\":[{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/www.tun.com\\\/home\\\/#listItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.tun.com\\\/home\\\/\",\"nextItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/www.tun.com\\\/home\\\/category\\\/ai\\\/#listItem\",\"name\":\"AI\"}},{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/www.tun.com\\\/home\\\/category\\\/ai\\\/#listItem\",\"position\":2,\"name\":\"AI\",\"item\":\"https:\\\/\\\/www.tun.com\\\/home\\\/category\\\/ai\\\/\",\"nextItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/www.tun.com\\\/home\\\/hugging-face-reproduced-2200-icml-papers-heres-what-they-found\\\/#listItem\",\"name\":\"Hugging Face Reproduced 2,200 ICML Papers. Here&#8217;s What They Found\"},\"previousItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/www.tun.com\\\/home\\\/#listItem\",\"name\":\"Home\"}},{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/www.tun.com\\\/home\\\/hugging-face-reproduced-2200-icml-papers-heres-what-they-found\\\/#listItem\",\"position\":3,\"name\":\"Hugging Face Reproduced 2,200 ICML Papers. Here&#8217;s What They Found\",\"previousItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/www.tun.com\\\/home\\\/category\\\/ai\\\/#listItem\",\"name\":\"AI\"}}]},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.tun.com\\\/home\\\/#organization\",\"name\":\"Home\",\"description\":\"The University Network\",\"url\":\"https:\\\/\\\/www.tun.com\\\/home\\\/\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.tun.com\\\/home\\\/author\\\/funky_junkie\\\/#author\",\"url\":\"https:\\\/\\\/www.tun.com\\\/home\\\/author\\\/funky_junkie\\\/\",\"name\":\"The University Network\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\\\/\\\/www.tun.com\\\/home\\\/hugging-face-reproduced-2200-icml-papers-heres-what-they-found\\\/#authorImage\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/13dd0893ea3180b8e23da81ab702e21a881401f7778d2b4aedbd75103e0b7e4f?s=96&d=mm&r=g\",\"width\":96,\"height\":96,\"caption\":\"The University Network\"}},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.tun.com\\\/home\\\/hugging-face-reproduced-2200-icml-papers-heres-what-they-found\\\/#webpage\",\"url\":\"https:\\\/\\\/www.tun.com\\\/home\\\/hugging-face-reproduced-2200-icml-papers-heres-what-they-found\\\/\",\"name\":\"Hugging Face Reproduced 2,200 ICML Papers. Here\\u2019s What They Found - TUN\",\"description\":\"Hugging Face organized a community hackathon in which over 1,200 participants used coding agents to reproduce papers from ICML 2026 \\u2014 one of AI's biggest conferences. The results: 23% of examined papers had at least one claim falsified or contested, and several accepted papers contained confirmed errors.\",\"inLanguage\":\"en-US\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.tun.com\\\/home\\\/#website\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.tun.com\\\/home\\\/hugging-face-reproduced-2200-icml-papers-heres-what-they-found\\\/#breadcrumblist\"},\"author\":{\"@id\":\"https:\\\/\\\/www.tun.com\\\/home\\\/author\\\/funky_junkie\\\/#author\"},\"creator\":{\"@id\":\"https:\\\/\\\/www.tun.com\\\/home\\\/author\\\/funky_junkie\\\/#author\"},\"datePublished\":\"2026-08-17T14:51:00+00:00\",\"dateModified\":\"2026-08-18T13:41:42+00:00\"},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.tun.com\\\/home\\\/#website\",\"url\":\"https:\\\/\\\/www.tun.com\\\/home\\\/\",\"name\":\"Home\",\"description\":\"The University Network\",\"inLanguage\":\"en-US\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.tun.com\\\/home\\\/#organization\"}}]}\n\t\t<\/script>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"Hugging Face Reproduced 2,200 ICML Papers. Here\u2019s What They Found - TUN","description":"Hugging Face organized a community hackathon in which over 1,200 participants used coding agents to reproduce papers from ICML 2026 \u2014 one of AI's biggest conferences. The results: 23% of examined papers had at least one claim falsified or contested, and several accepted papers contained confirmed errors.","canonical_url":"https:\/\/www.tun.com\/home\/hugging-face-reproduced-2200-icml-papers-heres-what-they-found\/","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"BlogPosting","@id":"https:\/\/www.tun.com\/home\/hugging-face-reproduced-2200-icml-papers-heres-what-they-found\/#blogposting","name":"Hugging Face Reproduced 2,200 ICML Papers. Here\u2019s What They Found - TUN","headline":"Hugging Face Reproduced 2,200 ICML Papers. Here&#8217;s What They Found","author":{"@id":"https:\/\/www.tun.com\/home\/author\/funky_junkie\/#author"},"publisher":{"@id":"https:\/\/www.tun.com\/home\/#organization"},"datePublished":"2026-08-17T14:51:00+00:00","dateModified":"2026-08-18T13:41:42+00:00","inLanguage":"en-US","mainEntityOfPage":{"@id":"https:\/\/www.tun.com\/home\/hugging-face-reproduced-2200-icml-papers-heres-what-they-found\/#webpage"},"isPartOf":{"@id":"https:\/\/www.tun.com\/home\/hugging-face-reproduced-2200-icml-papers-heres-what-they-found\/#webpage"},"articleSection":"AI, AlphaXiv, Anthropic, coding agents, Hugging Face, machine learning, open science, OpenAI, OpenResearch, peer review, reproducibility"},{"@type":"BreadcrumbList","@id":"https:\/\/www.tun.com\/home\/hugging-face-reproduced-2200-icml-papers-heres-what-they-found\/#breadcrumblist","itemListElement":[{"@type":"ListItem","@id":"https:\/\/www.tun.com\/home\/#listItem","position":1,"name":"Home","item":"https:\/\/www.tun.com\/home\/","nextItem":{"@type":"ListItem","@id":"https:\/\/www.tun.com\/home\/category\/ai\/#listItem","name":"AI"}},{"@type":"ListItem","@id":"https:\/\/www.tun.com\/home\/category\/ai\/#listItem","position":2,"name":"AI","item":"https:\/\/www.tun.com\/home\/category\/ai\/","nextItem":{"@type":"ListItem","@id":"https:\/\/www.tun.com\/home\/hugging-face-reproduced-2200-icml-papers-heres-what-they-found\/#listItem","name":"Hugging Face Reproduced 2,200 ICML Papers. Here&#8217;s What They Found"},"previousItem":{"@type":"ListItem","@id":"https:\/\/www.tun.com\/home\/#listItem","name":"Home"}},{"@type":"ListItem","@id":"https:\/\/www.tun.com\/home\/hugging-face-reproduced-2200-icml-papers-heres-what-they-found\/#listItem","position":3,"name":"Hugging Face Reproduced 2,200 ICML Papers. Here&#8217;s What They Found","previousItem":{"@type":"ListItem","@id":"https:\/\/www.tun.com\/home\/category\/ai\/#listItem","name":"AI"}}]},{"@type":"Organization","@id":"https:\/\/www.tun.com\/home\/#organization","name":"Home","description":"The University Network","url":"https:\/\/www.tun.com\/home\/"},{"@type":"Person","@id":"https:\/\/www.tun.com\/home\/author\/funky_junkie\/#author","url":"https:\/\/www.tun.com\/home\/author\/funky_junkie\/","name":"The University Network","image":{"@type":"ImageObject","@id":"https:\/\/www.tun.com\/home\/hugging-face-reproduced-2200-icml-papers-heres-what-they-found\/#authorImage","url":"https:\/\/secure.gravatar.com\/avatar\/13dd0893ea3180b8e23da81ab702e21a881401f7778d2b4aedbd75103e0b7e4f?s=96&d=mm&r=g","width":96,"height":96,"caption":"The University Network"}},{"@type":"WebPage","@id":"https:\/\/www.tun.com\/home\/hugging-face-reproduced-2200-icml-papers-heres-what-they-found\/#webpage","url":"https:\/\/www.tun.com\/home\/hugging-face-reproduced-2200-icml-papers-heres-what-they-found\/","name":"Hugging Face Reproduced 2,200 ICML Papers. Here\u2019s What They Found - TUN","description":"Hugging Face organized a community hackathon in which over 1,200 participants used coding agents to reproduce papers from ICML 2026 \u2014 one of AI's biggest conferences. The results: 23% of examined papers had at least one claim falsified or contested, and several accepted papers contained confirmed errors.","inLanguage":"en-US","isPartOf":{"@id":"https:\/\/www.tun.com\/home\/#website"},"breadcrumb":{"@id":"https:\/\/www.tun.com\/home\/hugging-face-reproduced-2200-icml-papers-heres-what-they-found\/#breadcrumblist"},"author":{"@id":"https:\/\/www.tun.com\/home\/author\/funky_junkie\/#author"},"creator":{"@id":"https:\/\/www.tun.com\/home\/author\/funky_junkie\/#author"},"datePublished":"2026-08-17T14:51:00+00:00","dateModified":"2026-08-18T13:41:42+00:00"},{"@type":"WebSite","@id":"https:\/\/www.tun.com\/home\/#website","url":"https:\/\/www.tun.com\/home\/","name":"Home","description":"The University Network","inLanguage":"en-US","publisher":{"@id":"https:\/\/www.tun.com\/home\/#organization"}}]},"og:locale":"en_US","og:site_name":"TUN - The University Network","og:type":"article","og:title":"Hugging Face Reproduced 2,200 ICML Papers. Here\u2019s What They Found - TUN","og:description":"Hugging Face organized a community hackathon in which over 1,200 participants used coding agents to reproduce papers from ICML 2026 \u2014 one of AI's biggest conferences. The results: 23% of examined papers had at least one claim falsified or contested, and several accepted papers contained confirmed errors.","og:url":"https:\/\/www.tun.com\/home\/hugging-face-reproduced-2200-icml-papers-heres-what-they-found\/","article:published_time":"2026-08-17T14:51:00+00:00","article:modified_time":"2026-08-18T13:41:42+00:00","twitter:card":"summary_large_image","twitter:title":"Hugging Face Reproduced 2,200 ICML Papers. Here\u2019s What They Found - TUN","twitter:description":"Hugging Face organized a community hackathon in which over 1,200 participants used coding agents to reproduce papers from ICML 2026 \u2014 one of AI's biggest conferences. The results: 23% of examined papers had at least one claim falsified or contested, and several accepted papers contained confirmed errors."},"aioseo_meta_data":{"post_id":"39406","title":null,"description":null,"keywords":null,"keyphrases":{"focus":{"keyphrase":"","score":0,"analysis":{"keyphraseInTitle":{"score":0,"maxScore":9,"error":1}}},"additional":[]},"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":"","og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"BlogPosting","isEnabled":true},"graphs":[]},"schema_type":"default","schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":"-1","robots_max_videopreview":"-1","robots_max_imagepreview":"large","priority":null,"frequency":"default","local_seo":null,"breadcrumb_settings":null,"limit_modified_date":true,"ai":{"faqs":[],"keyPoints":[],"schemas":[],"titles":[],"descriptions":[],"socialPosts":{"email":[],"linkedin":[],"twitter":[],"facebook":[],"instagram":[]}},"created":"2026-08-18 13:26:39","updated":"2026-08-18 14:19:56","seo_analyzer_scan_date":null,"focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"aioseo_breadcrumb":"<div class=\"aioseo-breadcrumbs\"><span class=\"aioseo-breadcrumb\">\n\t\t\t<a href=\"https:\/\/www.tun.com\/home\/\" title=\"Home\">Home<\/a>\n\t\t<\/span><span class=\"aioseo-breadcrumb-separator\">\u00bb<\/span><span class=\"aioseo-breadcrumb\">\n\t\t\t<a href=\"https:\/\/www.tun.com\/home\/category\/ai\/\" title=\"AI\">AI<\/a>\n\t\t<\/span><span class=\"aioseo-breadcrumb-separator\">\u00bb<\/span><span class=\"aioseo-breadcrumb\">\n\t\t\tHugging Face Reproduced 2,200 ICML Papers. Here\u2019s What They Found\n\t\t<\/span><\/div>","aioseo_breadcrumb_json":[{"label":"Home","link":"https:\/\/www.tun.com\/home\/"},{"label":"AI","link":"https:\/\/www.tun.com\/home\/category\/ai\/"},{"label":"Hugging Face Reproduced 2,200 ICML Papers. Here&#8217;s What They Found","link":"https:\/\/www.tun.com\/home\/hugging-face-reproduced-2200-icml-papers-heres-what-they-found\/"}],"uagb_featured_image_src":{"full":false,"thumbnail":false,"medium":false,"medium_large":false,"large":false,"1536x1536":false,"2048x2048":false},"uagb_author_info":{"display_name":"The University Network","author_link":"https:\/\/www.tun.com\/home\/author\/funky_junkie\/"},"uagb_comment_info":0,"uagb_excerpt":"Hugging Face organized a community hackathon in which over 1,200 participants used coding agents to reproduce papers from ICML 2026 \u2014 one of AI's biggest conferences. The results: 23% of examined papers had at least one claim falsified or contested, and several accepted papers contained confirmed errors.","_links":{"self":[{"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/posts\/39406","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/comments?post=39406"}],"version-history":[{"count":11,"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/posts\/39406\/revisions"}],"predecessor-version":[{"id":39430,"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/posts\/39406\/revisions\/39430"}],"wp:attachment":[{"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/media?parent=39406"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/categories?post=39406"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.tun.com\/home\/wp-json\/wp\/v2\/tags?post=39406"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}