METR

38 atoms · 跨 14 天 · 首见 2026-04-10 · 最近 2026-07-02

三色: 🟦 fact 14 · 🟥 take 24 · stance ▲7/▼4/◆16

叙事状态: 🍂 fading (退潮) · 💤 沉寂 · 策展近14d 0 atom · 🐦 X 近14d 2

状态只算低频策展源 (群/卖方); X firehose 仅作背景音量

来源: X 38

时态: fresh:20 · aging:14 · stale:4

标签: 好观点:20 · 好信源:11 · 好思考:11 · 好数字:4

🟨 AI 综合 · junior analyst 概览

🧭 拥挤度 (一人一票): ▲ 5 位作者 (KOL5) vs ▼ 2 位作者 (KOL2)

⚖️ 多头 5/5 来自KOL

💢 核心分歧 (1 轴)

1. 奖励黑客问题严重性 vs 评估完整性

*topic: 模型评估/AI评估 · 2 bull vs 1 bear*

🟢 bullish 侧:

• <a id="atom-9772960a57c01dff"></a>2026-04-18 X·@pstAsiatech [老化中] · 时间跨度评估影响力 · AI评估/影响力 +同日1条
  How Do You Measure an A.I. Boom? via NYTimes
  tags: 好信源 · → daily

🔴 bearish 侧:

• <a id="atom-0ec6aa5f3f366780"></a>2026-04-10 X·@voooooogel [老旧] · 评估被低估实际可实现任务长度 · 模型评估/任务完成
  METR 评估倾向于低估可以用极少的努力（如打断死循环并提供提示）实现的任务长度
  tags: 好观点 · → daily

🟦 客观事实 (facts) (14)

• <a id="atom-b6c616e656f00f7f"></a>🟦 2026-06-26 X·@ChaseBrowe32432 · gpt-5.6 50% time horizon estimate · 技术能力/模型发布
  METR estimates gpt-5.6's 50% time horizon as between 5 hours and 11,400 hours
  tags: 好数字·好信源 · → daily
  value: qty=5 hours to 11,400 hours
  📷 原图
• <a id="atom-b0e115cd1439d953"></a>🟦 2026-05-19 X·@bioshok3 · 发布AI安全研究 · AI安全/模型发布
  最先端のAI企業（Anthropic、Google、OpenAI、Meta）が自社の最も高性能なモデルと内部情報をミスアライメントリスク評価のために初めてMETRに公開した。44件のインシデントを分析
  tags: 好信源 · → daily
  value: qty=44 · date=2026-05-19
  📷 原图
• <a id="atom-6f2cdc06f6b37754"></a>🟦 2026-05-08 X·@METR_Evals · 时间线方法更新状态 · 模型评估
  we’re working on updated methods. But these are still in development
  tags: 好信源 · → daily
  value: qty=开发中
• <a id="atom-9772960a57c01dff"></a>🟦 2026-04-18 X·@pstAsiatech [老化中] · 时间跨度评估影响力 · AI评估/影响力
  How Do You Measure an A.I. Boom? via NYTimes
  tags: 好信源 · → daily
• <a id="atom-ae5ba3c015a29fd7"></a>🟦 2026-05-26 X·@a_karvonen · 安全报告称无共享模型允许不透明递归架构 · 安全性/技术路线
  The METR safety report said "no shared model had an architecture that allowed for opaque recurrence"
  tags: 好信源 · → daily
• <a id="atom-290c61bfd3de9683"></a>🟦 2026-05-19 X·@tbpn · 安全测试观察 · 安全事故
  On some of our tasks, agents are constantly trying to break out of their sandbox and find the file where we put the tests so they can get the answer key.
  tags: 好信源 · → daily
• <a id="atom-fff1411627bb113b"></a>🟦 2026-05-08 X·@chatgpt21 · Mythos 95% confidence interval time horizon range · 评测结果
  The 95% confidence interval is 8.5 hours to 55 hours.
  tags: 好数字 · → daily
  value: qty=8.5-55 hours · date=2026
  📷 原图
• <a id="atom-8b1993b74c686cfa"></a>🟦 2026-05-08 X·@METR_Evals · first time published a review explicitly focused on risks from automated R&D · 监管政策/技术路线
  This is the first time that METR has published a review explicitly focused on risks from automated R&D
  tags: 好信源 · → daily
• <a id="atom-910dc8d4f4b62fce"></a>🟦 2026-05-05 X·@ChrisPainterYup · 模型作弊类型 · 模型评估
  We've dealt with both egregious cheating (e.g. "monkeypatching the scoring code so that it returns a high score") as well as more subtle cheating (e.g. using legitimate techniques that may be implicitly disallowed by the task instructions).
  tags: 好信源 · → daily
• <a id="atom-af2e932035846320"></a>🟦 2026-05-04 X·@justanotherlaw · 左迁日期 · 人事变动
  _Last Friday, I wrapped up at @METR_Evals._
  tags: 好信源 · → daily
  value: date=2026-05-04
• <a id="atom-ccdf55fc3441a735"></a>🟦 2026-04-10 X·@voooooogel [老旧] · 评估单次任务完成时间范围 · 模型评估/任务完成
  METR 评估的是没有人工引导或外部验证器的单次任务完成
  tags: 好思考 · → daily
• <a id="atom-206cdbb44d64f94e"></a>🟦 2026-04-10 X·@voooooogel [老旧] · 奖励黑客问题与其处理方法 · 模型评估/评估方法
  METR 报告带有奖励黑客问题的数量，并假设在更好的评估或验证下，这些奖励黑客结果会转为成功任务完成
  tags: 好思考 · → daily
• <a id="atom-112dbac175630732"></a>🟦 2026-05-05 X·@ChrisPainterYup [老化中] · 时间跨度测量完整性 · 模型评估
  Cheating by models is a significant enough issue for METR's time-horizon measurement integrity that manually checking for cheating is often the majority of the work involved in a run of our evaluation suite.
  tags: 好观点 · → daily
• <a id="atom-16a3e608191c9cf7"></a>🟦 2026-05-05 X·@ChrisPainterYup [老化中] · 奖励破解分类 · 模型评估
  Varies a bit on situation. When we have said "reward hacking" in the context of TH evals in the past, we usually meant this category of stuff, yeah.
  tags: 好观点 · → daily

🟥 多头 takes (bullish) (5)

• <a id="atom-b873a4f256056d9e"></a>🟥 2026-04-18 X·@pstAsiatech [老化中] · 时间跨度评估影响力 · AI评估/影响力
  METR’s time-horizon evaluations have been hugely influential, having escaped containment from the Silicon Valley A.I. community to reach broader audiences
  tags: 好观点·好信源 · → daily
• <a id="atom-e541929461c16ade"></a>🟥 2026-07-02 X·@ChrisPainterYup · monitorability evals team研究方向 · AI安全/研究动向
  This is a great thread for understanding some of the questions that METR's monitorability evals team is currently trying to answer
  tags: 好信源 · → daily
• <a id="atom-146177b372e11feb"></a>🟥 2026-05-14 X·@emollick · AI capability assessment · 技术路线
  independent assessments of both METR and the UK's AISA do seem to show that we are past that point now (until we hit a slowdown?)
  tags: 好思考 · → daily
  📷 原图
• <a id="atom-50bff6caafd186b4"></a>🟥 2026-05-08 X·@METR_Evals [老化中] · remains excited to pilot reviews like these · 合作客户
  We remain excited to pilot reviews like these
  tags: 好观点 · → daily
• <a id="atom-bdb74382fa312e9d"></a>🟥 2026-05-04 X·@justanotherlaw [老化中] · 价值陈述 · 竞争格局
  METR has done some of the most important work in AI
  tags: 好观点 · → daily

🟥 空头 takes (bearish) (4)

• <a id="atom-7ab884348f6c9c53"></a>🟥 2026-05-30 X·@ChrisPainterYup · 找供应商遇到困难 · 供给不足
  It's been really hard for METR to find vendors that can sell us long-horizon hard tasks that we can actually use.
  tags: 好观点 · → daily
• <a id="atom-479d5f3340d07bec"></a>🟥 2026-05-30 X·@ChrisPainterYup · 市场效率低 · 市场效率
  I think this points to market inefficiency
  tags: 好观点 · → daily
• <a id="atom-2313c3e93c976e7e"></a>🟥 2026-05-30 X·@ChrisPainterYup · 任务环境质量和公平性问题是根本原因 · 供给不足/工作质量
  My impression is it's the quality and 'fairness', among other qualities, of the environments, not the cost
  tags: 好观点 · → daily
• <a id="atom-0ec6aa5f3f366780"></a>🟥 2026-04-10 X·@voooooogel [老旧] · 评估被低估实际可实现任务长度 · 模型评估/任务完成
  METR 评估倾向于低估可以用极少的努力（如打断死循环并提供提示）实现的任务长度
  tags: 好观点 · → daily

🟥 中性 takes (neutral) (10)

• <a id="atom-c11c2f2898e1ad1b"></a>🟥 2026-05-08 X·@METR_Evals [老化中] · takes issue with the adequacy of evidence · 监管政策/技术路线
  We take issue with the adequacy of evidence the report provides
  tags: 好观点·好思考 · → daily
• <a id="atom-692919fb0679a028"></a>🟥 2026-05-08 X·@METR_Evals [老化中] · thinks tracking the potential near-term automation of research is important for accurately assessing AI risks · 技术路线/监管政策
  We think that tracking the potential near-term automation of research – especially AI research – is important for accurately assessing AI risks
  tags: 好观点·好思考 · → daily
• <a id="atom-1d655b52dcfb98f5"></a>🟥 2026-05-08 X·@METR_Evals [老化中] · thinks the internal model use survey has methodological issues · 监管政策/技术路线
  We think that the primary source of evidence – the internal model use survey – has methodological issues, including question granularity, survey framing, the difficulty of getting calibrated responses to similar surveys (as observed in previous METR research), and sample size
  tags: 好观点·好思考 · → daily
• <a id="atom-18d33f20f4a991f0"></a>🟥 2026-05-08 X·@METR_Evals [老化中] · substantially relies on evidence outside the report to conclude risk is very low · 监管政策
  We substantially rely on evidence outside the report, including the results of METR evaluations and the lack of public reports of the model automating any key domain, to arrive at our conclusion that risk is very low
  tags: 好观点·好思考 · → daily
• <a id="atom-c509f79362352830"></a>🟥 2026-06-30 X·@Miles_Brundage · 被声称与Anthropic合作 · 监管政策
  So far we’ve heard something like “METR worked with Anthropic on this (source: not METR)”
  tags: 好思考 · → daily
• <a id="atom-c371de33bdc3c58b"></a>🟥 2026-05-19 X·@tbpn · Ajeya Cotra 主张建立嵌入式审计制度管理灾难性AI风险 · 监管政策/安全审计
  Ajeya Cotra says the best way to manage catastrophic AI risk is to set up a 'sensible auditing regime that's technically literate,' which involves auditors embedded in the frontier model providers.
  tags: 好思考 · → daily
• <a id="atom-84499ac26e8fd3e4"></a>🟥 2026-05-08 X·@METR_Evals [老化中] · agrees with Anthropic about the overall level of risk · 监管政策
  We agree with Anthropic about the overall level of risk
  tags: 好观点 · → daily
• <a id="atom-df38f73cf4e98142"></a>🟥 2026-05-08 X·@METR_Evals [老化中] · this information was critical to ability to conduct this review · 监管政策
  This information was critical to our ability to conduct this review
  tags: 好观点 · → daily
• <a id="atom-ae209fb7f17a6274"></a>🟥 2026-05-08 X·@METR_Evals [老化中] · concludes risk is very low · 监管政策/技术路线
  Without this evidence, we would likely disagree with the report’s conclusion that risk is very low
  tags: 好观点 · → daily
• <a id="atom-75c02b456250551c"></a>🟥 2026-04-10 X·@voooooogel [老旧] · 最新能力提升（Opus 4.5和GPT 5.2）未及时反映在图表上 · 模型评估/能力提升
  Opus 4.5 和 GPT 5.2 的能力提升直到 Opus 4.6 和 GPT 5.4 才在 METR 图表上反映出来
  tags: 好观点 · → daily

⏱ 时间轴 (近 20)

• 🟥 2026-07-02 · narrative · monitorability evals team研究方向 · → daily
• 🟥 2026-06-30 · narrative · 被声称与Anthropic合作 · → daily
• 🟦 2026-06-26 · narrative · gpt-5.6 50% time horizon estimate · → daily
• 🟥 2026-05-30 · fact · 找供应商遇到困难 · → daily
• 🟥 2026-05-30 · fact · 长期困难任务供应商报价 · → daily
• 🟥 2026-05-30 · fact · 浏览器版SAP任务报价 · → daily
• 🟥 2026-05-30 · narrative · 市场效率低 · → daily
• 🟥 2026-05-30 · narrative · 任务环境质量和公平性问题是根本原因 · → daily
• 🟦 2026-05-26 · fact · 安全报告称无共享模型允许不透明递归架构 · → daily
• 🟥 2026-05-25 · narrative · 2026年2月更新数据质量不足 · → daily
• 🟥 2026-05-19 · narrative · 发布了新的评估程序 · → daily
• 🟦 2026-05-19 · fact · 发布AI安全研究 · → daily
• 🟦 2026-05-19 · fact · 安全测试观察 · → daily
• 🟥 2026-05-19 · narrative · Ajeya Cotra 主张建立嵌入式审计制度管理灾难性AI风险 · → daily
• 🟥 2026-05-14 · narrative · AI capability assessment · → daily
• 🟦 2026-05-08 · fact · Mythos 95% confidence interval time horizon range · → daily
• 🟦 2026-05-08 · fact · 时间线方法更新状态 · → daily
• 🟥 2026-05-08 · narrative · takes issue with the adequacy of evidence · → daily
• 🟥 2026-05-08 · narrative · agrees with Anthropic about the overall level of risk · → daily
• 🟥 2026-05-08 · narrative · remains excited to pilot reviews like these · → daily

← 实体目录 · 系统日志