poolside

8 atoms · 跨 4 天 · 首见 2026-04-28 · 最近 2026-07-02

三色: 🟦 fact 7 · 🟥 take 1 · stance ▲0/▼1/◆2

来源: X 8

时态: fresh:7 · aging:1

标签: 好思考:4 · 好信源:3 · 好问题:1 · 好数字:1

叙事 (narrative) (1)

• 2026-04-28 [老化中] · 使用了 moe_router_logit_softcapping 参数
  _there is a "moe_router_logit_softcapping" parameter in the config set to 0 by default, but this is non-standard i think, means that they likely saw some instabilities in the router scores?_
  tags: 好思考·好问题 · → daily
  value: qty=0 · date=default

事实 (fact) (7)

• 2026-07-02 · Laguna XS 2.1 发布
  Today we’re releasing Laguna XS 2.1. It’s a small upgrade to the Laguna XS.2 model, the same 33B total / 3B active MoE and stronger results on multilingual coding and terminal-style tasks.
  tags: 好信源 · → daily
  value: qty=33B total / 3B active MoE · date=2026-07-02
• 2026-06-29 · forked from torchtitan
  ye and poolside also forked from torchtitan i believe
  tags: 好信源 · → daily
• 2026-06-29 · AI实验量
  100 experiments a week
  tags: 好数字 · → daily
  value: qty=100 · date=每周
• 2026-05-26 · 发布Laguna模型技术报告
  Today we’re publishing the technical report behind Laguna M.1 and Laguna XS.2.
  tags: 好信源 · → daily
  value: date=2026-05-26
• 2026-04-28 · 使用了 per head gated attention
  they use per head gated attention like qwen or stepfun, people usually use sigmoid, but they use softplus! they also use it on both swa and full attention which is nice!
  tags: 好思考 · → daily
• 2026-04-28 · SWA 层使用了更多 attention heads
  more attention heads on swa layers, xiaomi v2 flash did higher kv heads for swa layers
  tags: 好思考 · → daily
• 2026-04-28 · YARN 的超参 beta_fast 设为 64
  _they use a different beta_fast for yarn than most models (64 instead of 32) and partial (half) rope for the full attention layers_
  tags: 好思考 · → daily
  value: qty=64

时间轴 (近 20)

• 2026-07-02 · fact · Laguna XS 2.1 发布 · → daily
• 2026-06-29 · fact · forked from torchtitan · → daily
• 2026-06-29 · fact · AI实验量 · → daily
• 2026-05-26 · fact · 发布Laguna模型技术报告 · → daily
• 2026-04-28 · fact · 使用了 per head gated attention · → daily
• 2026-04-28 · narrative · 使用了 moe_router_logit_softcapping 参数 · → daily
• 2026-04-28 · fact · SWA 层使用了更多 attention heads · → daily
• 2026-04-28 · fact · YARN 的超参 beta_fast 设为 64 · → daily

← 实体目录 · 系统日志