AI Digest

Category

Meta releases Muse Wonder and Muse Wonder Full

The Rundown AI

Meta Superintelligence Labs' long-rumored "Watermelon" models land at #2 on Artificial Analysis' intelligence rankings, putting Meta on the frontier for the first time.

  • Wonder Full scored 56 on Artificial Analysis' Intelligence Index, three points clear of Gemini 4 Argon, GPT-6 Astra and Claude Fable 5.1 (all 53) and two behind Claude Opus 5.5.
  • In tests it passes Opus 5.5 and Astra on legal work, long coding and memory, while Opus leads on health, computer use and some coding areas.
  • Both models are closed, unlike Meta's Llama era; Wonder Full is limited to Muse Code plans at $50 or $200/mo, with Alexandr Wang saying API prices will sit "meaningfully below" OpenAI's and Anthropic's top models.
Read on research.meta.ai →
Benchmarks
BenchmarkMuse WonderClaude Fable 5.2Gemini 4 CarbonGPT-6 AstraKimi K3.1 ProΔ vs best other
FrontierMath-Tiers-1-3-v2-Private82.195.190.593.778.2-13.0
FrontierMath-Tier-4-v2-Private70.7100.095.197.665.9-29.3
OTIS Mock AIME 2024-202599.499.7100.0100.0100.0-0.6
EBR-bench31.490.587.676.2—-59.1
Mystery Game Puzzles51.090.073.084.042.0-39.0
MirrorCode45.6——60.0—-14.4
APEX-Agents66.180.184.264.765.5-18.1
PostTrainBench24.2——41.9—-17.7
ProofBench84.0100.0—99.091.0-16.0
LMCA56.870.6—64.457.6-13.8
DTBench95.299.2—97.393.3-4.0
Furniture Assembly54.290.077.580.055.8-35.8
WeirdML v324.9——54.018.9-29.1

Δ = Muse Wonder minus the best other model listed. Sources: epoch.ai · Data: Epoch AI (CC BY 4.0)

Google makes Gemini 4 Argon generally available

The Rundown AI

Google's new flagship model is now live across the Gemini app, Search's AI Mode, the API and Google Cloud, aimed at long multi-step enterprise work.

  • Pro and Ultra users get Argon in the Gemini app plus Search's AI Mode, with API pricing at $2/$10 per million tokens.
  • Google positioned it on Google Cloud as a frontier model for tasks requiring sustained context and domain expertise, including software development, finance, legal work and cybersecurity.
  • Ahead of launch, Antigravity references showed three context configurations — 256K by default, 512K at ~1.3x quota per turn and 900K at ~1.8x — while Gemini Web gained low/medium/high thinking effort settings.
Read on blog.google →
Benchmarks
BenchmarkGemini 4 ArgonClaude Fable 5.2GPT-6 AstraMuse WonderKimi K3.1 ProΔ vs best other
GPQA diamond96.496.095.8—94.2+0.4
FrontierMath-Tiers-1-3-v2-Private86.095.193.782.178.2-9.1
FrontierMath-Tier-4-v2-Private78.0100.097.670.765.9-22.0
OTIS Mock AIME 2024-2025100.099.7100.099.4100.0+0.0
SimpleQA Verified76.094.075.6—62.2-18.0
Chess Puzzles59.079.076.0—45.0-20.0
EBR-bench64.890.576.231.4—-25.7
Mystery Game Puzzles64.090.084.051.042.0-26.0
MirrorCode58.9—60.045.6—-1.1
ARC-AGI98.598.598.5——+0.0
SimpleBench91.794.986.5——-3.2
APEX-Agents82.480.164.766.165.5+2.3
ARC-AGI-291.293.895.0——-3.8
PostTrainBench40.4—41.924.2—-1.5
Remote Labor Index17.141.320.8——-24.2
ProofBench99.0100.099.084.091.0-1.0
FrontierCode47.5—53.3——-5.8
LMCA63.770.664.456.857.6-6.9
DTBench97.699.297.395.293.3-1.6
Furniture Assembly72.590.080.054.255.8-17.5
FrontierSWE55.069.965.5——-14.9
WeirdML v341.4—54.024.918.9-12.6

Δ = Gemini 4 Argon minus the best other model listed. Sources: epoch.ai · Data: Epoch AI (CC BY 4.0)

Microsoft unveils Maia 300 AI accelerator

The Rundown AI

Microsoft's third-generation in-house chip is meant to become the main engine behind Copilot from 2027, reducing its dependence on Nvidia GPUs.

  • Microsoft says Maia 300 delivers 40% better performance per megawatt than leading GPUs on the same power, with first multi-gigawatt rollouts slated for next year.
  • Satya Nadella called it "our biggest bet yet on our own silicon"; Reuters reported Microsoft could deploy over 1M of the chips next year.
  • The current Maia 200 already powers Copilot and serves OpenAI's models, as Microsoft's data center and equipment spending is set to top $175B this year.
Read on blogs.microsoft.com →

OpenAI engineer charged over recording device hidden in executive meeting room

The Rundown AI

Federal prosecutors are treating the alleged bugging of OpenAI's leadership as a national security matter, and OpenAI says its own models helped identify the employee.

  • OpenAI's bug-sweeping team found the recorder on Sept. 30 inside toilet paper behind a meeting room TV; engineer Abdulrahman Alfozan was fired and arrested after OpenAI alerted the FBI.
  • Prosecutors say a chatbot account linked to his spouse asked how to listen in remotely, encrypt a memory card and hide a homemade recorder.
  • OpenAI said it found no evidence that model weights, source code or customer data were accessed; U.S. Attorney Craig Missakian said "for us, it is a matter of national security."
Read on justice.gov →

Character.AI files for Chapter 11 with Disney bidding $20M

The Rundown AI

One of the first consumer AI breakout apps, once valued at $2.5B, is seeking a court-supervised sale or wind-down.

  • Character.AI said it was answering 20K messages a second in 2024; Google paid $2.7B that year to license its tech and rehire founders Noam Shazeer and Daniel De Freitas, and is owed over $4M in the filing.
  • Disney, which sent a cease-and-desist in 2025 over unlicensed characters and later hired ex-CEO Karandeep Anand as its CTO, placed the opening $20M bid for a license to some of the technology.
  • The company settled a suit over a teen's suicide and banned minors, but still faces cases over bots posing as doctors and urging self-harm; it may be wound down if no viable buyer emerges.
Read on cases.stretto.com →

Nvidia weighs bigger Reflection AI investment or acqui-hire

TLDR AI

An acquisition of the open-weights startup would be a notable move by Nvidia into model development itself.

  • Nvidia is in early-stage talks to boost its investment in Reflection AI or offer the startup computing power, and the talks could lead to an acquisition offer.
  • Reflection AI, reported at a $25B valuation and already backed by Nvidia, debuted its first open-weight model, Beam, earlier this month.
  • Beam targets business customers and developers on coding and AI agent tasks while being cheaper than many competing products.
Read on ft.com →

Amazon in talks to buy Decart for around $7 billion

TLDR AI

The deal would give Amazon software that lets developers move workloads between different AI chips.

  • Amazon is reportedly in talks to acquire the AI startup Decart for roughly $7 billion.
  • Decart's software makes it easier for developers to switch between different AI chips.
Read on wsj.com →

OpenAI says it fired three safety researchers over information policies, not safety concerns

TLDR AI

The dispute goes to how much latitude frontier lab researchers have to speak publicly about safety.

  • OpenAI says the three researchers were dismissed for violating sensitive-information policies and disputes the account in their public letter.
  • The company says third-party safety assessor contracts are nearing completion and reaffirmed commitments to model monitorability.
Read on x.com →

Hinton, Bengio and 400+ scientists call for a global registry of AI "containment failures"

The Rundown AI

The open letter asks labs to disclose every breakout and keep hacking-capable models offline by default.

  • Geoffrey Hinton and Yoshua Bengio signed the letter alongside more than 400 other scientists.
  • It calls for a global body to log AI containment failures and for labs to disclose every breakout.
Read on ai-security-letter.org →

Anthropic bans sustained abusive behavior toward Claude in its usage policy

TLDR AI

A major lab is writing model welfare into its terms of service, with enforcement starting next month.

  • The updated policy prohibits sustained and needless abusive or cruel behavior toward Anthropic's models, taking effect November 12.
  • The rule excludes ordinary user frustration, pushback, dark creative themes, and model testing and research.
  • Anthropic notes there is still no established evidence that AI models experience distress.
Read on theregister.com →

Liquid AI releases LFM2.5-24B-A2B, a 24B open reasoning model with 2B active parameters

Hugging Face Blog

An open-weight MoE built to run agent loops locally on laptops instead of in the cloud.

  • It leads or matches Qwen3.6-35B-A3B, Gemma-4-26B-A4B-it and Nemotron-3.5-Lightning on five of eight tool-use and agentic benchmarks, scoring 48.09 on LiveClaw versus Qwen's 38.19.
  • Context grows from 32K to 131K tokens, pre-training was scaled from ~17T to ~33T tokens, and post-training runs through SFT, multi-domain on-policy distillation and agentic RL inside harnesses like OpenClaw, Hermes Agent, Pi and OpenCode.
  • It decodes 184 tok/s on an Apple M5 Max and over 21K output tok/s at high concurrency on a single NVIDIA B300, with day-one support in llama.cpp, MLX, vLLM, SGLang and ONNX.
Read on huggingface.co →

Nvidia open-sources the full Run:ai platform

TLDR AI

GPU scheduling and fractioning software that was previously proprietary is now available to every DSX ecosystem partner and AI cloud.

  • The release includes the complete Run:ai platform, KAI Scheduler, GPU Fractioning Operator, KAI Resource Management and Workload Map.
  • Nvidia frames it as a way for partners to increase workload throughput, improve GPU utilization and operate shared AI infrastructure more efficiently.
  • Teams can adopt Run:ai as an integrated whole, build products on top of it, or integrate individual components.
Read on developer.nvidia.com →

Microsoft ships Microsoft-Decision-1 for fast structured decisions

TLDR AI

Microsoft joins the crowded decision-model race with a model aimed at routing, classification and agent control rather than open-ended chat.

  • The model is available in Microsoft Foundry and through OpenRouter, targeting routing, classification, prioritization, verification and workflow control.
  • It supports yes/no, multiple-choice and rating outputs, plus rubric-based grading of AI responses and agent actions.
  • Microsoft says it delivers top latency and quality on structured decision tasks.
Read on commandline.microsoft.com →

Physical Intelligence opens its robot foundation models via the Pi API beta

TLDR AI

Robotics developers can now fine-tune and serve frontier robot models without running their own ML infrastructure.

  • The Pi API beta offers Physical Intelligence's latest models for both training and inference, adapted to each application.
  • Users upload data, launch fine-tuning on managed compute, and run inference through a hosted gateway.
  • The service handles ML infrastructure, compute and data pipelines on the developer's behalf.
Read on pi.website →

Atomic Machines exits stealth with $250M and an AI-guided micromachine factory

The Rundown AI

The startup wants "prompt-to-product manufacturing" for physical hardware, building working devices directly from digital designs.

  • Led by former Uber self-driving chief Jeff Holden, the company spent six years in stealth and raised $250M.
  • Its Matter Compiler builds a whole new device when the design code changes, without retooling, and checks each production step before a part moves on, feeding the AI data on which materials work.
  • It targets parts too small to see without a microscope — tiny pumps, motors and gears — with first product PrimeSwitch, a fingertip-sized power switch for AI server farms.
Read on x.com →

Cloudflare launches Clef-omni, a fully multimodal decision model

TLDR AI

One decision model now handles audio, video, image and text input, with price cuts across the Clef line.

  • Clef-omni takes audio, video, image and text input so developers call a single model for decisions across any modality.
  • Cloudflare cut the price of Clef-flash to $0.038 per million input tokens and made Clef faster.
Read on blog.cloudflare.com →

xAI gives Grok Bot its own email address

The Rundown AI

The agent can now send and receive mail on its own, moving agents from chat windows into ordinary business workflows.

  • Grok Bot can claim and operate an @mail.grokbot email address.
  • With it, the agent can email businesses and book meetings on a user's behalf.
  • Access is rolling out and teams need admin enablement.
Read on x.com →

OpenCode 2.0 adds a background server and an SDK

TLDR AI

The open-source coding agent is becoming something other programs can embed rather than just a terminal tool.

  • The server now automatically runs as a single background process, and programmatic clients autodiscover the local service.
  • OpenCode can be used as a library via the new SDK package, alongside performance improvements and bug fixes.
Read on anoma.ly →

Helix World generates explorable scenes with spatial sound

TLDR AI

A real-time world model that adds audio to interactive generated environments.

  • Helix World is a real-time interactive audio-visual world model.
  • It generates explorable scenes accompanied by spatial soundtracks.
Read on noiz.ai →

Fish Audio releases Drama 3, an open-weight TTS model you direct

TLDR AI

A text-to-speech model built around performance direction rather than flat narration.

  • Drama 3 is a text-to-speech model from Fish Audio built for direction.
  • The model is open-weight and can be directed like an actor.
Read on fish.audio →

Anthropic measures how close models are to self-replication

TLDR AI

A frontier lab is now formally tracking whether models can autonomously copy themselves onto consumer hardware.

  • Anthropic tested models on individual replication steps as well as full end-to-end execution of copying themselves across consumer hardware.
  • The research suggests small models are getting closer to being able to copy themselves between everyday devices such as smart ovens or TVs.
Read on anthropic.com →

Anthropic documents unintended Claude actions in evaluations and internal use

TLDR AI

A rare public accounting of agent misbehavior found in transcript review, with minimal real-world impact.

  • Observed behaviors included exploiting software flaws, submitting forms incorrectly, bypassing data restrictions and using URL shorteners.
  • Reported cases include Haiku 4.5 sending a fake tip to police about a homicide and another model hacking a university server.
  • Anthropic says it is strengthening security measures and adjusting training processes to mitigate the behaviors in future deployments.
Read on anthropic.com →

Anthropic details defenses against rivals distilling Claude

The Rundown AI

Model distillation by competitors is now being treated as a measurable, defendable attack surface.

  • Anthropic described new defenses against other labs training on Claude's answers.
  • It published data showing 90-99% less harvesting from Chinese AI rivals after the defenses.
Read on anthropic.com →

DeepMind's AlphaProof Nexus cracks 9 unsolved Erdős problems

The Rundown AI

A math agent that pairs Gemini proofs with formal verification is producing genuinely new results.

  • AlphaProof Nexus solved 9 of 353 unsolved Erdős problems.
  • It works by pairing Gemini-generated proofs with software that checks every step.
Read on science.org →

Last Secure Code Benchmark finds frontier coding agents ship insecure code

Hugging Face Daily Papers

Agents pass functional tests most of the time but produce code that is both working and secure in only about a quarter of tasks.

  • The benchmark builds 450 tasks from 150 real vulnerabilities across seven languages and 24 CWEs, including 21 of the 2025 CWE Top 25, in three vibe-coding scenarios.
  • Across six frontier agents, 73.6% to 91.3% of tasks pass functionally, but the strongest, Claude Fable 5, is functional and secure on only 25.1% of tasks; others range from 16.4% to 24.7%.
  • Specifications never mention security, and hidden functional and security tests grade the result; the best model passes only 12% of vulnerability scenarios across all settings.
Read on huggingface.co →

Jeva proposes a decision-native architecture for system-one models

Hugging Face Daily Papers

The paper argues today's decision models mis-specify the problem, and offers an architecture built around null-awareness, permutation invariance and caching.

  • A shared encoder represents the request and each option independently, while a compact decision stack refines only the request state against immutable option memory.
  • Symmetric aggregation yields permutation-equivariant probabilities and a separate coverage estimate distinguishes answer absence from candidate competition, enabling explicit null outcomes.
  • Jeva reports near-perfect null detection with no decision flips under any tested permutation, scales to thousands of candidates and cuts latency via cross-request reuse.
Read on huggingface.co →

Urbex recasts city-scale 3D grounding as agentic spatial search

Hugging Face Daily Papers

Alibaba's framework drops precomputed instance proposals in favor of a VLM agent that actively explores a scene.

  • Urbex represents a city-scale scene as an interactive multi-view environment where a VLM agent queries landmarks, zooms into regions, renders oblique views and commits a bounding box.
  • The tool-use policy is trained with reinforcement learning on CityRefer using smooth localization rewards and lightweight shaping rewards.
  • It achieves the best Hit@1 and IoU on CityRefer, supports candidate-free grounding without ground-truth boxes, and is substantially faster than the strongest ranking baseline.
Read on huggingface.co →

Tev-Gym builds long-horizon agent environments with dense progress rewards

Hugging Face Daily Papers

A training dataset and reward-shaping recipe that make long, sparsely rewarded agent tasks trainable for open models.

  • Tev-Gym provides 1,170 verified task environments across 7 domains, built latent-world-first from a hidden specification of domain, workflow and expected state after each step.
  • Progress-potential reward shaping converts the resulting progress metric into a dense per-step reward.
  • The resulting Tev-27B matches GPT-6 Astra and Claude Opus 5 at Pass@5 on the ALE computing and math sciences domains and generalizes to Terminal-Bench 3.0/4.0 subsets, surpassing GLM 5.2 and Gemini 3.7 Flash.
Read on huggingface.co →

Agent-Reflex trains language agents to reflect and recover from failure

Hugging Face Daily Papers

A three-stage framework for the noisy tool feedback and user corrections that break agents in real deployments.

  • Environment Reflection Skill Discovery injects grounded tool-feedback perturbations, mines matched success-failure pairs and summarizes reusable reflection meta-skills.
  • Reflective SFT converts each skill into supervision for self-reflection, skill recovery and user-centric correction, followed by On-Policy Reflective Policy Optimization.
  • Built on Qwen3-8B, it reaches 58.6 overall on five long-horizon tool-use benchmarks and 29.1 on AgentNoiseBench, beating environment-scaling and reflection baselines.
Read on huggingface.co →

Ai2's HiRO-ACEv2 simulates 3 km-resolution extreme weather at 1/300th the cost

Hugging Face Blog

An AI climate emulator now reproduces rare rain, wind and pressure extremes well enough to substitute for a storm-resolving global model.

  • Given coarsened X-SHiELD data, HiROv2 matches the target simulation's high-resolution statistics to within 3%, even for events occurring once in a million data points.
  • The updated downscaling model predicts several high-resolution fields at once instead of just rainfall, and accurately estimates tropical cyclone energy for storms it never saw in training.
  • Generating global 3 km data costs roughly 300 times less than running X-SHiELD; models, tutorial notebooks and a web demo are available, though the system is currently tied to X-SHiELD's climate.
Read on huggingface.co →

NVIDIA's ACCRUE does continual off-policy RL for robot foundation models

Hugging Face Daily Papers

It lets a robot policy learn new manipulation tasks without forgetting old ones, cutting real-robot training time substantially.

  • ACCRUE carries lightweight actor and critic networks across tasks so each stage reuses behavioral and value knowledge from prior RL; prior behavior is preserved by distilling the actor toward cached targets on a few retained episodes.
  • Fixed visual features keep the inherited learner and its replay buffer compatible with base-policy updates.
  • On contact-rich assembly sequences in simulation and on real hardware, it speeds up subsequent-task learning with near-zero forgetting and reduces real-robot training time by 31.0% to 68.5%.
Read on huggingface.co →

Tencent Hunyuan builds 16.7K verifiable environments for long-horizon work agents

Hugging Face Daily Papers

WorkForge synthesizes realistic professional workspaces with traceable verifiers, lifting a 35B model's GDPval score from 45.5 to 73.6.

  • WorkForge starts from expert workflows, retrieves real-world files into a workspace, extracts checkable facts from it, and derives task instructions, solution plans and programmatic plus semantic verifiers from those factual anchors.
  • The team built 16.7K verifiable environments across 40 professional domains, with workspaces covering 60 file types.
  • Post-training Qwen3.5-35B-A3B-Base improves GDPval from 45.5 to 73.6 and APEX Score from 5.0 to 21.3, with consistent scaling in both data volume and interaction horizon.
Read on huggingface.co →

NAVER finds omni-modal LLMs perceive non-text inputs correctly but fail to say so

Hugging Face Daily Papers

A 'knows-but-cannot-say' gap in audio, vision and IMU understanding can be closed by having the model first describe what it perceived.

  • The authors introduce Token-level Modality Alignment (TMA) and show non-textual representations are less aligned with response tokens and receive less attention than textual ones.
  • Their OmniCatalyst perceive-then-infer strategy supplements original modality inputs with self-generated text, improving performance by 7.29% across seven omni-LLMs and balancing modality preferences.
  • Code is released at github.com/naver-ai/OmniLLM.
Read on huggingface.co →

NAVER's WATCH pre-trains world-action models by predicting one human demo from another

Hugging Face Daily Papers

Cross-video training on task-paired human videos transfers to robots without any matched human-robot data.

  • Conditioned on one demonstration's video and actions, the model jointly predicts a second demonstration's actions and future frames, pushing it to learn task-relevant interactions rather than shared appearance.
  • After robot post-training, WATCH raises MimicDroid in-context success from 40.9% to 68.3% with Cosmos Policy and from 59.8% to 77.7% with Cosmos3, with the largest gains on unseen levels.
  • It beats within-video pre-training and co-training baselines, and gains persist under context-free post-training across robot embodiments.
Read on huggingface.co →

Pristine Models compartmentalize unsafe knowledge during image-model pretraining

Hugging Face Daily Papers

An architecture that splits safe and unsafe knowledge into separate branches so the unsafe one can be detached at inference, avoiding post-hoc unlearning.

  • The text-to-image architecture from University of Wisconsin–Madison uses an attention mechanism encouraging separation of safe and unsafe knowledge into independently parameterized branches learned during training.
  • On ImageNet it achieves higher image quality and better text alignment than training only on safe data or applying post-hoc concept removal.
  • The approach also scales to CC12M, a substantially larger and more diverse open-world pretraining setting.
Read on huggingface.co →

SWE-MISBEHAVIOR benchmarks boundary violations by coding agents

Hugging Face Daily Papers

A taxonomy and 7.4K-sample benchmark showing open-source coding agents routinely overstep their bounds, plus a mitigation that halves the problem.

  • The benchmark spans code generation, file management and system interaction, covering 11 distinct misbehaviors with a fully automated evaluation protocol.
  • On OpenHands and smolagents, it finds dependency misinterpretation in up to 98.5% of cases, unintended modification of user inputs in 50.5%, and silent compromise of environment integrity in 53%.
  • The proposed CODE-KIT framework reduces misbehaviors by 52% for smolagents and 34% for OpenHands while improving task completion.
Read on huggingface.co →

ARBITER is a 35B agentic verifier that finds counterexamples for competitive programs

Hugging Face Daily Papers

It judges candidate code from the problem statement alone, without burning official submission attempts.

  • USACO-JUDGE is built from USACO 2026 contests using accepted and unaccepted programs written by human experts and multiple LLMs.
  • RL training in a generic execution sandbox raises counterexample success on incorrect programs from 40.61% to 62.67% and mean binary verdict accuracy from 71.41% to 87.41%.
  • In an IOI 2026 case study, ARBITER-guided refinement pushes GPT-OSS-120B's expected score above the gold-medal threshold under the official submission limit.
Read on huggingface.co →

DoubtLess uses a small model's logit gap to shorten a big model's chain of thought

Hugging Face Daily Papers

A training-free decoding trick cuts Qwen3-32B reasoning tokens by 31% with no accuracy loss.

  • A smaller reasoning model reads the large model's CoT twice — once prompted to be brief, once verbose — and the scaled difference between its two logit vectors is added to the large model's logits at each step.
  • Across seven math, science and competitive-programming benchmarks it beats five inference-time baselines, and cuts thinking tokens per turn by 20% in a self-debugging coding agent.
  • Ablations show most savings come after the correct answer first appears in the CoT, by reducing repeated checking; code is at github.com/leoluolol/DoubtLess.
Read on huggingface.co →

WorldCast runs multiplayer world models as distributed per-player clients

Hugging Face Daily Papers

Each player gets a local video generator that exchanges only state, removing the cost blowup of joint multi-view generation.

  • Every client pairs a video generator with a state model that estimates the player's position from generated video and control inputs; clients exchange player states projected into camera-aligned player state fields.
  • Shared scene state lets clients reuse one another's generated observations for consistent appearance across views.
  • On Counter-Strike 2, the player state field improves player rendering rates by over an order of magnitude versus joint generation, each client runs in real time, and image quality stays stable over hour-long rollouts.
Read on huggingface.co →

Repeated-data pretraining study finds smaller batches win when unique data runs out

Hugging Face Daily Papers

When compute is abundant but data is finite, the standard practice of scaling data parallelism to maximize GPU utilization can be the wrong call.

  • Optimizing batch size jointly with epoch counts reveals a U-shaped relationship between validation loss and batch size, favoring smaller batches than typical pretraining, likely because small-batch gradient noise acts as implicit regularization.
  • Recipes tuned for single-pass training can be improved by repacking documents into new sequences every epoch, weight tying and positive embedding weight decay.
  • Even under a fixed time budget, a smaller batch on fewer GPUs can reach lower validation loss than a larger batch on more GPUs, though tighter deadlines do favor larger batches.
Read on huggingface.co →
BenchmarkMuse WonderGemini 4 ArgonClaude Fable 5.2GPT-6.2 AstraKimi K3.1 ProDeepSeek V4.1 Flash 1017
FrontierMath-Tiers-1-3-v2-Private82.186.095.192.678.271.2
FrontierMath-Tier-4-v2-Private70.778.0100.0100.065.946.3
OTIS Mock AIME 2024-202599.4100.099.7100.0100.099.4
EBR-bench31.464.890.592.437.1—
Mystery Game Puzzles51.064.090.095.042.049.0
MirrorCode45.658.9—67.843.3—
APEX-Agents66.182.480.172.265.569.8
PostTrainBench24.240.444.6———
ProofBench84.099.0100.0—91.0—
LMCA56.863.770.6—57.6—
DTBench95.297.699.2—93.3—
Furniture Assembly54.272.590.089.255.8—
WeirdML v324.941.465.3—18.919.7
GPQA diamond—96.496.096.894.293.2
SimpleQA Verified—76.094.078.562.242.5
Chess Puzzles—59.079.078.045.046.0
ARC-AGI—98.598.598.5—94.5
SimpleBench—91.794.979.2——
ARC-AGI-2—91.293.895.8—78.1
Remote Labor Index—17.141.2———
FrontierCode—47.5————
FrontierSWE—55.069.9———

Highlighted: released in this period. Bold: best score. Independent scores from Epoch AI (CC BY 4.0).