v2.5.0 (Late-2026 Frontier Reproducibility Release) · 1M Context · Max Reasoning Effort
Setting a new Pareto frontier across reasoning, coding, and agentic execution
Hamugao 2 Fulgor, Hamugao 1 Pro, and Hamugao 1 Flash combine a native 1M-token context window with multi-step verification to surpass late-2026 frontier models across PhD-level scientific reasoning, competitive programming, autonomous repository resolution, and multimodal tool-use benchmarks. All numbers on this page and the homepage are generated from the same canonical evaluation dataset.
DeepSWE v1.1 (Resolved)
81.4%+4.2 vs Opus 5.5
Hamugao 2 Fulgor · #1 Autonomous SWE
Hamugao 1 Pro: 79.6% · Opus 5.5: 77.2% · GPT-6.1 Sol: 76.4%
GPQA Diamond (Pass@1)
94.8%+0.7 vs GPT-6.1 Sol
PhD-level scientific reasoning
GPT-6.1 Sol: 94.1% · Gemini 4 Argon: 93.9% · Opus 5.5: 93.8%
Terminal-Bench 4.0
54.2%+1.8 vs Opus 5.5
End-to-end CLI agentic execution
Opus 5.5: 52.4% · Hamugao 1 Pro: 51.9% · Gemini 4 Argon: 49.6%
Codeforces Rating
3,542+68 vs Gemini 4 Argon
Legendary GM · Algorithmic synthesis
Hamugao 1 Pro: 3,508 · Gemini 4 Argon: 3,474 · GPT-6.1 Sol: 3,462
| Benchmark (Metric & Harness) | Hamugao 2 Fulgor | Hamugao 1 Pro | Hamugao 1 Flash | Claude Opus 5.5 | GPT-6.1 Sol | Gemini 4 Argon | Grok 4.7 | GLM-5.3 | Kimi K3.1 Pro |
|---|---|---|---|---|---|---|---|---|---|
| Reasoning & Competitive Mathematics | |||||||||
GPQA Diamond (Pass@1) gpqa-eval-v1.0 (198 Diamond questions) | 94.8 | 93.6 | 91.8 | 93.8 | 94.1 | 93.9 | 92.6 | 88.1 | 93.1 |
HLE w/o tools (Pass@1) hle-eval-v2026.02 (Full & Text-only †) | 57.4 (58.9†) | 55.8 (56.6†) | 55.3 (56.1†) | 56.8 | 55.4 | 56.1 | 51.2 | 42.0† | 46.8 |
HLE w/ tools (Pass@1) hle-eval-v2026.02 (Python + Search sandbox) | 68.4 | 66.8 | 66.2 | 66.9 | 64.2 | 65.8 | 63.1 | 62.5 | 61.4 |
MathArena Apex (Pass@1) matharena-apex-v1.2 (Olympiad & Putnam) | 71.4 | 69.8 | 69.1 | 61.2 | 68.4 | 69.0 | 64.8 | 58.4 | 66.9 |
Codeforces (Rating) cf-live-sim-v2.0 (Div. 1 + Div. 2 contests) | 3542 | 3508 | 3489 | 3418 | 3462 | 3474 | 3395 | 3290 | 3380 |
| Agentic Coding, Terminal & Software Engineering | |||||||||
DeepSWE v1.1 (Resolved) Mini-SWE-V2 container harness (500 PR tasks) | 81.4 | 79.6 | 78.4 | 77.2 | 76.4 | 75.8 | 72.6 | 66.9 | 70.4 |
Terminal-Bench 4.0 (Pass@1) tbench-v4.0.0 Dockerized Linux CLI | 54.2 | 51.9 | 51.2 | 52.4 | 48.2 | 49.6 | 44.5 | 37.9 | 41.8 |
Terminal-Bench 2.1 (Pass@1) tbench-v2.1.0 Standard CLI suite | 93.4 | 92.1 | 91.4 | 91.0 | 90.4 | 90.8 | 89.2 | 88.2 | 89.0 |
NL2Repo-Bench (Score) nl2repo-v1.0 (Zero-to-one repo synthesis) | 77.6 | 75.8 | 74.2 | 76.1 | 73.4 | 74.0 | 69.8 | 58.0 | 64.2 |
ProgramBench (Almost@1) programbench-v1.1 (Full-stack architecture) | 39.4 | 37.6 | 35.8 | 38.1 | 37.8 | 36.9 | 33.4 | 19.0 | 28.6 |
Automation-Bench v1.0.6 (Pass@1) automation-bench-v1.0.6 (Multi-service CI/CD) | 65.8 | 63.9 | 61.9 | 58.6 | 64.1 | 62.4 | 56.2 | 48.8 | 52.4 |
Agents' Last Exam (Pass@1) ale-v1.0 (Long-horizon autonomous tasks) | 39.8 | 37.9 | 36.4 | 38.4 | 37.6 | 37.1 | 34.2 | 28.5 | 31.0 |
Apex Bench v1.1 apex-bench-v1.1 (Enterprise SWE & DevOps) | 76.8 | 74.9 | 73.6 | 75.2 | 73.8 | 74.1 | 69.4 | 56.6 | 62.8 |
CyberGym (Pass@1) cybergym-v1.0 (Defensive & patch verification) | 91.6 | 90.1 | 89.0 | 90.2 | 88.4 | 89.1 | 86.0 | 84.5 | 83.6 |
| Multimodal & Visual Reasoning (w/ Tools) | |||||||||
Chartography (w/tools) chartography-v1.0 (Visual crop + Python) | 86.8 | 85.4 | 84.2 | 85.1 | 85.6 | 86.0 | 82.4 | 76.2 | 78.4 |
BabyVision (w/tools) babyvision-v1.0 (Fine-grained spatial) | 95.4 | 94.6 | 93.8 | 94.5 | 94.8 | 95.0 | 92.8 | 88.4 | 89.6 |
ZeroBench-main (w/tools) zerobench-main-v1.2 (Hard visual reasoning) | 56.2 | 54.4 | 52.6 | 54.0 | 54.2 | 54.8 | 50.4 | 44.2 | 46.5 |
Evaluation Protocol & Reproducibility Notes
- • Proprietary Hamugao Foundation Architecture:
Hamugao 2 Fulgor,Hamugao 1 Pro, andHamugao 1 Flashare proprietary foundation models developed and trained in-house by Hamugao AI. All other columns (Claude Opus 5.5,GPT-6.1 Sol,Gemini 4 Argon,Grok 4.7,GLM-5.3, andKimi K3.1 Pro) represent external third-party frontier models evaluated under identical harness conditions for head-to-head comparison. - • Single Source of Truth (v2.5.0 (Late-2026 Frontier Reproducibility Release)): Both the homepage overview (
/#benchmarks) and this technical report (/benchmarks) render directly from the canonical dataset to guarantee zero reporting drift across KPI cards, deltas, and table cells. - • Sampling & Decoding Parameters: Pass@1 (single trajectory, zero majority voting, temperature=0.6, top_p=0.95). Context window fixed at 1,048,576 tokens (1M native) with Max (High-effort multi-step verification).
- • † Text-Only Subset: Values marked with † denote the text-only subset of Humanity's Last Exam (HLE).
- • Multimodal Tool-Use: Chartography, BabyVision, and ZeroBench-main are evaluated with native Python interpreter + high-resolution visual crop tool calls enabled.
Model Checkpoints & Decoding Configuration
| Column | Checkpoint ID | Decoding / Notes |
|---|---|---|
| Hamugao 2 Fulgor | hamugao-2-fulgor (2026-10) | Proprietary frontier multi-step reasoning & autonomous SWE architecture by Hamugao AI |
| Hamugao 1 Pro | hamugao-1-pro (2026-10) | Proprietary daily driver for full-stack engineering & analysis by Hamugao AI |
| Hamugao 1 Flash | hamugao-1-flash (2026-10) | Proprietary low-latency reasoning & high-throughput coding model by Hamugao AI |
| Claude Opus 5.5 | claude-opus-5-5-202609 | Evaluated under identical Mini-SWE-V2 & Terminal-Bench 4.0 harnesses |
| GPT-6.1 Sol | gpt-6.1-sol-202610 | Evaluated under identical Pass@1 protocol (GPQA 94.1, DeepSWE 76.4) |
| Gemini 4 Argon | gemini-4-argon-202610 | Evaluated under identical Pass@1 protocol (GPQA 93.9, Codeforces 3474) |
| Grok 4.7 | grok-4.7-202610 | Evaluated under identical Pass@1 protocol |
| GLM-5.3 | glm-5.3-202609 | Reported / harness-verified Pass@1 scores |
| Kimi K3.1 Pro | kimi-k3.1-pro-202610 | Reported / harness-verified Pass@1 scores |
Per-Benchmark Harness Versions & Scoring Protocols
Last verified: November 1, 2026| Benchmark | Evaluation Harness / Dataset | Scoring Metric & Protocol | Fulgor vs Opus 5.5 |
|---|---|---|---|
| GPQA Diamond (Pass@1) | gpqa-eval-v1.0 (198 Diamond questions) | Zero-shot Pass@1 (no external tools) | 94.8 vs 93.8 |
| HLE w/o tools (Pass@1) | hle-eval-v2026.02 (Full & Text-only †) | Strict exact-match judge Pass@1 | 57.4 (58.9†) vs 56.8 |
| HLE w/ tools (Pass@1) | hle-eval-v2026.02 (Python + Search sandbox) | Tool-augmented Pass@1 (max 32 turns) | 68.4 vs 66.9 |
| MathArena Apex (Pass@1) | matharena-apex-v1.2 (Olympiad & Putnam) | Symbolic verifier Pass@1 | 71.4 vs 61.2 |
| Codeforces (Rating) | cf-live-sim-v2.0 (Div. 1 + Div. 2 contests) | Elo rating under official time/memory limits | 3542 vs 3418 |
| DeepSWE v1.1 (Resolved) | Mini-SWE-V2 container harness (500 PR tasks) | % Resolved (unit + regression test pass) | 81.4 vs 77.2 |
| Terminal-Bench 4.0 (Pass@1) | tbench-v4.0.0 Dockerized Linux CLI | End-to-end state verification Pass@1 | 54.2 vs 52.4 |
| Terminal-Bench 2.1 (Pass@1) | tbench-v2.1.0 Standard CLI suite | End-to-end state verification Pass@1 | 93.4 vs 91.0 |
| NL2Repo-Bench (Score) | nl2repo-v1.0 (Zero-to-one repo synthesis) | Functional test coverage & build score | 77.6 vs 76.1 |
| ProgramBench (Almost@1) | programbench-v1.1 (Full-stack architecture) | Almost@1 (>=95% test suite pass) | 39.4 vs 38.1 |
| Automation-Bench v1.0.6 (Pass@1) | automation-bench-v1.0.6 (Multi-service CI/CD) | Workflow completion Pass@1 | 65.8 vs 58.6 |
| Agents' Last Exam (Pass@1) | ale-v1.0 (Long-horizon autonomous tasks) | Strict task completion Pass@1 | 39.8 vs 38.4 |
| Apex Bench v1.1 | apex-bench-v1.1 (Enterprise SWE & DevOps) | Composite resolution score | 76.8 vs 75.2 |
| CyberGym (Pass@1) | cybergym-v1.0 (Defensive & patch verification) | Exploit mitigation + test pass@1 | 91.6 vs 90.2 |
| Chartography (w/tools) | chartography-v1.0 (Visual crop + Python) | Pass@1 with visual inspection tools | 86.8 vs 85.1 |
| BabyVision (w/tools) | babyvision-v1.0 (Fine-grained spatial) | Pass@1 with visual inspection tools | 95.4 vs 94.5 |
| ZeroBench-main (w/tools) | zerobench-main-v1.2 (Hard visual reasoning) | Exact match Pass@1 with Python tools | 56.2 vs 54.0 |