Coding Agent Leaderboard v0.1.3

Performance, efficiency, coverage, and reliability across coding-agent benchmarks. Each result is one model + harness run on one benchmark.

93benchmark results

19models

6harnesses

7benchmarks

90%token coverage

98%cost coverage

99%timing coverage

Cross-benchmark ordering uses within-benchmark percentiles rather than averaging incompatible raw score scales. Missing metrics remain missing and reduce coverage; they are never converted to zero.

Rankings

Paired comparisons are shown first, followed by benchmark-specific metric rankings.

Paired Comparisons

Rankings computed from head-to-head benchmark results using a Bradley–Terry paired-comparison model. Handles missing data and inconsistent orderings.

Rank by

Open models only

Open harnesses only

Benchmarks

RH SWE-Bench SWE-Bench Pro -- Ansible SWE-Bench Verified

Harness ranking


1	Claude Code	0.445	0.535	0.626


1	Claude Code	0.445	0.535	0.626
2	OpenCode	0.347	0.551	0.618
3	Pi	0.305	0.488	0.522
4	OpenClaw		0.406	0.588

Model ranking


10	Mistral-Small-4-119B-2603-NVFP4	0.633	0.823	0.813


1	Opus 4.6	0.633
2	Qwen3.8-27B		0.823	0.813
3	Qwen3.8-27B-FP8		0.819	0.806
4	Opus 4.8		0.74	0.851
5	GPT 5.5 - high		0.604	0.798
6	Sonnet 4.6	0.557	0.5	0.796
7	Qwen3.6-27B-FP8	0.467	0.528	0.677
8	Qwen3.6-35B-A3B-NVFP4		0.431	0.611
9	Gemma4-31B-FP8		0.434	0.597
10	Nemotron-3-Super-120B-NVFP4	0.249	0.377	0.46
11	GPT-OSS-120B	0.28	0.333	0.312
12	Mistral-Small-4-119B-2603-NVFP4		0.326	0.223

Model + harness ranking


10	Mistral-Small-4-119B-2603-NVFP4 + Claude Code	0.633	0.896	0.936


1	Qwen3.8-27B + OpenCode		0.896	0.94
2	Qwen3.8-27B-FP8 + OpenCode		0.875	0.936
3	Opus 4.6 + Claude Code	0.633
4	Qwen3.8-27B + Claude Code		0.865	0.868
5	Qwen3.8-27B-FP8 + Claude Code		0.823	0.822
6	Opus 4.8 + OpenCode		0.781	0.834
7	Opus 4.8 + Claude Code		0.698	0.868
8	Qwen3.8-27B-FP8 + Pi		0.76	0.66
9	Sonnet 4.6 + Claude Code	0.557	0.5	0.796
10	Qwen3.6-27B-FP8 + Claude Code	0.493	0.521	0.694
11	Qwen3.8-27B + Pi		0.708	0.632
12	Qwen3.6-27B-FP8 + Pi	0.468	0.49	0.694
13	Qwen3.6-27B-FP8 + OpenCode	0.44	0.573	0.642
14	Qwen3.6-35B-A3B-NVFP4 + Pi		0.479	0.65
15	Qwen3.6-35B-A3B-NVFP4 + Claude Code		0.458	0.632
16	Gemma4-31B-FP8 + Pi		0.469	0.574
17	Gemma4-31B-FP8 + Claude Code		0.417	0.612
18	Gemma4-31B-FP8 + OpenCode		0.417	0.606
19	Qwen3.6-35B-A3B-NVFP4 + OpenClaw		0.406	0.588
20	Nemotron-3-Super-120B-NVFP4 + Claude Code	0.224	0.432	0.472
21	Qwen3.6-35B-A3B-NVFP4 + OpenCode		0.375	0.548
22	GPT-OSS-120B + Claude Code	0.317	0.375	0.376
23	Mistral-Small-4-119B-2603-NVFP4 + OpenCode		0.385	0.318
24	Nemotron-3-Super-120B-NVFP4 + Pi	0.216	0.375	0.498
25	Nemotron-3-Super-120B-NVFP4 + OpenCode	0.308	0.323	0.41
26	GPT-OSS-120B + OpenCode	0.294	0.333	0.326
27	GPT-OSS-120B + Pi	0.23	0.292	0.234
28	Mistral-Small-4-119B-2603-NVFP4 + Pi		0.333	0.232
29	Mistral-Small-4-119B-2603-NVFP4 + Claude Code		0.26	0.118

Metric rankings

Use the shared controls below to choose a benchmark and display options for all metric charts. Tables are capped to a scrollable height so the visualizations stay primary.

Benchmark

Color by

Model Harness

Chart order

Color palette

Image background

Score

Score (%). Higher is better.

Token usage

Total tokens per task. Lower is better.

Cost

Cost per task (USD). Lower is better.

Response time

Total time per task (seconds). Lower is better.

Reliability

Execution error rate (%). Lower is better.

Tokens per successful task

Tokens per successful task. Lower is better.

Cost per successful task

Cost per successful task (USD). Lower is better.

Time per successful task

Time per successful task (seconds). Lower is better.

Ranking data

One table for the page, placed after all charts. It includes the score, resource, timing, reliability, and per-success values for the selected benchmark.

Table benchmark

Sort table by

Table order


RedHatAI/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4	Claude Code	SWE-Bench Verified	93.6	11	96.55	2450380	0.06	1193	1112	14.6	2606787	0.06	1374


Qwen/Qwen3.8-27B	OpenCode	SWE-Bench Verified	94	1	100	2450380	0.06	669	617	0.8	2606787	0.06	712
Qwen/Qwen3.8-27B-FP8	OpenCode	SWE-Bench Verified	93.6	2	96.55	2270268	0.05	706	639	0.8	2425500	0.05	754
Opus 4.8	Claude Code	SWE-Bench Verified	86.8	3	91.38	1591190	0.79	319	208	0	1833168	0.91	368
Qwen/Qwen3.8-27B	Claude Code	SWE-Bench Verified	86.8	3	91.38	3379852	0.11	1193	1112	2.4	3893839	0.13	1374
Opus 4.8	OpenCode	SWE-Bench Verified	83.4	5	86.21	1398234	0.64	343	229	0	1676540	0.77	411
Qwen/Qwen3.8-27B-FP8	Claude Code	SWE-Bench Verified	82.2	6	82.76	3063936	0.12	1659	1560	14.6	3727416	0.15	2018
GPT 5.5 - high	Codex	SWE-Bench Verified	79.8	7	79.31	1611158	0.89	283	185	0	2018995	1.12	355
Sonnet 4.6	Claude Code	SWE-Bench Verified	79.6	8	75.86	null	null	null	null	null	null	null	null
RedHatAI/Qwen3.6-27B-FP8	Claude Code	SWE-Bench Verified	69.4	9	70.69	2024199	0.17	822	457	0.6	2916713	0.24	1184
RedHatAI/Qwen3.6-27B-FP8	Pi	SWE-Bench Verified	69.4	9	70.69	1098072	0.1	817	261	1.4	1582236	0.14	1177
Qwen/Qwen3.8-27B-FP8	Pi	SWE-Bench Verified	66	11	65.52	1906310	0.06	848	783	2	2888348	0.09	1285
RedHatAI/Qwen3.6-35B-A3B-NVFP4	Pi	SWE-Bench Verified	65	12	62.07	1595035	0.08	437	309	1.2	2453900	0.12	672
RedHatAI/Qwen3.6-27B-FP8	OpenCode	SWE-Bench Verified	64.2	13	58.62	797177	0.07	749	185	1.6	1241709	0.11	1167
RedHatAI/Qwen3.6-35B-A3B-NVFP4	Qwen Code	SWE-Bench Verified	63.8	14	55.17	1227106	0.07	357	264	0.6	1923364	0.11	560
Qwen/Qwen3.8-27B	Pi	SWE-Bench Verified	63.2	15	50	1858036	0.12	1348	1236	2.2	2939930	0.19	2133
RedHatAI/Qwen3.6-35B-A3B-NVFP4	Claude Code	SWE-Bench Verified	63.2	15	50	2224704	0.07	343	245	0.2	3520101	0.11	543
RedHatAI/gemma-4-31B-it-FP8-block	Claude Code	SWE-Bench Verified	61.2	17	44.83	1311796	0.21	1338	1049	7	2143458	0.34	2186
RedHatAI/gemma-4-31B-it-FP8-block	OpenCode	SWE-Bench Verified	60.6	18	41.38	1059493	0.11	889	332	2.4	1748338	0.18	1467
RedHatAI/Qwen3.6-35B-A3B-NVFP4	OpenClaw	SWE-Bench Verified	58.8	19	37.93	null	0.07	400	240	0.6	null	0.12	680
RedHatAI/gemma-4-31B-it-FP8-block	Pi	SWE-Bench Verified	57.4	20	34.48	763314	0.08	770	244	0	1329815	0.14	1341
RedHatAI/Qwen3.6-35B-A3B-NVFP4	OpenCode	SWE-Bench Verified	54.8	21	31.03	949488	0.06	370	240	0.8	1732642	0.11	675
RedHatAI/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4	Pi	SWE-Bench Verified	49.8	22	27.59	2008071	0.07	893	288	1.2	4032271	0.14	1793
RedHatAI/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4	Claude Code	SWE-Bench Verified	47.2	23	24.14	1778039	0.09	821	348	0	3767032	0.19	1739
RedHatAI/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4	OpenCode	SWE-Bench Verified	41	24	20.69	1461850	0.04	1037	186	0.2	3565488	0.1	2529
RedHatAI/gpt-oss-120b	Claude Code	SWE-Bench Verified	37.6	25	17.24	1678615	0.01	401	301	2.6	4464402	0.03	1066
RedHatAI/gpt-oss-120b	OpenCode	SWE-Bench Verified	32.6	26	13.79	408602	0.01	644	70	6.6	1253380	0.03	1975
RedHatAI/Mistral-Small-4-119B-2603-NVFP4	OpenCode	SWE-Bench Verified	31.8	27	10.34	755531	0.03	1086	98	1.2	2375884	0.09	3415
RedHatAI/gpt-oss-120b	Pi	SWE-Bench Verified	23.4	28	6.9	182024	0.03	352	293	9.4	777880	0.13	1504
RedHatAI/Mistral-Small-4-119B-2603-NVFP4	Pi	SWE-Bench Verified	23.2	29	3.45	395947	0.02	917	58	2.4	1706668	0.09	3953
RedHatAI/Mistral-Small-4-119B-2603-NVFP4	Claude Code	SWE-Bench Verified	11.8	30	0	624547	0.02	1111	108	10.6	5292771	0.17	9415