results

every browser-runnable model, ranked by how well it actually answered — scored on real MMLU questions, in real browsers, by real GPUs.

8B·4.9 GB download·~6.8 GB vram

community72%avg · n=28

your bestnot yet on this device

publishedno reference

accuracy by subject

science73%

math71%

engineering71%

cs70%

best run100%

median tok/s17.9

questions tested199

hardware mix

9× apple silicon8× nvidia6× intel5× amd

runs

4B·2.4 GB download·~3.5 GB vram

community71%avg · n=31

your bestnot yet on this device

publishedno reference

accuracy by subject

science76%

math74%

engineering71%

cs68%

best run100%

median tok/s34.3

questions tested171

hardware mix

12× apple silicon9× nvidia6× intel4× amd

runs

7B·4.2 GB download·~5.9 GB vram

community63%avg · n=40

your bestnot yet on this device

published63%Mistral 2023

accuracy by subject

cs76%

engineering70%

math66%

science59%

best run100%

median tok/s23.9

questions tested255

hardware mix

15× nvidia12× apple silicon7× amd6× intel

runs

3.8B·2.1 GB download·~3.0 GB vram

community61%avg · n=37

your bestnot yet on this device

published69%Microsoft 2024

accuracy by subject

math80%

engineering68%

science58%

cs41%

best run100%

median tok/s37.1

questions tested231

hardware mix

12× nvidia12× apple silicon7× amd6× intel

runs

3B·1.8 GB download·~2.5 GB vram

community55%avg · n=49

your bestnot yet on this device

published63%Meta 2024

accuracy by subject

science68%

math65%

cs55%

engineering50%

best run100%

median tok/s40.1

questions tested317

hardware mix

16× nvidia16× apple silicon9× amd8× intel

runs

2B·1.3 GB download·~1.8 GB vram

community50%avg · n=23

your bestnot yet on this device

published51%Google 2024

accuracy by subject

engineering54%

science50%

cs46%

math46%

best run100%

median tok/s55.0

questions tested141

hardware mix

8× nvidia8× apple silicon4× intel3× amd

runs

1B·720 MB download·~1.0 GB vram

community42%avg · n=61

your bestnot yet on this device

published49%Meta 2024

accuracy by subject

math51%

science48%

engineering48%

cs42%

best run100%

median tok/s74.0

questions tested381

hardware mix

21× apple silicon20× nvidia10× amd10× intel

runs

1.7B·1.1 GB download·~1.5 GB vram

community41%avg · n=44

your bestnot yet on this device

publishedno reference

accuracy by subject

engineering56%

science40%

cs38%

math33%

best run100%

median tok/s62.6

questions tested258

hardware mix

16× nvidia15× apple silicon7× amd6× intel

runs

0.6B·400 MB download·~600 MB vram

community39%avg · n=76

your bestnot yet on this device

publishedno reference

accuracy by subject

math38%

engineering38%

cs35%

science26%

best run80%

median tok/s96.0

questions tested429

hardware mix

28× apple silicon24× nvidia12× amd12× intel

runs

frontier · published mmlufor scale

—claude 3.5 sonnet89%anthropic 2024

—llama 3.1 70b86%meta 2024

—gpt-3.5 turbo70%openai 2023

how this is scored →run benchmark →