Model Index
Model Index compares AI models by the job, the price, the speed and the license, with each fact kept in its own column. The table lists real models released by October 2026 and the class of machine their published weights fit, from a laptop to a cloud API.
Beginner
Five columns, kept apart
A model is the program that reads what you type and writes the reply. Before you trust a ranking, separate three facts. The job: an ordinary conversation, writing or fixing code, or a question with a checkable answer. The cost: a fee for each token, a small chunk of text, or the electricity on a computer you already own. The license: the legal text for the weights, the saved numbers that make the model work. Open weights means you can download that file. A closed model stays on the maker's computers.
Keep five columns in view: best at conversation, best at code, cheapest to run, runs on a machine you control, and open license. Sort by best for the price when the budget is the limit. Sort by strongest on that job when a wrong answer is expensive. A model can lead the chat column and lag the code column. Leave the columns side by side. One combined percentage does not show how the model does on each job.
Context is how much text the model can hold at once, counted in thousands of tokens. A long article needs more than a one-line question. Small open models fit a laptop. Mid-size models fit a home PC with a strong graphics card. The largest open models need a shared workstation or a cloud service.
There are no member test logs yet. The model cards linked from the table still give the size, the context length and the license.
Pick a model for one job
I need help choosing an AI model. Ask me what I want to do, my budget, whether I can install software on my own computer, and whether I need the conversation in Swedish. Then recommend one option and say which of these it is strongest on: conversation, code, price, running on my own machine, and an open license. Keep those as separate points. Explain any term you use, in plain words, and name the official model card I should read.
Intermediate
What each public test actually measures
Public tests answer different questions, so a high score on one can sit beside a weak score on another. MMLU, from a 2020 paper, asks exam-style questions across many subjects. GPQA asks hard science questions written so a non-expert cannot look up the answer quickly. SWE-bench asks a model to fix real issues in real software repositories. LMArena, published as a dataset of crowd votes, compares chat replies by asking people which answer they prefer. MLPerf Inference, from MLCommons, measures how fast hardware serves a fixed model, including tokens per second under a stated scenario. Artificial Analysis publishes price, speed and quality charts for models sold through APIs. EleutherAI's lm-evaluation-harness is a program labs use so the same academic test can be run twice the same way.
Write down the model name, the exact weight file, the date and the hardware. A newer file from the same family can move the score. An English test says little about Swedish instructions until someone repeats it in Swedish. Price stays in its own column: the fee per million tokens, or the electricity and the machine if you run the weights yourself. Speed stays separate too. Time to first token is the wait before the first word. Tokens per second is how fast the rest of the answer arrives.
Polaris will keep its own log instead of pasting in a table from another site. Every future run gets its own row, with quality, price, speed and the test date stored apart. A shared DGX Station could run the largest open models for the tests on this page.
Read a benchmark without mixing the scores
I will paste a model name. Explain, separately, what MMLU, GPQA, SWE-bench and a chat preference vote each measure. Tell me which facts I must copy down before I trust a number: exact model file, date, hardware, price per million tokens, time to first token, and tokens per second. Point me to the official page for each test. Do not combine the scores into one grade.
Expert
A row is a set of measurements
Record a run as separate measurements. Quality is the score on a named test, with the prompt template, the sample count and the temperature beside it. Price is the API rate that day, or the energy and hardware cost of a local run. Speed splits into time to first token and tokens per second after that. Context stability repeats the same questions at short, medium and long prompts, including a length-extension setting such as YaRN. The advertised maximum is an upper limit. Test quality at the length you use. Confidence is the spread across repeats, stored next to the point score. Also store the test date, the weight-file name, the quantization and the test machine.
Quantization stores the weights with fewer bits. gpt-oss ships with MXFP4 on its expert weights, and DeepSeek-R1 publishes FP8 weights. A 4-bit file can fit a smaller machine than the 16-bit file, and a score from the larger file does not transfer on its own.
A mixture-of-experts model lists two sizes. DeepSeek-R1 has 671 billion parameters in total and 37 billion active on each token. Llama 4 Scout has 109 billion total and 17 billion active. Memory follows the total size. NVIDIA rates DGX Spark for models up to 200 billion parameters in 128 GB, and DGX Station for up to 1 trillion parameters in 748 GB of coherent memory, 252 GB of it fast HBM3e.
Polaris keeps these fields apart, with no single score out of 100. To find a cheap model for code, sort by the code test and the price side by side; for a strong chat model, sort by the conversation test and check the date.
Draft an empty test log
Write a blank test log for an open-weight model run. Include fields for quality on one named test, price, time to first token, tokens per second, context length used, whether quality held as the prompt grew, repeat spread, test date, weight file, quantization, and machine. Add one line on why these stay in separate columns. Use DeepSeek-R1 or Llama 4 Scout only as an example of total versus active parameters, and leave every score cell empty.
What the machine column means
The machine column is a class for the published weights, taken from the model card and from NVIDIA's memory guides. It is a starting label for where a file can live.
- laptop — about 16 GB is enough. OpenAI says gpt-oss-20b runs in 16 GB.
- home PC — one strong desktop graphics card. Mistral says a quantized Small 3.1 fits one RTX 4090, and the 16-bit weights need about 55 GB.
- DGX Spark — NVIDIA's guide: up to about 200 billion parameters in 128 GB of unified memory.
- DGX Station — NVIDIA's page: up to about 1 trillion parameters, with 748 GB of coherent memory (252 GB HBM3e beside the GPU and 496 GB LPDDR5X beside the CPU).
- cloud — the weights stay on the maker's service.
A quantized file, meaning weights stored with fewer bits, can move a model down a class. context_k is the context window in thousands of tokens: 128 is the usual 128k window (131,072 tokens), 1000 is 1 million, and 10000 is the 10 million tokens on Llama 4 Scout Instruct. Qwen's own cards for Qwen3-8B and Qwen3-235B-A22B state 32,768 tokens natively and 131,072 with YaRN, a length-extension setting. Qwen2.5-72B-Instruct carries the Qwen license named on its card. The 7B and Coder 32B cards use Apache 2.0. Llama and Gemma weights are downloadable only after you accept a community license with extra use rules. Parameter counts are totals. Mixture-of-experts models also have a smaller active count, listed on the card.
How a row will be filed
When a Polaris run is filed, one row stores these fields and nothing is averaged into a single grade.
- Model name, maker, and whether the weights are open
- Weight-file name and quantization
- Test name, date, and machine, including which memory held the weights
- Quality on that one test, with the prompt setup beside it
- Price that day, as an API rate or as measured energy
- Time to first token, and tokens per second
- Context length actually used, and whether quality held as the prompt grew
- The spread across repeats
This log has no member rows yet. Until then, read the maker's card and the public tests linked below, and keep each number in its own column.
A shared community DGX Station would let members run the open-weight models that need more memory than a laptop or a DGX Spark, and file each run with the date and the machine name. About the shared station
Sources
- Measuring Massive Multitask Language UnderstandingThe MMLU paper: exam-style questions across many subjects, and how the test is scored.
- GPQA datasetGraduate-level science questions written so a non-expert cannot look the answer up quickly.
- SWE-benchA test that asks a model to fix real issues in real software repositories.
- LMArena leaderboard datasetCrowd votes on which chat reply people prefer. Polaris does not copy the score table.
- MLPerf Inference v6.1 resultsMLCommons results for how fast hardware serves fixed models, including tokens per second.
- Artificial AnalysisSeparate charts of quality, price and speed for models sold through APIs.
- lm-evaluation-harnessEleutherAI's program for running academic tests the same way twice. MIT-licensed software.
- NVIDIA DGX StationOfficial specs: 748 GB coherent memory and support for models up to about 1 trillion parameters.