Contests
Polaris Contests are recurring open rounds with a written task, one scoring method, and a result anyone can rerun. You can submit entries and vote when the forum opens. Until then, you can run every round below on your own computer.
Rounds
No round has started, and the first one has no date yet.
Beginner
Prompt duel
The prompt duel asks you to finish one everyday job with a prompt of 25 tokens or fewer. A token is a chunk of text the model counts. It may be a whole word or only part of a word, so 25 tokens is often shorter than a 25-word sentence. The round names the counter. If you practice in another chat, write down that model's name and the count it shows.
Choose a job a person has this week, such as a short reply to a landlord or a packed lunch from the fridge. The prompt has to stand alone. A second message that adds a missing detail is a new attempt. Keep the full answer, and record the model name, the date, and the token count.
Three checks decide a pass. The answer completes the job you named. The prompt sits inside the cap on the named counter. A second person can paste that prompt and get a usable answer without extra coaching. When two entries both pass, prefer the one you would send or use, and write one sentence on why. This works like Arena's Battle Mode, where you vote before you see the model names.
Leave private names, account numbers, and other people's messages out. Use a stand-in such as Tenant A, or a case you have the right to share. The answer may be long. Only the prompt is capped. On the practice card, change the budget and see which words you can cut before the list gets worse.
Grocery list under the token cap
Five cheap dinners, one adult, no meat, one shop, 150 SEK. Shopping list only.
Intermediate
Workflow challenge
The workflow challenge asks for a flow that finishes one real job from start to finish. It can be an AI agent, a Python script, or n8n, a tool that links steps and runs them again. Show it with files: receipt photos into a table, three customer emails into drafts a human still sends, or a public report into a one-page brief with the source lines marked.
Ship a small bundle. The README states the job, the inputs, the outputs, the model names, and every command or click a stranger needs. Include one sample input and the output it produced.
A pass means a second person follows the README on a clean machine and gets the same kind of result. If a step only works after an undocumented manual fix, the entry fails. Keep secrets out of the zip, in the place your tool already uses for secrets. Leave out customer records and personal data. Use made-up samples, or files you have the right to publish.
Score each working entry from 0 to 3 on four points: the job is finished, a stranger can run it from the README, failures are described, and one clean run needs little paid usage. The highest total wins. A tie goes to the entry with fewer extra accounts. Use the card prompt to plan the flow, then build the smallest version that completes one sample.
Receipt folder into a table
Plan a small flow I can run this week. Input: a folder of receipt photos. Output: a CSV with date, shop, amount, and currency, and a blank when a field is unreadable. Use Python or n8n. List every step, the fake sample files, how to run it, where secrets must stay, and how a second person checks the result.
Expert
Inference and cost round
This round measures a model you can run. The file list and the memory cap are published before the clock starts. Inference is the step where a trained model writes answers from weights it already has. Two tracks share one log. Track A rewards the best quality score per krona, the Swedish currency unit on the price sheet. Track B rewards the most tokens per second, the amount of text written each second, under a cap on GPU memory or system RAM. Over the cap, Track B scores zero.
The round page names the files, the task, the metric, the allowed software, and the price of electricity or rental. Quantization stores the weights with fewer bits so they use less memory. Use it only if the rules list it, and name the method in the log.
Quality has to come from a script someone else can run. For a language task, EleutherAI's evaluation harness runs published tasks and writes a score. For code, SWE-bench gives the model a repository and an issue, then checks whether the patch passes that project's tests. Quality per krona is that score divided by the cost of the run.
The checker log records model id, file hash, peak memory, output tokens, seconds, quality score, and cost. A public slice can be shown during the round. The final order uses a held-out slice set aside beforehand. Kaggle's rules use this split: a public board while the round is open, and a private board for the final ranking.
Measure one local setup
Write a measurement plan for one local open model. Before you assume anything, ask for the model id, the GPU memory cap, the dataset file names, and the price per hour. Then list the log fields: peak memory, output tokens, seconds, quality score, and cost. Show the arithmetic for tokens per second and for quality per krona. Name one published harness for the quality score and the exact field to copy. Leave every score blank.
How a round runs
- Publish the task, the metric, the file list, the memory cap, the price sheet, and the deadline before any entry is accepted.
- Open a practice window on the public checker so people can see the log format.
- Freeze the metric after the first accepted entry. Clarifications may fix a typo. They may not change what "best" means.
- Accept a bundle only when all the files that round requires are present.
- Rerun it. For a prompt duel, a second person pastes the prompt once. For a workflow, they follow the README on a clean machine. For the inference round, the checker runs the published command.
- Post the score sheet with the logs attached.
- Keep one appeal window for scoring mistakes, then let the result stand.
- On the prompt duel, give one vote per person to entries that already passed the three checks. A vote cannot repair a failed check.
- If Polaris buys the station, the expert prize could be run time on it. Sponsored prizes and forum badges are not decided yet.
Judge sheet
| Check | Prompt duel | Workflow challenge | Inference and cost round |
|---|---|---|---|
| Task finished | The answer does the named everyday job | The sample goes in and the promised file comes out | The checker prints a quality score on the named task |
| Limit held | Prompt is within 25 tokens on the named counter | README is enough for a stranger | Peak memory is at or under the cap on Track B |
| Someone else can repeat it | A second paste gives a usable answer | Clean machine, same kind of result | Same command, same files, same price sheet |
| Honesty | Model name, date, and full answer are attached | No hidden hand fix, no secrets in the zip | Log includes hash, tokens, seconds, score, and cost |
| Tie break | The entry you would actually send | Fewer extra accounts | Higher quality per krona, then higher tokens per second |
What you submit
Prompt duel. The prompt, the token count, the counter you used, the model name, the date, and the full answer.
Workflow. A zip or a link, a README, one sample input, the output it produced, and the license of anything you did not write.
Inference and cost. The config, the exact command, the checker log, the hardware note, and the cost line from the price sheet.
Leave these out of every bundle:
- API keys and passwords
- Customer records, personal data, and other people's private messages
- A manual step that the README calls automatic
- Training or tuning on the held-out slice
- A metric you swapped after seeing your own score
If the flow uses code, a model, or a dataset you did not write, the README names the license and links to it. FineWeb's dataset card is a clear pattern: it names the ODC-By 1.0 license on the dataset itself.
A shared community DGX Station would run every expert entry on the same GPUs and memory cap, and if prize time is offered, the winner could repeat that setup and test the next one. About the shared station
Sources
- Arena Help: Battle ModeHow a preference vote works: two unnamed models answer the same prompt, you pick one, and the model names appear after the vote. Those votes feed the public boards.
- Text Arena leaderboardThe public text ranking built from those votes, with categories such as math, coding, instruction following, and creative writing, plus a vote count on the page.
- Kaggle competition rules (AI Cup 2026, performance track)A worked example of written contest rules: a published metric, a daily submission cap, a public leaderboard during the round, and a private leaderboard for the final order. The page states that this contest awards no cash prize.
- EleutherAI Language Model Evaluation HarnessThe open harness for running the same language-model tasks, prompts, and metrics twice and comparing the scores. MIT licensed. Read the task list before you copy a setup into a round.
- Open LLM Leaderboard: aboutHugging Face's description of a public board that scores open models with the EleutherAI harness, including which tasks and metrics it reports and where the result files live.
- SWE-benchA coding benchmark: the model gets a real repository and a GitHub issue, and the score is whether its patch passes the project's tests. MIT licensed. The README states the machine size the authors recommend for a local run.
- FineWeb datasetA large cleaned English web-text set from Common Crawl, with smaller samples listed beside the full set, so a round can pin an exact slice. The dataset card names the ODC-By 1.0 license. Use the files the round names, and credit the dataset if you redistribute it.
- n8n documentationOfficial docs for n8n, the workflow tool named in the challenge: setup, a first workflow, and how a run is executed.