POLARISAI ACADEMY

The Stack

The Stack is four ready packages for autumn 2026: Browser, Desktop, Home PC and Shared machine. Each has a model, a memory budget and a way to measure it. Use the smallest package that does the job, and move up when the files, the speed or privacy need more.

Beginner

Browser first, then one model on the laptop

Start in the browser. Move up a package when the answer is too weak, the files must stay on your computer, or the model no longer fits in memory.

Browser means a chat site on a phone or any computer. You install nothing. The company that runs the site holds the model, and the free plan sets how much you can send that week. Write the task, the facts the model must stick to, and one example of a good answer. The free Hugging Face LLM Course runs in the browser and shows how a Transformer model is loaded and called.

Desktop means one model file on the laptop. Download Ollama for macOS, Windows, or Linux, and run ollama run gemma4:e2b. Ollama's quickstart says that download is about 7.2 GB and suggests 8 GB of free graphics memory, or unified memory on a Mac. A longer chat needs more. With less graphics memory, Ollama uses ordinary system memory and replies slow down. Programs on your machine call it at http://localhost:11434, including an OpenAI-style address at /v1/chat/completions. Keep one paid chat plan for tasks this small model still misses, and save working prompts in a text file.

The market and the job board open later; the commands on this page run on your own machine today.

Write one prompt you can reuse

You are helping me with one task.
Task: [one sentence].
Facts you must stick to: [list them, or write none].
If a fact is not in that list, say you do not have it.
Shape of a good answer: [paste one short example].
Now do the task.

Intermediate

A retrieval assistant on your own files

A retrieval assistant answers from your files. The usual name is RAG, retrieval-augmented generation. You split documents into chunks, turn each chunk into a list of numbers called an embedding, store those lists, and fetch the chunks closest to the question. The model writes from those chunks. Tell it to say when the chunks lack the fact.

Keep the files beside the model, on the Desktop or Home PC package. The Ollama library lists nomic-embed-text for embeddings. For chat on a laptop, keep gemma4:e2b, or use qwen3.5:9b, about 6.6–7.6 GB with a 256,000-token context on the October 2026 library page. On a 16 GB graphics card, gemma4:12b is about 7.7–8.0 GB, leaving room for a moderate cache. qwen3.5:27b and gemma4:26b are about 16–20 GB, so they want a 24 GB card or a Mac with unified memory above the file size. The cache sits on top of the weights.

Split on headings and paragraphs, and overlap one short paragraph so a boundary sentence is stored twice. Start near 400 to 800 tokens per chunk. Write ten questions you can already answer, and count how often the right passage is in the top five. Change the chunk size before you change the model. LocalAI's documentation lists retrieval among its features, and the runtime starts as one container: docker run -ti --name local-ai -p 8080:8080 localai/localai:latest. The Hugging Face agents course then shows how to let the model call that retriever as a tool.

Set the chunk size with a test you can score

I am building a retrieval assistant on my own computer. The documents are: [describe them, and paste one real page].
Split on headings and paragraphs, overlap one short paragraph, and start with a chunk size between 400 and 800 tokens.
Write 10 questions whose answers are on the page I pasted.
For each question, quote the sentence that contains the answer so I can check whether that passage comes back in the top 5.
If the page does not contain an answer, say so.

Expert

Memory, quantization, and the shared machine

Quantization stores each weight in fewer bits. vLLM documents this as less precision for a smaller footprint. Four-bit storage is 0.5 bytes per weight and 16-bit storage is 2 bytes, so 31 billion parameters are about 15.5 GB before embeddings, a vision encoder, and the key-value cache. The cache grows with context length and with replies running at the same time. In October 2026 Ollama lists gemma4:31b at 19–20 GB, a short-context fit for one 24 GB card, and qwen3.5:122b at 81 GB, which needs several consumer cards or a larger pool.

vLLM serves that file on Linux with Python 3.11 to 3.14. The command vllm serve opens an OpenAI-compatible API on port 8000. Tensor and pipeline parallelism split one model across several GPUs. The faster the link between the cards, the less time they spend waiting for each other. vLLM loads GPTQ, AWQ, GGUF, FP8, INT8, and INT4. A LoRA is a small adapter on a frozen base, and one base can carry many of them.

Keep time to first token separate from decode speed. Decode speed is output tokens divided by the seconds after the first token. Fix the prompt and the length, run three times, and keep the median. Rerun one lm-evaluation-harness task with the same flags when the model or the format changes. Multiply prices in LiteLLM's model_prices_and_context_window.json by your monthly tokens, and compare with watts times hours times your tariff. When weights plus cache exceed one home GPU, you need a larger machine. Store fine-tunes as LoRA files.

Fit a model to the memory you have

Decide whether this model fits the machine I describe. Use only the numbers I give, and mark every assumption.
Model file size: [GB].
Context I want: [tokens].
Graphics memory or unified memory: [GB].
People generating at once: [number].
Estimate weight memory at 0.5 bytes per parameter if I only know the parameter count, then say what is left for the key-value cache.
Tell me which package this is: one 16 GB card, one 24 GB card with a short context, several GPUs under vLLM, or a DGX Station-class machine with 748 GB of shared memory.
List the three records I should keep: time to first token, output tokens per second over three runs with the median, and one fixed lm-evaluation-harness task.

Autumn 2026 packages

Four packages, with file sizes taken from the Ollama library pages for Gemma 4 and Qwen3.5 in October 2026, and the shared-machine figures taken from NVIDIA's DGX Station specification. A published file size is the download. The key-value cache for the context window sits on top of it. Card and memory prices move, so read the shop price the week you buy.

Package Machine Start here Memory check What to count each month
Browser Phone or any computer, in the browser A free chat plan, plus the Hugging Face LLM Course No model file The free allowance, then the provider's paid plan if you pass it
Desktop Laptop ollama run gemma4:e2b (about 7.2 GB; Ollama suggests 8 GB of free graphics memory, or unified memory on a Mac). Tighter memory: qwen3.5:4b, about 3.3–4.0 GB, 256K context. One paid chat plan only for tasks the local file still misses Longer chats need more memory than the file. With less graphics memory, Ollama uses system memory and slows down The subscription, if you take one
Home PC A desktop graphics card, or a Mac with a large unified-memory pool 16 GB card: gemma4:12b (about 7.7–8.0 GB, 256K context) or qwen3.5:9b (about 6.6–7.6 GB). 24 GB card: gemma4:26b (about 16–19 GB; mixture-of-experts, about 3.8 billion active parameters out of 25.2 billion) or qwen3.5:27b (about 17–20 GB). Embeddings: nomic-embed-text qwen3.5:35b is listed at 22 GB, so a 24 GB card has almost no room left for context. A 19 GB file is already larger than 16 GB Electricity for that PC, set beside an API bill from the LiteLLM price file
Shared machine Linux, several GPUs, or a DGX Station class system qwen3.5:122b, listed at 81 GB with a 256K context. Serve with vllm serve on port 8000, or LocalAI in Docker on port 8080 One PCIe Gen5 x16 slot and two x16 slots wired as x8, as DGX Station lists them. Power specified at 1,600 W and a 20 A circuit. Memory: 252 GB HBM3e plus 496 GB LPDDR5X (748 GB coherent), Ubuntu, up to seven Multi-Instance GPU slices, models up to 1 trillion parameters Hours run times measured watts times your tariff

Before you buy or swap

Write this down for the package you are about to use.

  1. The task, in one sentence: chat, coding, your own documents, or images.
  2. Which files must stay on the machine. Those tasks skip the Browser package.
  3. The weight-file size in GB, copied from the model page.
  4. The context length you will actually use. A 256,000-token window costs cache memory on top of a small-looking file.
  5. Graphics memory or unified memory, with several GB free above the file.
  6. Time to first token, and output tokens per second. Same prompt, same length, three runs, median.
  7. For a retrieval stack: ten questions you can already answer, and how often the right passage is in the top five.
  8. One lm-evaluation-harness task, same flags every time you change a model or a quant format.
  9. A month of real token counts times the per-token price in the LiteLLM price file at https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json, next to local watt-hours times your tariff.
  10. The license on the model card, read before the weights go into anything you ship.

What would belong on a shared machine

A job would belong on a shared machine when the weight file plus the key-value cache will not fit one home GPU. Right now that means a model the size of qwen3.5:122b (81 GB), a long context whose cache exceeds a 24 GB card, a batch run of lm-evaluation-harness, or a LoRA fine-tune of one of those large bases. vLLM can keep the base loaded and attach a separate adapter per project.

Leave these on the other packages: browser chat, gemma4:e2b and qwen3.5:4b on a laptop, gemma4:12b and qwen3.5:9b on a 16 GB card, and gemma4:26b or qwen3.5:27b on a 24 GB card with a short context.

On a Linux GPU server you already control, one person loads a model, does the work, and unloads it. DGX Station's published split is 252 GB of HBM3e and 496 GB of LPDDR5X joined by NVLink-C2C at 900 GB/s, with Ubuntu installed and Multi-Instance GPU able to cut the GPU into as many as seven isolated slices. Plan power at the specified 1,600 W on a 20 A circuit, and plan lanes from the listed slots: one PCIe Gen5 x16, and two PCIe Gen5 x16 slots wired electrically as x8.

Models too big for one home graphics card are why Polaris plans to raise money for a community station. About the shared station

Sources

  • Ollama quickstartInstall steps, the local command for Gemma 4 E2B, the about-7.2 GB download, the 8 GB graphics-memory suggestion, and the local API on port 11434.
  • Gemma 4 on OllamaOctober 2026 file sizes and context lengths for E2B, E4B, 12B, 26B, and 31B, plus the 26B mixture-of-experts parameter counts.
  • Qwen3.5 on OllamaOctober 2026 file sizes from the 4B (about 3.3–4.0 GB) through the 9B, 27B, 35B (22 GB), and 122B (81 GB), each with a 256K context.
  • LiteLLM model price fileA maintained JSON list of per-token prices and context windows, used here to turn a month of real token counts into an API bill you can set beside local electricity.
  • vLLM documentationTensor and pipeline parallelism, multi-LoRA serving, and the quantization formats. The quickstart on the same site covers Linux, Python 3.11–3.14, and vllm serve on port 8000.
  • vLLM quantizationWhat quantization changes in memory, and which formats (including GPTQ, AWQ, GGUF, FP8, INT8, and INT4) run on which GPU generations.
  • NVIDIA DGX Station748 GB coherent memory (252 GB HBM3e and 496 GB LPDDR5X), NVLink-C2C at 900 GB/s, PCIe slot wiring, 1,600 W on a 20 A circuit, Ubuntu, up to seven MIG slices, and support for models up to 1 trillion parameters.
  • lm-evaluation-harnessHow to score a named task such as HellaSwag against a local model or a vLLM server, with the same flags every time. Read LICENSE.md in the repo before you reuse the code.

Explore next

Back to top