How to use Ollama: run AI models on your own computer

What Ollama is

Ollama is an open source program, under the MIT license according to its GitHub page, that downloads open AI models and runs them on your own machine. An open model is one whose weights you can download, the large files that hold what the model learned. Ollama is built on llama.cpp, the open source engine that also runs GGUF model files inside LM Studio.

It runs on Windows, macOS and Linux, from a terminal or from a desktop app. Behind both sits a small server on your computer at 127.0.0.1:11434, and the ollama command talks to it. Running local models is free and needs no account. The FAQ says Ollama does not see your prompts or data for them. Cloud models are the exception: they run on ollama.com and need a sign-in.

What Ollama is good for

• A private chat with an open model: ollama run gemma4:e2b, the model the quickstart starts with, opens a chat in the terminal.
• One question, one answer: put a prompt in quotes after the model name and Ollama prints a single answer, which suits scripts.
• Reading pictures: with a model that accepts images, write the file path inside the prompt and ask what the picture shows.
• A local API for your own programs: the server at 127.0.0.1:11434 answers requests from curl, Python or any app that can call it.
• Text turned into numbers: ollama run embeddinggemma "Hello world" prints the list of numbers a search tool compares.
• Local models for coding agents: ollama launch sets up OpenCode, Claude Code, Codex, VS Code or Droid against your own server, as the CLI reference lists.
• Models from Hugging Face: ollama run hf.co/{username}/{repository} runs a GGUF repository from the Hub, as Hugging Face's Ollama page describes.

When to use Ollama, and when to pick another tool

Ollama fits when you want a model on your own hardware. When you want the strongest model available, a hosted assistant wins, because the largest models do not fit on a typical computer.

How to use Ollama, step by step

Install Ollama first with the one-line command in lesson 1. The five pictures below show what the commands printed on a Windows PC, in order.

1. Open a new terminal and run ollama -v. A line that starts with ollama version means Ollama is installed and the terminal can find the command.

   

2. Run ollama pull gemma4:e2b to download a first model, about 7.2 GB, then ollama ls to list the models on your disk. The computer in the pictures already held qwen2.5:7b, so they show that model. Type gemma4:e2b wherever they show it.

   

3. Ask one question without opening a chat: ollama run gemma4:e2b "Why is the sky blue?". The first answer waits while Ollama loads the model. Leave out the quoted prompt and you get a >>> chat prompt instead, where /bye ends the chat.

   

4. Run ollama ps straight afterwards. It lists the models held in memory. The PROCESSOR column says where each one runs, and here it reads 96%/4% CPU/GPU, so most of the model sits in system RAM. The last column counts down to when Ollama unloads it.

   

5. Free the memory with ollama stop gemma4:e2b, which prints nothing. A second ollama ps then shows only its header row, so no model is loaded.

   

Real output of Ollama 0.40.14 on Windows, shown in a terminal frame, taken on 6 November 2026.

The tutorials at the top of this page show these steps on screen. The Windows and macOS installs also include a desktop app, which the Sam Witteveen video covers.

An example to try

These four lines download the quickstart's first model, ask it one question, show where it sits in memory and unload it. The download is large, so start it on a connection you can leave running.

ollama pull gemma4:e2b
ollama run gemma4:e2b "Why is the sky blue?"
ollama ps
ollama stop gemma4:e2b

• The answer is a short explanation in plain text. Which model wrote it decides the wording, so yours will differ from the picture.
• The first ollama ps lists the model with a PROCESSOR value such as 100% GPU, or a split like 48%/52% CPU/GPU when part of it did not fit on the card.
• The second ollama ps, after the stop, prints only its header row.

Tips for better results

• Read the size column in Ollama's library before you pull. The gemma4 page lists tags from e2b up to 31b, and the larger ones need far more memory.
• Start with a small tag and move up while ollama ps still shows 100% GPU.
• Closing the chat does not free memory. Ollama keeps a model loaded for 5 minutes, so run ollama stop before a game or a video export.
• To move the model files to a bigger drive, set OLLAMA_MODELS to the new folder and restart Ollama.
• For a long document, raise the context with OLLAMA_CONTEXT_LENGTH or the slider in the app settings.

Limits to know

• The quickstart's first model wants about 8 GB of free VRAM. With less, part of it runs from system RAM and answers slow down. The hardware lesson works out what fits your machine.
• Cloud models are not local. They need ollama signin, they run on Ollama's servers and the FAQ says prompts sent to them are processed there.
• Local requests need no API key. Setting OLLAMA_HOST=0.0.0.0:11434 with no firewall rule lets anyone who reaches the port use your models.
• Deleting a model with ollama rm is final. It has to be pulled again, download and all.

Learn more about Ollama

1. 1Local AI course: run AI models on your own computer · Lesson 1 of 4Ollama tutorial: run AI models on your own computerAn Ollama tutorial for Windows, macOS and Linux: install it, download and chat with a first local model, then list, stop and remove models and move their files.11 min
2. 2Local AI course: run AI models on your own computer · Lesson 2 of 4Ollama API: call a local model from your codeCall a local model from code with the Ollama API: /api/generate and /api/chat with curl and Python, streaming, the OpenAI-compatible endpoint and a Modelfile.11 min
3. 4Local AI course: run AI models on your own computer · Lesson 4 of 4Local LLM hardware requirements: how much VRAM a model needsLocal LLM hardware requirements, worked out: how parameters and GGUF quantization set a model size, how much VRAM it needs, and what CPU, GPU or a Mac changes.10 min

Frequently asked questions

Sources

1. Ollama Docs: Quickstart (read November 5, 2026)https://docs.ollama.com/quickstart
2. Ollama Docs: CLI reference (read November 5, 2026)https://docs.ollama.com/cli
3. Ollama Docs: FAQ (read November 5, 2026)https://docs.ollama.com/faq
4. GitHub: ollama/ollama (read November 6, 2026)https://github.com/ollama/ollama
5. Ollama library: gemma4 (read November 5, 2026)https://ollama.com/library/gemma4
6. Hugging Face Docs: Use Ollama with any GGUF model on the Hub (read November 5, 2026)https://huggingface.co/docs/hub/ollama