# Ollama API: call a local model from your code

> The Ollama API is the set of web endpoints the local server answers at `http://localhost:11434/api`, with no key required. `/api/generate` answers a single prompt and `/api/chat` takes a list of messages. Both stream by default. The same models also answer the OpenAI SDK at `/v1`, and a Modelfile saves a system prompt and settings under a new model name.

Mindket editors · Published 2026-11-05 · Updated 2026-11-05 · https://mindket.com/learn/local-ai/ollama-api

## In short

- Send a prompt to your own machine from curl or a short Python script.
- Keep a conversation going by resending its history with every call.
- Swap a cloud base URL for your laptop without rewriting the client code.
- Ship a reusable assistant as one model name with its instructions built in.

## Before you start

You need Ollama running, as set up in [lesson 1](https://mindket.com/learn/local-ai/ollama-tutorial), and one model on disk. The examples use `gemma4`, the model in most of Ollama's API examples, so pull it first or swap in any tag you already have. For the code you also need Python 3 with two packages, `ollama` and `openai`.

An API here is a set of web addresses that take JSON and return JSON. The local server answers on port `11434`. Requests from your own machine need no key, and the server listens on `127.0.0.1` only, so another computer cannot reach it until you change `OLLAMA_HOST`.

Ollama does not version this API strictly. Its [introduction page](https://docs.ollama.com/api/introduction) says the API is expected to stay stable and backwards compatible, and there are official libraries for Python and JavaScript.

There are two ways in, and the choice decides what you can reach. The native routes under `/api` expose everything Ollama does, including per-request options, `keep_alive` and thinking control. The `/v1` routes copy OpenAI's format, so code and tools written for OpenAI run unchanged, minus a few fields.

The same API also runs in Ollama's cloud. For a cloud model the base becomes `https://ollama.com/api`, and each request carries an `Authorization: Bearer` header with an `OLLAMA_API_KEY` from your [account settings](https://ollama.com/settings/keys). That key belongs on a server, never in browser code or in source control. Everything below uses the local server, where no key exists to leak.

The API grew in steps, each announced on Ollama's blog. A tutorial written before a step will not mention it, so check its date against this list.

````bash
ollama pull gemma4
pip install ollama openai
````

**When each part of the Ollama API arrived**

1. **February 8, 2024** OpenAI compatibility. The `/v1` routes let code written for OpenAI's API call a local model.
2. **July 25, 2024** Tool support. `/api/chat` takes `tools`, and a model that supports them answers with `tool_calls`.
3. **December 6, 2024** Structured outputs. `format` accepts a JSON schema, so the reply comes back in the shape you define.
4. **May 28, 2025** Streaming with tool calls. Replies keep streaming while the model calls tools.
5. **January 16, 2026** Anthropic API compatibility. Claude Code runs on Ollama from v0.14.0, through an Anthropic-style Messages API.
6. **August 25, 2026** Claude Desktop support. Select Claude in the Ollama app and switch it on. The gateway sets itself up, for local and cloud models.

Sources: [Ollama blog: OpenAI compatibility](https://ollama.com/blog/openai-compatibility) (read 2026-11-05), [Ollama blog: Tool support](https://ollama.com/blog/tool-support) (read 2026-11-05), [Ollama blog: Structured outputs](https://ollama.com/blog/structured-outputs) (read 2026-11-05), [Ollama blog: Streaming responses with tool calling](https://ollama.com/blog/streaming-tool) (read 2026-11-05), [Ollama blog: Claude Code with Anthropic API compatibility](https://ollama.com/blog/claude) (read 2026-11-05), [Ollama blog: Claude Desktop support with Ollama](https://ollama.com/blog/claude-desktop) (read 2026-11-05)

## Send your first request with curl

Start with the simplest endpoint. The generate endpoint, `/api/generate`, takes one prompt and returns one answer. Adding `"stream": false` makes the server wait and send a single JSON object, which is easier to read the first time. The chat endpoint, `/api/chat`, takes a list of messages instead, each with a `role` and its `content`.

Neither endpoint remembers earlier calls. With `/api/chat` you send the whole conversation every time: the earlier questions, the assistant's earlier answers, then the new question. Leave a turn out and the model never sees it. Every turn also makes the prompt longer, and the `prompt_eval_count` in each reply shows it growing.

Both endpoints accept more than text. An `options` object holds the settings for one request: `temperature`, `top_p`, a `seed` for repeatable output, `stop` sequences, `num_predict` to cap the length of the reply and `num_ctx` for the context length. For a model that reads pictures, `images` takes a list of base64-encoded files.

Thinking models add one more field. With `think` you ask for the model's reasoning or switch it off, and some models, such as `gpt-oss`, take a level like `"low"` instead of true or false. The reasoning comes back in its own `thinking` field, apart from the answer. To see which values a model accepts, ask `/api/show`.

A request with a model name and nothing else is useful too. It loads the model without generating anything, so the first real question does not wait for the load.

````bash
# is the server up, and which models does it have?
curl http://localhost:11434/api/tags

# one prompt, one answer
curl http://localhost:11434/api/generate -d '{
  "model": "gemma4",
  "prompt": "Why is the sky blue?",
  "stream": false
}'

# a conversation: send the messages so far
curl http://localhost:11434/api/chat -d '{
  "model": "gemma4",
  "messages": [
    {
      "role": "user",
      "content": "why is the sky blue?"
    }
  ],
  "stream": false
}'
````

1. Run the `/api/tags` call. It returns a `models` list with each model's name, its size on disk in bytes and details such as `parameter_size` and `quantization_level`. An error here means the server is not running.
2. Run the `/api/generate` request. The answer is in the `response` field, and `done` is `true`.
3. Run the `/api/chat` request. The answer now sits in `message.content`, and its `role` is `assistant`.
4. Append that assistant message and a follow-up question to `messages`, then send the request again. The model can now refer to its own first answer.
5. Remove `"stream": false` from either request and run it once more. The reply arrives as many lines, one JSON object per line, each holding the next piece of text.

|  | `/api/generate` | `/api/chat` |
| --- | --- | --- |
| What you send | `prompt`, one string | `messages`, a list of `role` and `content` pairs |
| System prompt | The `system` field | A message whose role is `system` |
| Earlier turns | None, each call stands alone | Whatever history you put in `messages` |
| Where the answer is | `response` | `message.content` |
| Extra inputs | `suffix` for fill-in-the-middle models, `raw` to skip the prompt template | `tools` for function calling, and `tool` messages for the results |

## Call it from Python: the Ollama library or the OpenAI SDK

The official Python library wraps the same endpoints. Its `chat()` function takes a model name and a list of messages. With `stream=True` the call returns chunks instead of one answer, and each chunk holds the next piece of the reply in `message.content`. The loop below prints them as they arrive, so the first words show up before the model has finished.

Streaming is the server's default, and it uses newline-delimited JSON: one complete JSON object per line, sent as soon as there is more text. A chunk carries its piece of the answer and `done: false`. The last chunk has `done: true`, a `done_reason` such as `stop`, and the statistics for the whole run, as the [streaming page](https://docs.ollama.com/api/streaming) shows.

Those statistics answer most questions about speed. Every duration is in nanoseconds. Tokens per second, the usual speed figure, is `eval_count` divided by `eval_duration`, times one billion. A token is a piece of text, often part of a word, and [what tokens are](https://mindket.com/blog/what-are-tokens-in-ai) covers why models count in them.

The generate reference prints one reply in full as its example. It came from `gemma4` and ends with `done_reason: stop`, `prompt_eval_count: 11` and `eval_count: 18`, so that prompt cost eleven input tokens and the answer eighteen.

Chat is one call of many. Next to it sit `generate`, `list`, `show`, `create`, `copy`, `delete`, `pull`, `push`, `embed` and `ps`, which cover what the command line does, so a script can pull a model before it uses it.

To reach a server on another address, build a client yourself: `Client(host='http://localhost:11434')` takes the host and any extra headers.

Plenty of tools already speak OpenAI's API, and Ollama answers the same requests under `/v1`. You change the base URL and keep the rest of the code. The client refuses to start without an API key, and Ollama ignores whatever you pass, so any string works.

The second snippet is the compatibility page's own example with the model swapped for `gemma4`. The routes it lists are `/v1/chat/completions`, `/v1/completions`, `/v1/models`, `/v1/embeddings` and `/v1/responses`, plus audio routes for speech, transcription and translation.

Chat completions ignore `logprobs`, `tool_choice`, `logit_bias`, `user` and `n`, and images must be base64 data, not URLs. The Responses route is stateless, so `previous_response_id` and `conversation` don't work and you resend the history each time.

Nothing in the OpenAI API sets the context length. The workaround is a new model built from a Modelfile with a larger `num_ctx`, which any OpenAI client can then name in `model`.

JavaScript works the same way. The OpenAI client there takes `baseURL: "http://localhost:11434/v1/"` and `apiKey: "ollama"`, and the rest of the call does not change.

Code written for Claude has a route as well. Ollama supports a subset of Anthropic's Messages API, so the Anthropic client works with its base URL set to `http://localhost:11434`. That client adds `/v1/messages` to the address by itself.

````python
from ollama import chat

stream = chat(
  model='gemma4',
  messages=[{'role': 'user', 'content': 'Why is the sky blue?'}],
  stream=True,
)

for chunk in stream:
  print(chunk['message']['content'], end='', flush=True)

# the same model through the OpenAI SDK
from openai import OpenAI

client = OpenAI(
    base_url='http://localhost:11434/v1/',
    api_key='ollama',  # required but ignored
)

chat_completion = client.chat.completions.create(
    messages=[
        {
            'role': 'user',
            'content': 'Say this is a test',
        }
    ],
    model='gemma4',
)
print(chat_completion.choices[0].message.content)
````

| Field | What it holds |
| --- | --- |
| `prompt_eval_count` | Input tokens in the prompt |
| `prompt_eval_cached_count` | Prompt tokens read from the cache instead of processed again |
| `eval_count` | Tokens the model generated |
| `load_duration` | Time spent loading the model, which shows up on a cold start |
| `prompt_eval_duration` | Time spent on the prompt tokens that were not cached |
| `eval_duration` | Time spent generating the answer |
| `total_duration` | Time for the whole request |

## Save a system prompt in a Modelfile

A Modelfile is a short text file that builds a new model from one you already have. The `FROM` line names the base model. Each `PARAMETER` line sets an option, such as `num_ctx`, the context length in tokens, or `temperature`, which controls how random the wording gets. The `SYSTEM` line holds the system prompt that every request to the new model carries.

`ollama create` reads the file and registers the name. From then on the new model works everywhere a model name does: in `ollama run`, in `/api/chat` and in the OpenAI SDK. The SYSTEM line is a prompt like any other, and the [system prompt lesson](https://mindket.com/learn/prompt-engineering/system-prompt) shows how to write a good one.

A model pulled from Hugging Face takes the same settings another way. A file named `system` in its repository sets the system prompt, and a JSON file named `params` sets the parameters.

The [Modelfile reference](https://docs.ollama.com/modelfile) lists more instructions: `TEMPLATE` for the full prompt template, `MESSAGE` for message history, `LICENSE` and `REQUIRES` for the minimum Ollama version.

````text
FROM gemma4
PARAMETER num_ctx 8192
PARAMETER temperature 0.3
SYSTEM You answer questions about the user's notes in plain English. If the notes do not say, answer that you do not know.
````

1. Save the four lines above as `Modelfile` in an empty folder.
2. Run `ollama create notes-helper -f ./Modelfile`.
3. Run `ollama run notes-helper` and ask it something your notes would not cover.
4. Print what you built with `ollama show --modelfile notes-helper`. Running the same command on `gemma4` shows the template and parameters your file inherits.

## Georgi Gerganov on why local models seem to fail

Georgi Gerganov builds ggml-org, the GitHub organization behind llama.cpp, and works as an engineer at Hugging Face. In a reply [on X](https://x.com/ggerganov/status/2038674698809102599) in March, he answered a question about local models. He recommended Qwen 3.5 for its range of sizes. Then he warned that most problems people hit sit outside the model: in the program that drives it, in the chat template and in the way the prompt is built.

His advice was to start with full-quality models that fit your hardware, and to know what the program around the model does. Hooking Claude Code or Codex to a local model and expecting magic, he wrote, does not work. Quantize for speed only once things click. For his own tests he picked Q8_0 variants of most models to keep most of the original quality.

The API puts the template in your hands. On `/api/chat`, Ollama applies the model's own chat template to your messages. The `raw` option on `/api/generate` skips it, so leave `raw` off unless you rebuild the template yourself. `ollama show --modelfile` prints the template a model uses. When a local model fails a task, read that template and the request your client sends before you blame the model.

> I think the consensus is that Qwen3.5 is a step change so atm I would recommend explore that, given that it covers a range of sizes suitable for all devices.
>
> Georgi Gerganov, Builds ggml-org, engineer at Hugging Face, [X](https://x.com/ggerganov/status/2038674698809102599) (read 2026-11-05)

[![Georgi Gerganov (@ggerganov) on X, 30 March 2026, replying about local models. He calls Qwen3.5 a step change with sizes for all devices, then starts a note on the main issues people face with local models.](https://mindket.com/blog/shots/x-ggerganov-local-models.webp?v=9109fffd4a60)](https://x.com/ggerganov/status/2038674698809102599)

Georgi Gerganov on X, 30 March 2026

> A screenshot of X's own embed of the post, taken on 5 November 2026, with the time in UTC. X cuts the post short with Show more. Select it to open the post on X.

## Common mistakes

- Parsing a streamed reply as one JSON document. Without `"stream": false` the body is many objects, one per line, so read it line by line.
- Sending only the newest message to `/api/chat`. The server keeps no conversation, so the model forgets every turn you leave out.
- Calling the API from a web page or an extension and getting blocked. By default Ollama accepts browser requests only from `127.0.0.1` and `0.0.0.0` origins. Add others to `OLLAMA_ORIGINS`, for example `chrome-extension://*`.
- Ignoring `done_reason`. A reply capped by `num_predict` can stop mid-sentence, and `done_reason` is the field that says why the reply ended.
- Letting the model unload in the middle of a batch job. Each request can carry `keep_alive`: a negative value keeps the model loaded, and `0` unloads it right after the answer.
- Putting a cloud key in a desktop app or a web page. Calls to `https://ollama.com/api` need `OLLAMA_API_KEY`, so send them from your own server instead.
- Asking for JSON in the prompt and hoping. Set `format` to `"json"`, or to a JSON schema object, and the reply comes back in that structure.
- Naming a model in `/v1` code that `ollama ls` does not list. The local server runs only models on disk, so the fix is `ollama pull`, not a new key.

## Try it yourself

- Send the same prompt to `/api/generate` twice, once with `"stream": false` and once without. How many lines does the streamed reply have?
- Work out tokens per second from `eval_count` and `eval_duration` in one response.
- List your models from Python with the library's `list` call and print each name.
- Change the OpenAI example's `base_url` back and forth between Ollama and a cloud service. Which lines changed?
- Add `PARAMETER num_ctx` to your Modelfile, create the model again and check the CONTEXT column in `ollama ps` while it runs.
- Send `"keep_alive": 0` with a request and watch the model leave `ollama ps` straight after the answer.
- Ask a thinking model the same question with `think` on and off, and compare the `thinking` field with the answer.

> Next: LM Studio runs the same kind of local server inside a desktop app, with a model browser and an OpenAI-compatible endpoint of its own. [Lesson 3](https://mindket.com/learn/local-ai/lm-studio-tutorial) sets it up and compares its `lms` command line with the Ollama commands you know now.

## Check yourself

1. Where is the answer in a non-streaming /api/chat response?

   A. response
   B. message.content
   C. choices[0].text
   D. done_reason

   Answer: B. The chat endpoint returns an assistant message, so the text is in message.content, while /api/generate uses response.

2. Your second /api/chat call ignores the first question. What is missing?

   A. A higher keep_alive
   B. An API key
   C. The earlier turns in the messages list

   Answer: C. The server keeps no conversation, so every call must carry the history you want the model to see.

3. What do you pass as the API key when the OpenAI SDK talks to Ollama?

   A. Any string, since Ollama ignores it
   B. Your OpenAI key
   C. The OLLAMA_HOST value
   D. Nothing, the client starts without one

   Answer: A. The client requires a value, and the local server ignores it.

4. How do you give an OpenAI-style client a longer context window?

   A. Add num_ctx to the OpenAI request
   B. Set stream to false
   C. Raise max_tokens
   D. Create a model from a Modelfile with PARAMETER num_ctx

   Answer: D. The OpenAI API has no context-length field, so the setting goes into a new model the client can name.

## Frequently asked questions

### How do I call the Ollama API from PowerShell?

Use `Invoke-WebRequest`, as the Windows page does: `(Invoke-WebRequest -method POST -Body '{"model":"llama3.2", "prompt":"Why is the sky blue?", "stream": false}' -uri http://localhost:11434/api/generate ).Content | ConvertFrom-json`. Swap `llama3.2` for a model you have pulled. Keep `"stream": false` in the body, because `ConvertFrom-json` expects one JSON object, not one per line.

### Can another computer on my network use the Ollama API?

Yes. Bind it to every network interface with `OLLAMA_HOST=0.0.0.0:11434`, set where the app or service reads its environment, then restart Ollama. Anyone who reaches the port can then send requests, so add a firewall rule that admits only your own machines.

### Does the Ollama API support tool calling?

Yes, through `/api/chat`. Pass function definitions in `tools`, and when the reply contains `tool_calls`, run the function and send the result back as a message with the role `tool`. The chat reference shows it with `qwen3` and a `get_current_weather` function. The model only asks for the call, and your code decides whether to run it.

### Can the Ollama API create embeddings?

Yes. Pull an embedding model such as `embeddinggemma`, then send text to `/v1/embeddings` or call `embed` in the Python library. Each text comes back as a list of numbers, the form a search index stores, and the RAG course builds on it.

## Sources

- [Ollama Docs: API introduction](https://docs.ollama.com/api/introduction) (read 2026-11-05)
- [Ollama Docs: Generate a response](https://docs.ollama.com/api/generate) (read 2026-11-05)
- [Ollama Docs: Generate a chat message](https://docs.ollama.com/api/chat) (read 2026-11-05)
- [Ollama Docs: List models](https://docs.ollama.com/api/tags) (read 2026-11-05)
- [Ollama Docs: Streaming](https://docs.ollama.com/api/streaming) (read 2026-11-05)
- [Ollama Docs: OpenAI compatibility](https://docs.ollama.com/api/openai-compatibility) (read 2026-11-05)
- [Ollama Docs: Anthropic compatibility](https://docs.ollama.com/api/anthropic-compatibility) (read 2026-11-05)
- [Ollama Docs: Modelfile reference](https://docs.ollama.com/modelfile) (read 2026-11-05)
- [Ollama Docs: FAQ](https://docs.ollama.com/faq) (read 2026-11-05)
- [Ollama Docs: Windows](https://docs.ollama.com/windows) (read 2026-11-05)
- [GitHub: ollama/ollama-python](https://github.com/ollama/ollama-python) (read 2026-11-05)
- [Ollama blog: OpenAI compatibility](https://ollama.com/blog/openai-compatibility) (read 2026-11-05)
- [Ollama blog: Tool support](https://ollama.com/blog/tool-support) (read 2026-11-05)
- [Ollama blog: Structured outputs](https://ollama.com/blog/structured-outputs) (read 2026-11-05)
- [Ollama blog: Streaming responses with tool calling](https://ollama.com/blog/streaming-tool) (read 2026-11-05)
- [Ollama blog: Claude Code with Anthropic API compatibility](https://ollama.com/blog/claude) (read 2026-11-05)
- [Ollama blog: Claude Desktop support with Ollama](https://ollama.com/blog/claude-desktop) (read 2026-11-05)
- [Georgi Gerganov on X: the main issues with local models](https://x.com/ggerganov/status/2038674698809102599) (read 2026-11-05)

[Browse MCP servers](https://mindket.com/mcp-servers)