Ollama Python Tutorial: Connect with the Ollama Package and OpenAI SDK

Install

pip install ollama

First, make sure the Ollama service is running and that you have downloaded the model you want to use (for example, ollama pull qwen3:8b).

Basic chat

from ollama import chat

response = chat(
    model="qwen3:8b",
    messages=[
        {"role": "system", "content": "Answer in English."},
        {"role": "user", "content": "Explain machine learning in three points."},
    ],
)

print(response.message.content)

You can also read the response object like a dictionary: response["message"]["content"].

Streaming output

Display text as it is generated:

from ollama import chat

stream = chat(
    model="qwen3:8b",
    messages=[{"role": "user", "content": "Write a 100-word sample self-introduction."}],
    stream=True,
)

for chunk in stream:
    print(chunk.message.content, end="", flush=True)
print()

Multi-turn chat: build a terminal chatbot

The model does not remember the conversation on its own. Store every message in a list:

from ollama import chat

messages = [{"role": "system", "content": "You are a friendly assistant. Answer in English."}]

while True:
    user_input = input("You: ").strip()
    if user_input in {"exit", "quit", "leave"}:
        break
    messages.append({"role": "user", "content": user_input})

    reply = ""
    print("AI：", end="", flush=True)
    for chunk in chat(model="qwen3:8b", messages=messages, stream=True):
        piece = chunk.message.content
        reply += piece
        print(piece, end="", flush=True)
    print()

    messages.append({"role": "assistant", "content": reply})

The longer the conversation gets, the more content is sent, and eventually it may exceed the context length. In practice, you can keep only the most recent turns or summarize older conversations.

Parameters and thinking mode

response = chat(
    model="qwen3:8b",
    messages=[{"role": "user", "content": "What is the sum of the numbers from 1 to 100?"}],
    options={"temperature": 0.2, "num_ctx": 8192},
    think=True,
    keep_alive="30m",
)

print("Thinking: ", response.message.thinking)
print("Answer: ", response.message.content)

Read images

response = chat(
    model="gemma3",
    messages=[{
        "role": "user",
        "content": "What is in this picture?",
        "images": ["./photo.jpg"],   # You can also pass bytes or a base64 string
    }],
)
print(response.message.content)

Other features

import ollama

ollama.generate(model="qwen3:8b", prompt="Introduce Taipei in one sentence")  # One-off generation
ollama.list()                                               # Downloaded models
ollama.ps()                                                 # Loaded models
ollama.pull("nomic-embed-text")                             # Download a model
ollama.show("qwen3:8b")                                     # Model information
ollama.embed(model="nomic-embed-text", input="Hello")      # Text embeddings

Connect to another host

from ollama import Client

client = Client(host="http://192.168.1.50:11434")
response = client.chat(model="qwen3:8b", messages=[{"role": "user", "content": "Hello"}])
print(response.message.content)

You can also set the OLLAMA_HOST environment variable, and the default chat() function will use it.

Async

Use this in an async framework such as FastAPI:

import asyncio
from ollama import AsyncClient


async def main():
    client = AsyncClient()
    response = await client.chat(
        model="qwen3:8b",
        messages=[{"role": "user", "content": "Hello"}],
    )
    print(response.message.content)


asyncio.run(main())

Error handling

import ollama

try:
    ollama.chat(model="not-exist", messages=[{"role": "user", "content": "hi"}])
except ollama.ResponseError as e:
    print("Error: ", e.error, e.status_code)
    if e.status_code == 404:
        print("Model not found. Run ollama pull first.")
except ConnectionError:
    print("Cannot connect to Ollama. Make sure the service is running.")

Switch to the OpenAI Python SDK

If your program already uses the OpenAI SDK, just change base_url:

pip install openai

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11434/v1/",
    api_key="ollama",  # Required, but Ollama does not check it
)

completion = client.chat.completions.create(
    model="qwen3:8b",
    messages=[{"role": "user", "content": "Hello, introduce yourself."}],
)
print(completion.choices[0].message.content)

This lets you switch the same program between local models and cloud APIs. Frameworks such as LangChain and LlamaIndex can also connect to Ollama the same way.

The next section covers JavaScript / Node.js.

Updated October 12, 2026