Contents

What Is Ollama? A Tutorial for Running Large Language Models Locally

In a nutshell

Ollama is a tool for downloading, managing, and running open-source large language models on your own computer. Think of it as "Docker for AI models":

Docker	Ollama
`docker pull nginx`	`ollama pull llama3.2`
`docker run nginx`	`ollama run llama3.2`
`docker ps`	`ollama ps`
Dockerfile	Modelfile

What can Ollama do?

Chat and answer questions: Have conversations like you would with ChatGPT, but the model runs on your computer.
Analyze images: Models with vision capabilities can describe images and recognize text.
Write code: Use models specialized for code to help write and explain code.
Act as an API server: Ollama provides an HTTP API in the background (default http://localhost:11434) that your programs, web interfaces, and other tools can call.
Generate text embeddings: Convert text into vectors for semantic search and RAG (so a model can answer questions based on your documents).

How it works

ollama run qwen3
     │
     ▼
Ollama service (runs in the background, listening on port 11434)
     │  1. Model isn't available locally → download it from ollama.com
     │  2. Load the model into GPU memory (VRAM) or system memory
     ▼
Inference engine generates a response → streams it to the terminal or API

Once downloaded, a model stays on your hard drive, so you can use it without an internet connection.

Local models vs. cloud AI services

	Local (Ollama)	Cloud (ChatGPT, Gemini, etc.)
Data privacy	Your data stays on your computer	Data is sent to the service provider
Cost	Free (excluding electricity and hardware)	Subscription or pay as you go
Offline use	Yes	No
Model capabilities	Limited by your hardware; large models may not run	Access to the most capable models
Speed	Depends on your graphics card	Usually fast
Setup and maintenance	You handle it yourself	Not required

Local models still lag behind the most capable cloud models, but medium-sized open-source models already work well for everyday tasks such as summarizing, translating, classifying, extracting data, and assisting with code.

Ollama also offers cloud models, so you can use larger models with the same commands and API. See Cloud models.

What kind of computer do you need?

The key factor in deciding how large a model you can run is memory, especially graphics card memory (VRAM).

Your computer	Models you can run smoothly (approx.)
8 GB of memory, no dedicated graphics card	Small 1B–4B models (slower)
Apple Silicon Mac with 16 GB of memory	7B–8B models
NVIDIA graphics card with 8 GB of VRAM	7B–8B models
16–24 GB of VRAM	14B–32B models
Mac with 32 GB or more of unified memory	30B-class models

(Estimates assume common 4-bit quantized versions. See Choosing a model and hardware requirements.)

NVIDIA graphics cards: The broadest support and fastest performance.
Apple Silicon (M-series) Macs: Unified memory lets Macs run larger models than PCs at a similar price, with good performance.
AMD graphics cards: Some models are supported.
CPU only: Models can run, but much more slowly; best suited to small models.

Popular open-source model families

Model	Developer	Highlights
Llama	Meta	One of the best-known open-source model families
Qwen	Alibaba	Strong Chinese-language capabilities and a wide range of sizes
Gemma	Google	Good performance at smaller sizes; some versions support image input
DeepSeek-R1	DeepSeek	Reasoning model that "thinks" before answering
gpt-oss	OpenAI	Open-weight models released by OpenAI
Mistral	Mistral AI	European team; efficient models

The full model list is on the Models page at ollama.com. New models are usually added soon after release.

Recap

Ollama lets you run open-source AI models locally with simple commands.
It also acts as an API server, making it easy to integrate with other programs.
The size of the model you can run depends on your memory, especially your graphics card memory.

Next, we'll install Ollama.

Updated October 12, 2026