Ollama Not Using GPU, Running on 100% CPU: 4-Step Fix (Tested Output, 2026)

Short answer: run ollama ps while the model is loaded and read the PROCESSOR column. If it says 100% CPU, Ollama did not find a usable GPU, or you told it not to use one. Next, read the first lines of the server log: the line inference compute names the device Ollama picked. The usual causes are an unsupported or too-old GPU driver, a container that cannot see the GPU, a leftover CUDA_VISIBLE_DEVICES=-1, or a model that is bigger than your VRAM. I ran Ollama 0.40.15 on a CPU-only Linux box and show the real output below. I had no GPU to test with, so the GPU fixes come from Ollama’s official docs and are marked that way.

Step 1: confirm it is really on the CPU

Load the model, then run ollama ps in another terminal. The official FAQ explains the PROCESSOR column:

• 100% GPU: the model sits fully in GPU memory.
• 100% CPU: the model sits fully in system memory.
• 48%/52% CPU/GPU (example from the FAQ): the model is split between both.

This is what I got on the CPU-only box with a tiny test model:

NAME           ID              SIZE      PROCESSOR    CONTEXT    RUNNER      UNTIL
tiny:latest    8cf91c7109e7    6.3 MB    100% CPU     2048       llamacpp    4 minutes from now

The API shows the same thing. curl localhost:11434/api/ps returned "size_vram":0 for that model. A model that sits fully on a GPU should report a non-zero size_vram; I could not confirm that here.

Step 2: read the server log for “inference compute”

Where the log lives depends on how you run Ollama. The troubleshooting page lists them: ~/.ollama/logs/server.log on a Mac, journalctl -u ollama --no-pager --follow --pager-end on Linux with systemd, docker logs <container-name> in a container, and the terminal itself when you run ollama serve by hand.

On my CPU-only box the start of the log said this:

msg="discovering available GPUs..."
msg="inference compute" id=cpu library=cpu compute="" name=cpu ... total="15.6 GiB" available="8.4 GiB"
msg="vram-based default context" total_vram="0 B" default_num_ctx=4096

library=cpu is the giveaway. On a machine where Ollama finds a GPU, that line should name the GPU instead; I could not check this because the box has none.

For more detail start the server with OLLAMA_DEBUG=1. On my box the debug log then showed why CUDA was skipped:

load_backend: failed to load .../cuda_v12/libggml-cuda.so: libcuda.so.1: cannot open shared object file: No such file or directory

That means no NVIDIA driver library was found. On a real GPU machine, the same kind of line would point at a driver problem.

Step 3: check for an override that forces the CPU

The docs say that setting CUDA_VISIBLE_DEVICES (NVIDIA), ROCR_VISIBLE_DEVICES (AMD) or GGML_VK_VISIBLE_DEVICES (Vulkan) to an invalid ID such as -1 forces CPU use. It is easy to set this once while testing and forget it. I tested it:

CUDA_VISIBLE_DEVICES=-1 OLLAMA_DEBUG=1 ollama serve
msg="user overrode visible devices" CUDA_VISIBLE_DEVICES=-1
msg="if GPUs are not correctly discovered, unset and try again"

Ollama even tells you to unset the variable. Run env | grep -i -E "visible_devices|OLLAMA" in the shell (or check the systemd unit or Docker command) to see what the server really got.

Step 4: fix the cause (from the official docs)

I could not test these on a GPU. They come from the hardware support page and the troubleshooting page.

NVIDIA

• The card needs compute capability 5.0 or higher and driver version 550 or newer. Cards with compute capability 5.0 to 6.2 need driver 570 or newer.
• After a suspend and resume on Linux, Ollama can lose the GPU and fall back to CPU. The docs suggest reloading the UVM driver: sudo rmmod nvidia_uvm && sudo modprobe nvidia_uvm.
• If the log shows error codes such as 3, 46, 100 or 999, the docs suggest: check the container runtime, run sudo nvidia-modprobe -u, reload nvidia_uvm, reboot, and update the driver. To get more detail, set CUDA_ERROR_LEVEL=50.

AMD

• On Linux, Ollama needs the AMD ROCm v7 driver. An older ROCm 6 driver can make discovery hang until it times out and fall back to CPU. The log then shows failure during GPU discovery.
• Your user may need to be in the video or render group to open /dev/kfd.
• For some unsupported cards the docs describe HSA_OVERRIDE_GFX_VERSION, for example 10.3.0 for an RX 5400. This is an experiment, not a guarantee.

Docker

• Check that the container runtime works first: docker run --gpus all ubuntu nvidia-smi. If that fails, Ollama cannot see the GPU either.
• If Ollama starts on the GPU and later drops to CPU, the docs say to set "exec-opts": ["native.cgroupdriver=cgroupfs"] in /etc/docker/daemon.json.

The model does not fit in VRAM

A model larger than your VRAM is split, and ollama ps shows a mix such as 48%/52% CPU/GPU. That is normal behavior, not a fault. Use a smaller model or a smaller context window. See Ollama context length and the VRAM calculator.

Pick which GPU Ollama uses

With several GPUs, the docs give one variable per vendor: CUDA_VISIBLE_DEVICES (use UUIDs from nvidia-smi -L, because numbering can change), ROCR_VISIBLE_DEVICES (see rocminfo) and GGML_VK_VISIBLE_DEVICES for Vulkan. To switch Vulkan off, set OLLAMA_VULKAN=0. Set these on the server process, not in the shell where you run the client. Compare your card with the best-value GPUs for LLM inference and what an RTX 4090 can run before you buy.

Related errors

More Ollama and llama.cpp messages are in the error list. If the log instead says “model requires more system memory”, read that article.

FAQ

Does Ollama use the GPU automatically?

Yes, when it finds a supported one. The docs describe automatic discovery at start-up. On a Mac it uses Metal for Apple GPUs.

Why is a small model fast on CPU but a big one crawls?

A CPU has far less memory bandwidth than a GPU, so speed drops quickly as the model grows. I did not benchmark this on the test box, because the test model is only 6 MB.

Can I force CPU on purpose?

Yes. An invalid ID such as CUDA_VISIBLE_DEVICES=-1 does it, as tested above.

Sources and test notes

• Ollama docs: Hardware support (docs.ollama.com/gpu), Troubleshooting (docs.ollama.com/troubleshooting), FAQ (docs.ollama.com/faq).
• Tested myself: Ollama 0.40.15 on a CPU-only Linux box, ollama ps, /api/ps, the start-up log, OLLAMA_DEBUG=1, CUDA_VISIBLE_DEVICES=-1.
• Not tested: any real NVIDIA, AMD or Apple GPU, Docker GPU passthrough, driver fixes, HSA_OVERRIDE_GFX_VERSION.