Ollama models not loading after update (dual RTX 3060)

19 Oct 2025 - tsp
Last update 19 Oct 2025
Reading time 3 mins

TL;DR

On a box with two NVIDIA GeForce RTX 3060 (12 GB) GPUs, Ollama auto-updated to 0.12.4 and model loads silently stopped working, upgrading to 0.12.5 did not solve the problem. Downgrading to Ollama 0.12.3 fixed it immediately. NVIDIA driver upgrade from 572.82 to 581.29 did not change the behavior (other CUDA apps were fine)

When Ollama auto-updated on my small dual RTX 3060 headless setup, model loading suddenly stopped working - no errors, just silence and hanging clients. The GPUs were detected, yet nothing generated and loading of tensors failed from one day to the other. Upgrading drivers and even moving from version 0.12.4 to 0.12.5 did not help, while other CUDA applications ran perfectly.

After a few hours of debugging, the fix turned out to be simple: rolling back to Ollama 0.12.3 instantly restored normal behavior. If you are seeing lines like llama_model_load: vocab only - skipping tensors and key with type not found ... general.alignment, this post walks through what happened and how to get your models running again without tearing your hair out.

Setup

Symptoms and what I saw in the logs

Server was up and GPUs detected:

Listening on [::]:8182 (version 0.12.5)
inference compute ... CUDA0 / CUDA1 ... NVIDIA GeForce RTX 3060 ... total="12.0 GiB"

When a clients hit /api/show (metadata probe), logs looked normal but gave the impression models don’t load:

POST "/api/show"
llama_model_load: vocab only - skipping tensors

This is expected for /api/show, but in my case real generations never kicked in either.

Extra noise that wasn’t fatal but confused debugging:

key with type not found key=general.alignment default=32

and many lines like:

load: control token: 128... '<|reserved_special_token_...|>' is not marked as EOG

What I changed (that did not fix it)

Tried a driver update from 572.82 to 581.29. This yielded no change (at least OpenCL and CUDA in other apps remained OK).

Root cause (practical)

I didn’t dig deeper into whether it’s scheduling, multi-GPU runner selection, or a llama.cpp interface edge case in 0.12.4/0.12.5 - the point here is the quick fix.

Fix

Downgrade Ollama to 0.12.3. The models load and generate again.

Notes that might help your debugging

Hitting /api/show only opens the GGUF header and tokenizer. It prints:

llama_model_load: vocab only - skipping tensors

This is expected behaviour. You need /api/generate or /api/chat to actually allocate and load tensors into VRAM.

The general.alignment default=32 line is a benign warning from newer llama.cpp when an optional GGUF key is absent.

Conclusion

If your dual-GPU RTX 3060 rig suddenly stops loading models after Ollama auto-updates and you see lines like:

llama_model_load: vocab only - skipping tensors
key with type not found key=general.alignment default=32

try rolling back to Ollama 0.12.3 first. That immediately restored normal model loading and inference for me. This may solve the immediate headache before trying to get resolved how to upgrade without breaking everything.

Note: These links are Amazon affiliate links, this pages author profits from qualified purchases

This article is tagged:

Related articles

Running Ollama on a CPU-Only Headless FreeBSD 14 System

This mini guide walks through compiling and running Ollama on a headless, CPU-only FreeBSD 14.2 system with an Intel Alder Lake-N97 processor. It covers why one might opt for CPU-based inference, the Vulkan-related issue preventing execution without a GUI, and how to disable Vulkan by recompiling from ports. The article provides step-by-step instructions for extracting, modifying, compiling, and installing Ollama, along with details on running inference and transferring models to air-gapped systems.

Simple webcam access from C

This blog post summarizes how one can access common webcams from C on operating systems supporting the video4linux (V4L) API. This information can be used to built robots, surveillance systems or simply communication applications.

Setting Parameters like Context Length and Temperature in Ollama Models

This article provides a concise, practical overview of modifying Ollama models by editing or creating a MODELFILE. It explains how to adjust fundamental parameters such as num_ctx (context length), temperature, and sampling settings, as well as how to modify prompts, templates, and integrate LoRA adapters. The focus is on concrete configuration steps rather than retraining, making it a straightforward reference for those who need to adapt model behavior for specific workloads. The examples demonstrate increasing the context length for code-oriented models, altering generation variability, and attaching domain-specific LoRA modules. The discussion also addresses the technical implications of these changes, including VRAM requirements, inference speed, and the limitations imposed by a model’s original training context.

GPU size estimation for LLMs

Ever wondered how much VRAM you actually need to run a 13B, 65B, or even 500B language model? This compact, no-fluff guide - mostly a personal reference - breaks down GPU memory needs by model size, quantization, and task complexity. Includes real-world GPU examples with affiliate links for anyone thinking of building or upgrading their own AI rig. Great for hobbyists, tinkerers, and anyone asking, 'Can I run that locally?'

How to use Frama-C to proof correctness of AVR microcontroller code

This is a short article that shows how one can use Frama-C and the WP plugin to proof correctness or some properties of ANSI C code written for AVR microcontrollers.

Building Semantic Suggested Articles for a Static Blog (and How To Visualize Embeddings)

Semantic search is a trending topic - and as of today a suprisingly simple concept. Instead of relying on keywords, this article shows how to use embeddings to capture meaning and suggest genuinely related posts. Along the way, I will show how to visualize embeddings in 2D and 3D, making the abstract mathematics of similarity tangible through interactive plots that visualize this blog itself. From theory to practice, the article walks through a full pipeline. It covers chunking content, generating embeddings with Ollama or OpenAI, storing them in PostgreSQL with pgvector, and finally integrating related-article boxes directly into Jekyll. Code snippets, schema definitions, and configuration examples make it easy to reproduce - or adapt for your own projects.

Some simple image filters by convolution in Python and OpenCL

This short article contains a summary about image filters and how one can realize them. It will contain a short reminder of how convolution works, what separable filters are and some example of image filters and their (most inefficient but direct) implementation in Python. In the end it will also show how one might naively implement convolution in OpenCL to increase performance in comparison to the naive Python implementations shown. This article provides one with a small playground to quickly try out separable image filter kernels and execute them on the GPU using pyopencl from a Python testing environment.

The most simple Difference of Gaussian image pyramid with OpenCL

The following blog article summarizes a quick implementation of a Difference of Gaussian (DoG) pyramid as approximation of a Laplacian of a Gaussian pyramid. It does not utilize optimizations so performance (around 10ms for a simple 6 octave pyramid) is rather low.

Also on this blog

GIT and SVN interoperability or moving from SVN to GIT without loosing history

How to use the GIT client to interoperate with a remote SVN server or how to migrate an SVN repository to GIT including it's full history.

A cat wheel for IsiHi

Curious if your cats would actually use one of those big wooden exercise wheels? We have been - and assembled as well as tested one for IsiHi, loyal servant to two feline overlords, and documented every step - from unboxing and construction to the cats' cautious inspection and in a later update eventual zoomies. With practical tips, charming photos, and honest insights, this guide shows exactly what to expect and how to make it work. Whether you're a do it yourself cat guardian or just love seeing clever engineering meet furry curiosity, this article delivers a detailed, entertaining guide to what might become your pet's favorite furniture - or at least their new favorite mystery. Spoiler: there’s Velcro, rollercoaster vibes, and a very skeptical cat.

Getting started with node-red for home/lab/process automation

A short tutorial on how to setup node red and configure the first flows to be used with sonoff basic switches and other data sources as well as XMPP/MQTT interfaces and some other ideas how one can use node red in various areas

k-Means clustering algorithm

Short summary about the k-means algorithm and it's usage

Data protection policy

Dipl.-Ing. Thomas Spielauer, Wien (webcomplains389t48957@tspi.at)

This webpage is also available via TOR at http://rh6v563nt2dnxd5h2vhhqkudmyvjaevgiv77c62xflas52d5omtkxuid.onion/

Valid HTML 4.01 Strict Powered by FreeBSD IPv6 support