Setting Parameters like Context Length and Temperature in Ollama Models

10 Aug 2025 - tsp
Last update 10 Aug 2025
Reading time 2 mins

This guide explains (or rather takes a note for myself) how to modify parameters for Ollama models - such as context length, temperature, and more - using a custom MODELFILE. Many models available via Ollama come with relatively small context windows (often around 4k tokens) and fixed sampling parameters like a temperature of 0.8. By creating your own model definition, you can override these defaults.

Viewing and Editing a MODELFILE

You can view the MODELFILE of any installed model with:

ollama show --modelfile <model-name>

A MODELFILE gives you control over various aspects of a model. You can set core parameters such as:

num_ctx, which determines the context size in tokens
temperature, which controls output randomness (with 0 producing deterministic responses and higher values making them more creative)
seed, which fixes the random seed to produce reproducible output

You can also adjust sampler behavior through options like top_p, min_p, maximum prediction length, and repetition penalty. In addition, you can define prompts and adapters by specifying:

a TEMPLATE that matches the training format
a SYSTEM prompt to shape the models behavior
an ADAPTER to attach LoRA adapters that expand or change the model’s capabilities

For full parameter details, see Ollama’s MODELFILE documentation.

Example: Increasing Context and Temperature

Suppose you want to boost creativity and allow much longer context on qwen2.5-coder:32b:

FROM qwen2.5-coder:32b
PARAMETER num_ctx 128000
PARAMETER temperature 1.2

Save this as MY_MODELFILE and run:

ollama create qwen2.5-coder:32b_128k --file MY_MODELFILE

This creates a new model variant locally without duplicating the base weights.

Example: Custom System Prompt and LoRA Adapter

You can also set a system prompt and add a LoRA adapter:

FROM llama3.1:70b
PARAMETER num_ctx 32768
SYSTEM "You are a concise assistant specializing in data analysis."
ADAPTER /path/to/finance-lora

Considerations

Hardware usage: Larger num_ctx increases VRAM/RAM usage massively and increases inference time.
Model training limits: Increasing context length beyond the training maximum can degrade output quality. Always check the models capabilities.
Template compatibility: If you change TEMPLATE, ensure it matches the model’s original formatting to avoid degraded performance. Usually you want to use the same model as has been used during training.

By customizing a MODELFILE, you can fine-tune how Ollama models behave for your specific tasks, balancing performance, creativity, and context size to your needs.