Running local models

A step-by-step guide on how to run local models using Ollama

1. Download Ollama

   Visit Ollama official website at ollama.com↗ and download it.

2. Launch Ollama

3. Download and enable models

   Go to Kerlig Settings → Integrations → Ollama and enable the models you wish to use by toggling the switches.

   Please note that this will initiate the download process. Since models are typically several gigabytes in size, downloading them will take some time. You can close the settings window, the models will continue to download in the background.

Adding custom models

The list of Ollama models in the Settings displays only the most popular and newest models. However, Ollama supports 100+ models. Explore the full library at ollama.com/library↗. To use a model not listed in Settings, follow these steps:

1. Choose a model

   Visit ollama.com/library↗ and click on a model you wish to use.

2. Decide which version you want to use

   Models come with multiple versions (sizes), for example codegemma comes in 2b and 7b version, so the model names are codegemma:2b and codegemma:7b respectively.

   2b/7b/13b/33b etc. stands for billions of parameters used during training. The higher it is, the better the model and the more resources it uses.

3. Add custom model

   Go to Kerlig Settings → Integrations → Ollama, and in the Add Custom Model section, fill in the Model Display Name, e.g. Code Gemma, and the actual name (e.g. codegemma:7b). Then, click Add.

4. Download and enable models

   Once the model appears on the list, click the toggle. After the download is complete, the model will be ready for use.