Ollama Setup Guide

Here is the guide for using Ollama models in 16x Prompt.

1. Go to Ollama
2. Follow the instructions to install Ollama on your machine
3. Once installed, run the following command to download the models, for example deepseek-r1:14b:

ollama pull deepseek-r1:14b

Note: Running deepseek-r1:14b requires at least 16GB of RAM on a MacBook Pro. You can see the list of available models from Ollama Models.

4. Ollama defaults to 2048 context window size, which is too small for most coding tasks. You can adjust the context window size to 8192 by running the following command:

ollama run deepseek-r1:14b
/set parameter num_ctx 8192
/save deepseek-r1:14b-8k

5. This will create a new model called deepseek-r1:14b-8k with the context window size of 8192.
6. Serve the model by running the following command:

ollama serve

7. Open 16x Prompt and click on the API model settings button in the bottom right corner.
8. Go to "3rd Party API / Ollama Configuration" tab.
9. Enter http://127.0.0.1:11434/v1 into the "API Endpoint / Base URL" field.
10. Enter model name you want to use, for example deepseek-r1:14b-8k into the "Model ID" field.
11. Enter ollama (or anything else) into the "API Type" field.
12. Click on "Close".

You should now be able to use models from Ollama in 16x Prompt, and see the logs from Ollama in the console.