Squadbase
Squadbase Docs

Ollama

Learn how to self-host an LLM with Ollama.

Squadbase lets you self-host LLMs with Ollama.
Select the model you want from the dashboard, deploy it, and Squadbase issues an API-ready URL that you can call directly from Python (or any HTTP client).

Self-hosting an LLM

Deploy an LLM

Go to the LLM tab on the dashboard to open the Create LLM screen.

Enter any LLM Name and choose a Model Name. Create LLM

Currently available models:

ModelParameters
deepseek-r11.5 b, 7 b, 8 b, 14 b
llama3.28 b
phi414 b
qwen0.5 b, 1.8 b, 4 b, 7 b, 14 b

Verify the model

Select your newly created LLM under the LLM tab. LLM detail

In Logs, the status should read Deployment Succeeded when the model is ready.
If it’s still in progress, wait until deployment finishes.

The App URL is the endpoint Squadbase hosts for your LLM.
Combine this URL with your Squadbase API Key to send requests.

Send a request

Use the App URL and your API Key to query the LLM.

Below are Python and cURL examples.

pip install langchain-ollama
from langchain_ollama import ChatOllama
 
llm = ChatOllama(
    model="{MODEL_NAME_YOU_DEPLOYED}",
    base_url="{YOUR_SQUADBASE_LLM_APP_URL}",
    client_kwargs={
        "headers": {
            "x-api-key": "{YOUR_SQUADBASE_API_KEY}",
        }
    },
)
 
for token in llm.stream("Hello"):
    yield token.content

Docker

Learn how to deploy with Docker.

squadbase.yml

Learn how to configure `squadbase.yml`.

On this page