Use a local LLM for SQL suggestions

If you want an assistant to draft SQL without sending prompts to a hosted API, run a model locally behind an OpenAI-compatible endpoint. Ollama, LM Studio and llama.cpp’s server all expose /v1/chat/completions on localhost, so any client that lets you set a base URL can use them. A 7B-class coding model on a recent laptop is enough for “write a query that…” requests. It is not enough to trust the result without reading it.

Step 1: pick a runtime

Runtime	Default base URL	Notes
Ollama	`http://localhost:11434/v1`	CLI and background service; pulls models by name
LM Studio	`http://localhost:1234/v1`	Desktop app; start the local server from its developer view
llama.cpp `llama-server`	`http://localhost:8080/v1`	Single binary; you supply a GGUF file

All three ignore the API key by default, so the key can stay empty.

Step 2: pull a coding model

With Ollama:

ollama pull qwen2.5-coder:7b
ollama list

Code-tuned models follow “return one query in a code block” instructions more reliably than general chat models of the same size. Bigger models give better SQL and need more memory; try the 7B size first and move up if the answers are sloppy.

With llama.cpp, point the server at a downloaded GGUF file:

llama-server -m ./models/qwen2.5-coder-7b-instruct-q4_k_m.gguf --port 8080

Step 3: test the endpoint with curl

Check the endpoint before you configure any client. Otherwise you are debugging two things at once:

curl -s http://localhost:11434/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "qwen2.5-coder:7b",
    "messages": [
      {"role": "system", "content": "Answer with one MySQL query in a fenced code block."},
      {"role": "user", "content": "Orders per day for the last 7 days from table orders(created_at)"}
    ],
    "stream": false
  }' | jq -r '.choices[0].message.content'

If you get a model-not-found error, the model value must match the name the runtime lists exactly, including the tag.

Step 4: give the model names, not data

The model can only guess table and column names it has not seen. Giving it the names of your tables makes the answers much better. Giving it rows or credentials is never necessary for writing a query. Decide per prompt how much context to share, and keep it to names.

Even with a local model, treat the output as a draft:

read every generated query before running it,
run it against a read-only account,
watch for invented columns. Small models produce plausible names confidently.

Step 5: keep the endpoint on loopback

localhost traffic never leaves the machine. If you move the model to another box on your network, put HTTPS in front of it (a reverse proxy with a certificate), because prompts and any API key would otherwise cross the network in plain text.

Doing this in DevOpsPad

Open Settings, find the AI provider section (“Your copilot, your choice.”), and fill in:

API base URL: http://localhost:11434/v1 (the field’s placeholder is the Ollama default)
Model: qwen2.5-coder:7b, or whatever name your runtime lists
API key: leave empty for local providers

Then Save provider. DevOpsPad allows plain HTTP only for loopback addresses (localhost, 127.0.0.1, ::1). Any other host must use HTTPS. It does not follow redirects. Each request is a single non-streaming call to /chat/completions with a 60-second timeout, and prompts can be up to 16,000 characters.

By default only your prompt and the engine name (MySQL, Redis and so on) are sent. Include resource names is off until you enable it, and it adds only the visible resource names, never credentials or result rows. Each prompt is independent, so earlier chat messages are not sent as context. Suggested queries appear with Insert into editor, and nothing runs until you run it. Details: local models, configure an AI provider and what the AI sends. For why the design looks like this, read why the AI never sees your credentials.