Integrating LLM with Existing Database Systems: Best Practices

I recently shipped a natural language interface for an internal SQLite analytics database. Product managers needed ad-hoc revenue reports, but they did not want to write SQL. The agent inspects the live schema, generates read-only queries, executes them, and returns a plain English summary. I run the LLM layer on Oxlo.ai because the flat per-request pricing means I can pass the full schema context on every call without costs scaling with token count.

What you'll need

• Python 3.10 or newer
• An Oxlo.ai API key from https://portal.oxlo.ai
• The OpenAI SDK: pip install openai
• A local SQLite database file (we will create one in Step 1)

Step 1: Seed an existing database schema

Most teams already have a live database, so I will create a minimal e-commerce schema that mimics a real production system. It contains products and orders with a foreign key relationship. I insert a small set of rows so we can verify joins and aggregations. You can replace this with a connection to your existing PostgreSQL or MySQL instance later by swapping the sqlite3 connector for SQLAlchemy or psycopg2.

import sqlite3

conn = sqlite3.connect("sales.db")
cursor = conn.cursor()

cursor.executescript("""
CREATE TABLE IF NOT EXISTS products (
    id INTEGER PRIMARY KEY,
    name TEXT NOT NULL,
    category TEXT,
    price REAL
);

CREATE TABLE IF NOT EXISTS orders (
    id INTEGER PRIMARY KEY,
    product_id INTEGER,
    quantity INTEGER,
    order_date TEXT,
    FOREIGN KEY (product_id) REFERENCES products(id)
);

INSERT INTO products (id, name, category, price) VALUES
(1, 'Wireless Headphones', 'Electronics', 249.00),
(2, 'Mechanical Keyboard', 'Electronics', 189.00),
(3, 'USB-C Hub', 'Electronics', 79.00),
(4, 'Standing Desk', 'Furniture', 550.00);

INSERT INTO orders (id, product_id, quantity, order_date) VALUES
(1, 1, 10, '2024-03-05'),
(2, 2, 8, '2024-03-12'),
(3, 1, 15, '2024-03-20'),
(4, 3, 12, '2024-03-25'),
(5, 4, 2, '2024-04-01');
""")

conn.commit()
conn.close()

Step 2: Configure the Oxlo.ai client

Oxlo.ai is fully OpenAI SDK compatible, which makes integration trivial. I set the base_url to https://api.oxlo.ai/v1 and pull the API key from an environment variable. There are no custom clients to learn. I typically use llama-3.3-70b for this stage because it follows structured instructions reliably, though Oxlo.ai also offers qwen-3-32b and deepseek-v3.2 if you want to experiment with coding-specialized models.

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.environ.get("OXLO_API_KEY", "YOUR_OXLO_API_KEY")
)

Step 3: Define the system prompt

The system prompt is the most critical part of the integration. It acts as the contract between the LLM and the database. I embed the exact column names, types, and relationships so the model does not hallucinate tables. I also force JSON output by including explicit formatting rules and an example. This removes ambiguity and makes parsing trivial. I keep the prompt in a dedicated constant so I can diff it in version control alongside the application code.

SYSTEM_PROMPT = """You are a read-only SQL agent for an e-commerce database.
Your job is to translate user questions into valid SQLite SELECT statements.

Schema:
- products(id INTEGER, name TEXT, category TEXT, price REAL)
- orders(id INTEGER, product_id INTEGER, quantity INTEGER, order_date TEXT)

Relationships:
- orders.product_id references products.id

Rules:
1. Only generate SELECT queries. Never generate INSERT, UPDATE, DELETE, DROP, ALTER, or CREATE.
2. Use proper SQLite syntax.
3. When joining tables, use explicit JOINs.
4. Return a single JSON object with exactly two keys: "sql" and "explanation".
5. Do not wrap the JSON in markdown code fences.

Example:
{"sql":"SELECT * FROM products LIMIT 5;","explanation":"Returns the first 5 products."}
"""

Step 4: Generate SQL with Llama 3.3 70B

This function sends the user question and schema context to Oxlo.ai. I ask for a JSON object containing both the SQL and a short explanation. Parsing structured text from an LLM can be brittle, so I strip whitespace and use json.loads directly. If the model ever returns malformed JSON, the caller catches the exception and surfaces it. Using Oxlo.ai here is especially practical because long system prompts with full schema documentation do not inflate the per-request cost.

import json

def generate_sql(question):
    from openai import OpenAI
    import os

    client = OpenAI(
        base_url="https://api.oxlo.ai/v1",
        api_key=os.environ.get("OXLO_API_KEY", "YOUR_OXLO_API_KEY")
    )

    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": question},
        ],
    )

    raw = response.choices[0].message.content
    parsed = json.loads(raw.strip())
    return parsed["sql"], parsed["explanation"]

Step 5: Execute queries safely

Never execute raw LLM output against a production database without guards. I run a simple keyword check to block any statement that is not a SELECT. Then I open a connection to sales.db, set row_factory to sqlite3.Row so I get dictionary-like objects, and fetch all results. I close the connection immediately to avoid leaking file descriptors. If you are connecting to a remote database, this is where you would inject connection pooling or read-only user credentials.

import sqlite3

def execute_query(sql):
    forbidden = ["insert", "update", "delete", "drop", "alter", "create"]
    if any(keyword in sql.lower() for keyword in forbidden):
        raise ValueError("Only SELECT statements are allowed.")

    conn = sqlite3.connect("sales.db")
    conn.row_factory = sqlite3.Row
    cursor = conn.cursor()
    cursor.execute(sql)
    rows = [dict(row) for row in cursor.fetchall()]
    conn.close()
    return rows

Step 6: Format the answer

Raw JSON arrays are not friendly to stakeholders. I send the rows back to Oxlo.ai with a concise formatting prompt. The model receives the original question, the executed SQL, and the serialized results. It then writes a two to three sentence summary with specific numbers. This second LLM call is lightweight, and because Oxlo.ai charges per request, it costs the same whether the result set is ten rows or ten thousand.

def format_answer(question, sql, rows):
    from openai import OpenAI
    import os
    import json

    client = OpenAI(
        base_url="https://api.oxlo.ai/v1",
        api_key=os.environ.get("OXLO_API_KEY", "YOUR_OXLO_API_KEY")
    )

    prompt = f"""Question: {question}
SQL: {sql}
Results: {json.dumps(rows, indent=2)}

Summarize the results in 2-3 sentences for a non-technical user. Include specific numbers."""

    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": "You are a helpful data analyst."},
            {"role": "user", "content": prompt},
        ],
    )

    return response.choices[0].message.content

Step 7: Wire everything together

The orchestration layer ties the pipeline together. It calls generate_sql, validates and executes the query, then passes the results to format_answer. I wrap the entire flow in a broad try/except so that SQL syntax errors, JSON parse failures, or guard violations all return a clean dictionary with an error field instead of crashing the process. This makes the agent safe to embed inside a FastAPI route or a Slack bot.

def ask_database(question):
    try:
        sql, explanation = generate_sql(question)
        rows = execute_query(sql)
        answer = format_answer(question, sql, rows)
        return {
            "question": question,
            "sql": sql,
            "explanation": explanation,
            "row_count": len(rows),
            "answer": answer
        }
    except Exception as e:
        return {"error": str(e)}

Run it

I test the agent with a question that requires a JOIN, a WHERE clause, and an aggregation. This verifies that the schema was understood and that the guardrails allow legitimate SELECT statements. Run the script from your terminal with python agent.py.

if __name__ == "__main__":
    import json

    result = ask_database("What were the top 3 products by revenue in March 2024?")
    print(json.dumps(result, indent=2))

Example output:

{
  "question": "What were the top 3 products by revenue in March 2024?",
  "sql": "SELECT p.name, SUM(o.quantity * p.price) as revenue FROM orders o JOIN products p ON o.product_id = p.id WHERE o.order_date >= '2024-03-01' AND o.order_date < '2024-04-01' GROUP BY p.name ORDER BY revenue DESC LIMIT 3",
  "explanation": "Joins orders and products, filters for March 2024, calculates revenue per product, and returns the top 3.",
  "row_count": 3,
  "answer": "The top 3 products by revenue in March 2024 were Wireless Headphones with $6,225, Mechanical Keyboard with $1,512, and USB-C Hub with $948."
}

Next steps

Swap the sqlite3 connection for your existing database driver. If you want to reduce latency for repeated questions, cache embeddings of prior queries using Oxlo.ai's bge-large model and skip the LLM round-trip on exact semantic matches. You can also add a second guard that runs EXPLAIN QUERY PLAN before execution to catch Cartesian products. Because Oxlo.ai uses request-based pricing, you can stuff full schema documentation, sample rows, and business logic into the system prompt without the runaway token bills you see on usage-based providers. Check https://oxlo.ai/pricing for current plans.