Integrating LLM with Existing Engineering Systems: A Comprehensive Guide

In this guide we are building an on-call triage agent that ingests raw error logs, classifies severity, and writes structured incident tickets to a local JSON store. It plugs into any monitoring pipeline that can emit text, and it saves on-call engineers from manually copy-pasting stack traces into runbooks. I built this for my own team's Sentry webhook pipeline, and the core logic is what I will walk through below.

What you'll need

• Python 3.10 or newer. I developed this on 3.11.
• The OpenAI SDK installed with pip install openai. Oxlo.ai is fully OpenAI SDK compatible, so this is the only client library we need.
• An Oxlo.ai API key from https://portal.oxlo.ai. Oxlo.ai uses flat per-request pricing, which means a 50-line stack trace costs the same as a one-line error. For high-volume log pipelines, that predictability matters. See https://oxlo.ai/pricing for current plans.

Step 1: Set up the Oxlo.ai client

I start by instantiating the OpenAI client pointing at Oxlo.ai. Because Oxlo.ai is fully OpenAI SDK compatible, this is the only client I need for the entire project. There are no custom adapters and no extra dependencies. I keep the API key in an environment variable so it never hits disk in the repo, but for a quick test you can paste it directly into the string.

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.environ.get("OXLO_API_KEY", "YOUR_OXLO_API_KEY")
)

# Quick smoke test
response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[{"role": "user", "content": "Say OK"}],
)
print(response.choices[0].message.content)

Step 2: Scaffold the incident store

Instead of hitting a real Jira instance, I append structured tickets to a local JSONL file. This keeps the tutorial runnable without extra infrastructure, and the same dictionary can be forwarded to any issue tracker later. I choose JSONL because it is append-only and safe to write from concurrent workers without locking. Each ticket will carry fields that map cleanly to standard issue trackers: service, severity, summary, root_cause, and action.

import json
from datetime import datetime, timezone
from typing import Dict, Any

STORE_PATH = "incidents.jsonl"

def write_ticket(ticket: Dict[str, Any]) -> None:
    ticket["created_at"] = datetime.now(timezone.utc).isoformat()
    with open(STORE_PATH, "a", encoding="utf-8") as f:
        f.write(json.dumps(ticket, ensure_ascii=False) + "\n")

Step 3: Define the system prompt

The system prompt is the contract between our pipeline and the model. I keep it strict: expect raw logs, return only JSON, and adhere to our internal severity definitions. I do not use JSON mode here because the prompt itself constrains the output format tightly, and Llama 3.3 70B respects that boundary reliably on Oxlo.ai. If you later switch to a reasoning model like DeepSeek R1 for harder distributed system logs, you may want to add an explicit think step by step instruction before the JSON block.

SYSTEM_PROMPT = """You are an on-call triage assistant. Your job is to read a raw error log and produce a JSON object with exactly these keys:

- service: the microservice or module name inferred from the log
- severity: one of P1, P2, P3, or P4 based on customer impact (P1 = full outage, P4 = cosmetic)
- summary: a one-sentence description of the problem
- root_cause: a one-sentence hypothesis of what failed
- action: the immediate remediation step a human should take

Rules:
1. Output ONLY valid JSON. No markdown fences, no explanation.
2. If the service name is ambiguous, use "unknown".
3. If the log is not an error, set severity to P4 and summary to "Non-error log received"."""

Step 4: Build the triage function

This is the core integration point. The function takes a raw log string, sends it to Oxlo.ai with the system prompt, and returns a Python dictionary. I set temperature to 0.1 because severity classification is not a creative task, and I want deterministic output. I use Llama 3.3 70B here because it follows structured instructions reliably.

Because Oxlo.ai charges per request, I do not have to worry about a 200-line stack trace pushing me into a higher cost tier. Some providers bill by the token, which makes long logs expensive to classify. With Oxlo.ai, the cost stays flat no matter how verbose the traceback is. I also add a small sanitization step. Occasionally the model wraps JSON in markdown fences out of habit, so I strip triple backticks before parsing. This makes the pipeline robust against minor formatting drift.

import json
from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.environ.get("OXLO_API_KEY", "YOUR_OXLO_API_KEY")
)

def triage_log(raw_log: str) -> dict:
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": raw_log},
        ],
        temperature=0.1,
    )

    content = response.choices[0].message.content.strip()
    # Some models may return fenced JSON; strip fences if present
    if content.startswith("```"):
        content = content.split("\n", 1)[1].rsplit("```", 1)[0].strip()
    return json.loads(content)

Step 5: Wire the pipeline

Now I connect the ingestion side to the ticket store. In production this loop runs inside a Celery task or a FastAPI endpoint that receives webhooks from Sentry. For this tutorial, I will process a list of log strings and persist each ticket. The list simulates what you would pull from a Kafka topic, an SQS queue, or a log file tail. Each string is treated as an independent unit of work, so the loop is trivially parallelizable with a thread pool if throughput becomes a bottleneck.

I intentionally keep the example logs short so the tutorial is readable, but the same code handles multi-line Python tracebacks or Java heap dumps because Oxlo.ai does not penalize long inputs. If you are processing logs in a language other than English, Qwen 3 32B is a strong alternative on Oxlo.ai for multilingual reasoning without changing the client code.

logs = [
    """[ERROR] 2024-05-21T14:33:10Z payment-service Traceback (most recent call last):
  File "/app/payments.py", line 112, in charge
    gateway.timeout()
ConnectionError: upstream gateway unreachable after 30s""",
    """[WARN] 2024-05-21T14:35:02Z auth-service Deprecated token algorithm detected for client_id=12345""",
]

for log in logs:
    try:
        ticket = triage_log(log)
        write_ticket(ticket)
        print("Ticket created:", ticket["summary"])
    except Exception as e:
        print("Failed to triage log:", e)

Run it

After exporting OXLO_API_KEY and running the script, the terminal prints the summaries and the incidents.jsonl file contains one line per ticket. The first log triggers a P1 because a payment gateway timeout is a revenue-impacting outage. The second log is a warning about deprecated tokens, so it lands at P4. This kind of automatic severity tagging is what prevents alert fatigue in a busy Slack channel.

Here is what the terminal output looks like:

Ticket created: upstream gateway unreachable causing payment failure
Ticket created: Deprecated token algorithm detected for client_id=12345

The resulting JSON objects stored in incidents.jsonl:

{"service": "payment-service", "severity": "P1", "summary": "upstream gateway unreachable causing payment failure", "root_cause": "ConnectionError to upstream payment gateway after 30s timeout", "action": "Check gateway health and failover to secondary provider", "created_at": "2024-05-21T14:33:15+00:00"}
{"service": "auth-service", "severity": "P4", "summary": "Deprecated token algorithm detected for client_id=12345", "root_cause": "Client is using a deprecated signing algorithm", "action": "Notify client to migrate to RS256 before sunset date", "created_at": "2024-05-21T14:35:05+00:00"}

Next steps

Replace the local JSONL writer with a POST request to your issue tracker of choice, or stream the tickets into PagerDuty via their Events API v2. If you want to reduce false positives, add a second Oxlo.ai call that acts as a judge. Pass the original log and the generated ticket to a model like Kimi K2.6 and ask it to confirm or downgrade the severity before creation.

If your logs routinely exceed a few thousand lines, switch to deepseek-v3.2 or kimi-k2.6 on Oxlo.ai for stronger reasoning over long contexts. Because Oxlo.ai is fully OpenAI SDK compatible, changing the model string is the only edit required.