Guaranteed 15% off your current AI inference bill for team spending up to $20000 / month.

Book a call →
Back to Blogs
Engineering

Building Effective Agentic Workload Systems

We are building a meeting transcript action-item agent that turns raw notes into structured tasks with owners and deadlines. It targets engineering managers...

Building Effective Agentic Workload Systems

We are building a meeting transcript action-item agent that turns raw notes into structured tasks with owners and deadlines. It targets engineering managers and PMs who currently waste hours manually parsing meeting text into Jira or Asana. The agent runs entirely against Oxlo.ai, using a flat per-request pricing model that keeps costs predictable even when transcripts grow long. Unlike token-based providers, Oxlo.ai does not penalize you for stuffing full conversation history into the context window. That makes the agentic pattern, where the model reasons over large unstructured inputs, cheap enough to run on every meeting your team records.

What you'll need

Python 3.10 or newer, the official OpenAI SDK, and an active Oxlo.ai API key. Install the SDK with pip and then generate a key from the Oxlo.ai portal. I keep my key in an environment variable named OXLO_API_KEY so it never leaks into shell history or git. You will also want a model that supports tool use. I use Llama 3.3 70B for this tutorial because it is reliable and fast, but you can swap in Qwen 3 32B or Kimi K2.6 without changing any code.

Step 1: Bootstrap the Oxlo.ai client

First, instantiate the OpenAI client pointing at Oxlo.ai. Because Oxlo.ai is fully OpenAI SDK compatible, this is the only networking code we need. I pull the API key from the environment and set a ten-second timeout so that hung requests fail fast in CI. You can drop this client into existing test suites that mock the OpenAI interface without any changes.

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.getenv("OXLO_API_KEY", "YOUR_OXLO_API_KEY"),
    timeout=10.0
)

Step 2: Lock down the system prompt

The system prompt is the single most important control surface. I constrain the model to emit only tool calls, never conversational filler, and I explicitly forbid hallucinating owners or dates that do not appear in the text. I also tell the agent to preserve the exact name spelling found in the transcript. This matters because general-purpose models can be chatty if you let them. A strict system prompt keeps the output deterministic and easy to validate downstream. If you find the model still adding commentary, tighten the rules and add a penalty phrase like "Do not explain your reasoning."

SYSTEM_PROMPT = """You are a structured extraction agent.
Your job is to read a meeting transcript and extract every concrete action item.
For each action item, identify:
- owner: the person assigned, exactly as named in the transcript. If unclear, use "Unassigned".
- task: a concise, one-sentence description of the work.
- due_date: an ISO-8601 date if explicitly mentioned, otherwise null.
Rules:
- Do not summarize. Extract verbatim obligations.
- Do not invent names or dates not present in the text.
- Preserve exact name spelling as it appears in the transcript.
- Emit your findings by calling the create_task tool once per action item.
"""

Step 3: Define the tool schema

Instead of parsing free text, we force the model to emit JSON via function calling. The schema below declares a single tool, create_task, with strict properties. Oxlo.ai supports function calling across its entire Llama 3, Qwen 3, and Kimi K2 catalog, so this pattern is portable. Using tools also gives us validation for free. If the model omits a required field, the API raises a formatting error that we can catch and retry. I keep the schema minimal on purpose. Extra fields invite hallucination.

tools = [
    {
        "type": "function",
        "function": {
            "name": "create_task",
            "description": "Record an extracted action item.",
            "parameters": {
                "type": "object",
                "properties": {
                    "owner": {
                        "type": "string",
                        "description": "Person responsible, exactly as named in the transcript."
                    },
                    "task": {
                        "type": "string",
                        "description": "Concise description of the work."
                    },
                    "due_date": {
                        "type": ["string", "null"],
                        "description": "ISO-8601 date or null."
                    }
                },
                "required": ["owner", "task", "due_date"]
            }
        }
    }
]

Step 4: Build the extraction runner

We send the transcript as a user message and let the model decide how many tool calls to make. The response object follows the standard ChatCompletion shape, so tool_calls lives on the assistant message. I iterate over every call, parse the JSON arguments, and accumulate them in a list. I set temperature to 0.1 because extraction is not a creative task. Lower temperatures reduce variance across runs, which is important when you are diffing outputs in regression tests. In production you would persist these tasks to a database or queue, but here we return them in memory so the caller can inspect or filter them before committing.

import json

def extract_tasks(transcript: str):
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": transcript}
    ]

    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=messages,
        tools=tools,
        tool_choice="auto",
        temperature=0.1
    )

    tasks = []
    message = response.choices[0].message

    if message.tool_calls:
        for call in message.tool_calls:
            if call.function.name == "create_task":
                args = json.loads(call.function.arguments)
                tasks.append(args)

    return tasks

Step 5: Deduplicate across chunks

When we split a long transcript, the same action item can appear at the boundary of two chunks. I add a lightweight deduplicator that hashes the normalized task string. If two tasks share the same owner and an identical normalized description, we keep only the first occurrence. This prevents duplicate Jira tickets downstream without pulling in heavy fuzzy-matching libraries. You could upgrade this later with Levenshtein distance if your transcripts are noisy, but exact matching covers the common case where the model is consistent.

def normalize_task(t: dict) -> str:
    return f"{t['owner'].lower().strip()}|{t['task'].lower().strip()}"

def deduplicate(tasks: list) -> list:
    seen = set()
    out = []
    for t in tasks:
        key = normalize_task(t)
        if key not in seen:
            seen.add(key)
            out.append(t)
    return out

Step 6: Handle long transcripts without cost surprises

Meeting transcripts can run to tens of thousands of tokens. On token-based providers, that linearly increases cost, which makes long-context agentic workloads expensive to run in production. Oxlo.ai charges a flat rate per request, so a long transcript costs the same as a short one. That pricing structure directly enables the agentic pattern of dumping large context windows into the model and letting it reason across the full history. For current plan details, see https://oxlo.ai/pricing. For very long text that exceeds context limits, chunk it into paragraph-aligned windows and deduplicate the results. The helper below splits on blank lines because paragraph boundaries usually coincide with topic shifts. If a single paragraph is longer than the limit, fall back to sentence boundaries.

def chunk_text(text: str, max_chars: int = 12000):
    paragraphs = text.split("\n\n")
    chunks = []
    current = ""

    for p in paragraphs:
        if not current:
            current = p
        elif len(current) + len(p) > max_chars:
            chunks.append(current.strip())
            current = p
        else:
            current += "\n\n" + p
    if current:
        chunks.append(current.strip())
    return chunks

def extract_all_tasks(transcript: str):
    all_tasks = []
    for chunk in chunk_text(transcript):
        all_tasks.extend(extract_tasks(chunk))
    return deduplicate(all_tasks)

Run it

Here is a realistic transcript with multiple speakers, implicit assignments, and mixed deadlines. We run it through the agent and print the structured output. Notice that Alex's task has no explicit due date, so the model correctly emits null rather than guessing. I have run this exact transcript dozens of times against Llama 3.3 70B on Oxlo.ai and the results are stable. If you see variance, lower the temperature or add a few-shot example inside the system prompt.

if __name__ == "__main__":
    transcript = """
Sarah: We need to migrate the auth service to the new OIDC provider before the compliance audit.
Tom: I can handle the terraform changes, but I need the client IDs by Friday.
Sarah: I'll get those to you by June 10th. Tom, can you have the infra ready for review by June 14th?
Tom: Yes. Alex, please update the runbook once the cutover is done.
Alex: Will do, but I need a heads-up 24 hours before go-live.
Sarah: Let's schedule the go-live for June 20th. I'll send the calendar invite today.
"""

    tasks = extract_all_tasks(transcript)
    for t in tasks:
        print(t)

When I run this against Oxlo.ai, the output looks like this:

{'owner': 'Sarah', 'task': 'Get client IDs to Tom by June 10th', 'due_date': '2024-06-10'}
{'owner': 'Tom', 'task': 'Have infra ready for review by June 14th', 'due_date': '2024-06-14'}
{'owner': 'Alex', 'task': 'Update the runbook once auth service cutover is done', 'due_date': None}
{'owner': 'Sarah', 'task': 'Send calendar invite for go-live on June 20th', 'due_date': '2024-06-20'}

Next steps

Two concrete ways to harden this into production.

First, add a human-in-the-loop approval step. Store extracted tasks in a staging table and POST to a Slack channel with Approve and Reject buttons. Only push to Jira after confirmation. This eliminates any remaining hallucination risk and gives your team a chance to correct ambiguous assignments before they become tickets.

Second, replace the chunking heuristic with a semantic splitter. Embed paragraphs using the Oxlo.ai embeddings endpoint and merge adjacent chunks with high cosine similarity. This preserves context boundaries better than character counts. Because Oxlo.ai uses request-based pricing, you can afford the extra embedding calls without token math. If you need stronger reasoning on dense technical transcripts, swap Llama 3.3 70B for Kimi K2.6 or DeepSeek V3.2.

Ready to build with Oxlo.ai?

Get started building high-performance AI inference applications today.

Get started
Ox Assistant
Online
OxBot
OxBot

Hi there! Try our cost calculator to see what you'd save with Oxlo.ai.