What you'll need
- Python 3.10 or newer installed locally.
- The OpenAI SDK:
pip install openai. - An Oxlo.ai API key from https://portal.oxlo.ai.
- A sample paragraph to test against. You can use the string provided in the final step.
Step 1: Configure the Oxlo.ai client
I keep my API key in an environment variable so it never touches source control. Oxlo.ai exposes a fully OpenAI-compatible endpoint, so the only setup difference is the base_url. I use llama-3.3-70b here because it follows formatting instructions tightly and outputs clean JSON, but you can swap in qwen-3-32b or deepseek-v3.2 later without changing any other code.
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key=os.environ.get("OXLO_API_KEY")
)
# Quick connectivity check
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[{"role": "user", "content": "Say OK"}],
)
print(response.choices[0].message.content)Step 2: Define the system prompt
The system prompt is the contract between our code and the model. I restrict extraction to four entity types and demand character offsets so we can highlight the original text later. Many tutorials only return the entity string, which forces you to search for it again and breaks when the same name appears twice. By requiring exact start and end indices, we get anchors that survive duplication. I also tell the model to skip markdown fences and explanations, because JSON mode alone does not guarantee schema compliance.
SYSTEM_PROMPT = """You are a precise entity recognition engine.
Extract all named entities from the user text and return them as a JSON object.
The object must contain a single key "entities" whose value is a list.
Each item must be an object with these exact keys:
"text": the exact substring from the input,
"type": one of PERSON, ORGANIZATION, LOCATION, DATE,
"start": integer index of the first character,
"end": integer index of the last character plus one.
Do not include any explanation, markdown formatting, or text outside the JSON object.
If no entities are found, return {"entities": []}."""Step 3: Build the extraction function
This wrapper sends the raw text to Oxlo.ai with JSON mode enabled. I set temperature to 0.1 to keep the output deterministic. The user message contains only the raw text, because the system prompt already sets the context. The function parses the response and returns a plain Python list that the rest of our pipeline can consume.
import json
def extract_entities(text: str) -> list:
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": text},
],
response_format={"type": "json_object"},
temperature=0.1,
)
raw = response.choices[0].message.content
data = json.loads(raw)
return data.get("entities", [])Step 4: Add defensive parsing
In production, models occasionally return markdown fences or extra whitespace that breaks json.loads. I add a small cleaning helper to strip triple backticks and an optional json label. If parsing still fails, I return an empty list so the caller can log the raw response and decide what to do. Because Oxlo.ai charges per request rather than per token, resending a long document after a parse error does not inflate the bill the way it would on a token-based provider. That makes a simple retry strategy economically viable.
import re
def clean_json(raw: str) -> str:
raw = raw.strip()
if raw.startswith("```"):
raw = re.sub(r"^```(?:json)?\s*", "", raw)
raw = re.sub(r"\s*```$", "", raw)
return raw
def extract_entities(text: str) -> list:
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": text},
],
response_format={"type": "json_object"},
temperature=0.1,
)
raw = response.choices[0].message.content
try:
data = json.loads(clean_json(raw))
return data.get("entities", [])
except json.JSONDecodeError:
return []Step 5: Render highlights from the offsets
Exact offsets make visualization trivial. I write a small renderer that iterates over entities in reverse order so inserting HTML tags does not shift the indices of earlier matches. This lets us generate annotated output without regular expression trickery.
def highlight_entities(text: str, entities: list) -> str:
# Sort by start index descending so earlier offsets remain stable
for ent in sorted(entities, key=lambda x: x["start"], reverse=True):
start = ent["start"]
end = ent["end"]
label = ent["type"]
tagged = f'<mark title="{label}">{text[start:end]}</mark>'
text = text[:start] + tagged + text[end:]
return textRun it
Here is the complete script assembled, followed by the output it produces on a short sample sentence. I sanity-check the offsets by slicing the original string: sample[0:13] returns Alice Johnson, confirming the model gave us exact anchors.
import os
import json
import re
from openai import OpenAI
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key=os.environ.get("OXLO_API_KEY")
)
SYSTEM_PROMPT = """You are a precise entity recognition engine.
Extract all named entities from the user text and return them as a JSON object.
The object must contain a single key "entities" whose value is a list.
Each item must be an object with these exact keys:
"text": the exact substring from the input,
"type": one of PERSON, ORGANIZATION, LOCATION, DATE,
"start": integer index of the first character,
"end": integer index of the last character plus one.
Do not include any explanation, markdown formatting, or text outside the JSON object.
If no entities are found, return {"entities": []}."""
def clean_json(raw: str) -> str:
raw = raw.strip()
if raw.startswith("```"):
raw = re.sub(r"^```(?:json)?\s*", "", raw)
raw = re.sub(r"\s*```$", "", raw)
return raw
def extract_entities(text: str) -> list:
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": text},
],
response_format={"type": "json_object"},
temperature=0.1,
)
raw = response.choices[0].message.content
try:
data = json.loads(clean_json(raw))
return data.get("entities", [])
except json.JSONDecodeError:
return []
def highlight_entities(text: str, entities: list) -> str:
for ent in sorted(entities, key=lambda x: x["start"], reverse=True):
start = ent["start"]
end = ent["end"]
label = ent["type"]
tagged = f'<mark title="{label}">{text[start:end]}</mark>'
text = text[:start] + tagged + text[end:]
return text
if __name__ == "__main__":
sample = "Alice Johnson works at Oxlo.ai in San Francisco since January 2024."
entities = extract_entities(sample)
print(json.dumps(entities, indent=2))
print(highlight_entities(sample, entities))Example output:
[
{
"text": "Alice Johnson",
"type": "PERSON",
"start": 0,
"end": 13
},
{
"text": "Oxlo.ai",
"type": "ORGANIZATION",
"start": 23,
"end": 30
},
{
"text": "San Francisco",
"type": "LOCATION",
"start": 34,
"end": 47
},
{
"text": "January 2024",
"type": "DATE",
"start": 54,
"end": 66
}
]
<mark title="PERSON">Alice Johnson</mark> works at <mark title="ORGANIZATION">Oxlo.ai</mark> in <mark title="LOCATION">San Francisco</mark> since <mark title="DATE">January 2024</mark>.Next steps
If you want to take this into production, consider two improvements. First, feed the extracted offsets into a PDF or HTML renderer to generate annotated documents without any fuzzy string matching. Second, if you are processing multilingual text or very long reports, swap the model to qwen-3-32b or kimi-k2.6. Because Oxlo.ai uses flat, request-based pricing, passing a 100,000-token white paper in a single API call costs the same as a one-sentence query. That removes the usual incentive to chunk or truncate long context, which simplifies pipeline design considerably. See https://oxlo.ai/pricing for plan details.
