If you are building autonomous agents, offline workflows, or cognitive automation pipelines with local LLMs, you have undoubtedly run into the single most frustrating bottleneck: format inconsistency.
Picture this: you write a Python script asking a local model like Llama 3 or Qwen 2.5 to extract structured data from unstructured daily notes — say, logging personal expenses directly into a local SQLite database or your Obsidian vault. What your application needs is clean, deterministic JSON with predictable keys and types. But what the model often gives you is:
Conversational chatter: "Sure! Here is the JSON data you requested..."*
* Markdown wrappers: ``json ... `` code fences that break downstream parsers.
* Hallucinated keys, missing quotes, or numbers converted into arbitrary strings.
The immediate consequence? A catastrophic json.loads() failure triggering a JSONDecodeError, halting your automated pipeline dead in its tracks. Even with aggressive system prompts like "Reply ONLY in valid JSON. Never include explanations or greetings," statistical models will occasionally wander off track.
Fortunately, that era is over. With Ollama's native Structured Outputs paired with Python's Pydantic library, you can now guarantee that every single character emitted by your local LLM conforms strictly to your predefined data schema — running 100% offline with zero cloud fees and absolute data privacy.
---
### How Structured Outputs Work Under the Hood
Historically, getting an LLM to generate JSON relied on "prompting and hoping" — the model produced tokens based on raw probabilities, and you used regexes or retry loops to clean up the mess.
Ollama's Structured Outputs feature works at a fundamentally deeper level: the inference engine. It uses a technique known as Constrained Decoding (or Grammar-constrained Sampling):
1. You define your data schema in Python using standard Pydantic models.
2. Pydantic generates a standard JSON Schema, which is passed directly to Ollama.
3. Ollama translates that schema into grammar constraints enforced by its underlying engine.
4. During inference, at every single token generation step, the engine masks out any tokens that would violate the grammar. Any token that does not conform to the schema is assigned a probability of zero.
As a result, the model physically cannot output conversational filler, cannot forget closing braces, and cannot misspell property names. It is mathematically constrained to valid output.
---
### Prerequisites & Environment Setup
To get started, make sure your local Ollama instance is up to date, and install the official Python SDK along with Pydantic:
pip install ollama pydantic
For reliable and fast structured generation, modern lightweight instruction-tuned models like Qwen 2.5 (7B) or Llama 3.1 / 3.2 work brilliantly. Download your model locally:
ollama pull qwen2.5:7b
---
### Practical Walkthrough: Extracting Personal Expense Records
Let's build a real-world script that extracts financial transaction records from messy, unstructured notes into strongly typed Python objects.
First, define your data schema using Pydantic, incorporating Enum to restrict allowable categories:
from enum import Enum
from typing import List
from pydantic import BaseModel, Field
import ollama
# 1. Define allowable expense categories
class ExpenseCategory(str, Enum):
FOOD = "Food & Dining"
TRANSPORT = "Transportation"
UTILITIES = "Utilities & Bills"
ENTERTAINMENT = "Entertainment"
HEALTH = "Healthcare"
OTHER = "Other"
# 2. Define the schema for individual expense items
class ExpenseItem(BaseModel):
merchant: str = Field(description="Name of the vendor or recipient")
amount: float = Field(description="Transaction amount in currency")
category: ExpenseCategory = Field(description="Expense category")
is_subscription: bool = Field(description="Whether this is a recurring subscription")
# 3. Define the top-level report schema
class ExpenseReport(BaseModel):
summary: str = Field(description="Concise one-sentence summary of the transactions")
total_amount: float = Field(description="Total sum of all expenses")
items: List[ExpenseItem] = Field(description="List of all extracted expense items")
---
### Executing the Request with Type Safety
Now, feed your unstructured text into ollama.chat(), passing ExpenseReport.model_json_schema() into the format argument:
raw_note = """
Yesterday around noon I grabbed a quick pork rice lunch at Somsri Diner for $8.50.
In the evening, took the metro back home for $2.50.
Late at night, my card was charged for my monthly Netflix subscription at $15.99.
"""
response = ollama.chat(
model="qwen2.5:7b",
messages=[
{
"role": "user",
"content": f"Extract all expense items from the following note:\n{raw_note}"
}
],
format=ExpenseReport.model_json_schema(), # Enforces the strict JSON Schema
options={"temperature": 0.0} # Zero temperature for maximum deterministic precision
)
# Parse the raw response directly into your Pydantic object
report = ExpenseReport.model_validate_json(response['message']['content'])
# Inspect the validated, typed data
print(f"📊 Summary: {report.summary}")
print(f"💰 Total Amount: ${report.total_amount:.2f}\n")
for item in report.items:
print(f"- Merchant: {item.merchant} | Amount: ${item.amount:.2f} | Category: {item.category.value} | Recurring: {item.is_subscription}")
---
### The Output: Clean, Predictable, and Error-Free
Running this code produces cleanly parsed objects accessible via standard dot notation:
📊 Summary: Daily expenses consisting of lunch, public transit, and a streaming subscription.
💰 Total Amount: $26.99
- Merchant: Somsri Diner | Amount: $8.50 | Category: Food & Dining | Recurring: False
- Merchant: Metro | Amount: $2.50 | Category: Transportation | Recurring: False
- Merchant: Netflix | Amount: $15.99 | Category: Entertainment | Recurring: True
Notice how the model accurately computed the total, matched the categories to our predefined ExpenseCategory enum values, and correctly flagged the Netflix subscription as recurring — without writing a single line of string cleaning code or regex.
---
### Why Structured Outputs Matter for Private Brain OS
1. Absolute Data Privacy (100% Offline): Sensitive financial records, medical notes, or proprietary client data never leave your local hardware. You get the structured intelligence of modern LLMs without feeding corporate cloud telemetry.
2. Deterministic Pipeline Reliability: Eliminates parsing crashes completely. Whether you are piping data into SQLite, writing frontmatter in Obsidian notes, or triggering local home automation webhooks, your pipeline will never crash on malformed JSON.
3. Optimized Inference Speed: By constraining tokens to the exact schema, the model skips unnecessary conversational preambles and explanatory text, saving compute cycles and returning responses significantly faster.
---
### Conclusion
Local AI truly evolves from a conversational novelty into a dependable Cognitive Automation Engine when you gain deterministic control over its output. By combining Ollama's grammar-level constraints with Pydantic's expressive data modeling, you turn open-weight local models into powerful, private data processors ready for any production workflow.