By default, when you chat with a Local LLM through an AI chat interface, it can only respond with plain text. If you want the AI to help you search for files on your machine, check system status, or pull data from a local SQLite database, you're stuck copy-pasting commands back and forth by hand.
But with the Function Calling (aka Tool Use) feature now available on Ollama, you can transform your Local LLM into a smart AI Agent that analyzes user intent and selectively invokes the right Python functions or local scripts at the right moment — all running 100% Offline on your personal computer.
This article walks you through the Function Calling architecture, a step-by-step setup guide, and production-ready Python code you can use immediately.
---
### What Is Function Calling? Why It's a Game-Changer for Local AI Agents
Function Calling is the process where the AI doesn't just generate a text response — it returns a structured JSON payload specifying which function to call and what parameters (arguments) to pass to it.
For example, if a user types: "Check how much disk space is left on my machine."
1. Instead of hallucinating an answer, the AI outputs a JSON call: {"name": "check_disk_space", "arguments": {}}
2. Your Python script receives that JSON, executes the actual disk-check command on the OS, and gets back a result like 150 GB free
3. The script feeds the result back to the AI, which summarizes it in natural language for the user
#### Privacy & Security Benefits
* The LLM doesn't execute code directly: The AI only sends the function name and parameters. Your Python script performs validation before running anything — preventing accidental file deletions or dangerous command execution.
* 100% local processing: Neither the prompt nor the function payload ever leaves your machine over the internet.
---
### Prerequisites & Models That Support Function Calling in Ollama
Before writing any code, make sure your environment has the following:
1. Ollama (v0.3.0 or later): Install the Ollama engine
2. Python 3.10+ and the Ollama Python SDK: Install via:
```bash
pip install ollama
```
3. Choose a model with strong Function Calling support: Modern models tested for high-accuracy tool-call extraction include:
* qwen2.5:7b or qwen2.5:14b (highly recommended for logic-heavy tasks and multilingual support)
* llama3.1:8b
* mistral:7b
Pull a model to your local machine:
ollama pull qwen2.5:7b
---
### Writing Python to Drive Ollama Function Calling (Step-by-Step)
Let's build a Python script that lets your Local AI fetch the current time and search local notes using Function Calling.
import json
import datetime
import ollama
# 1. Define the actual functions in Python
def get_current_time():
"""Get the current system time"""
return datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")
def search_local_notes(query: str):
"""Simulated function to search local notes"""
notes_db = {
"password": "Home WiFi password is PrivateBrain2026",
"project": "Private Brain OS project deadline is end of this month"
}
results = [v for k, v in notes_db.items() if query.lower() in k or query.lower() in v]
return results if results else ["No matching entries found"]
# 2. Define the Tool Schema to send to Ollama
tools_schema = [
{
"type": "function",
"function": {
"name": "get_current_time",
"description": "Retrieve the current system date and time",
"parameters": {
"type": "object",
"properties": {},
"required": []
}
}
},
{
"type": "function",
"function": {
"name": "search_local_notes",
"description": "Search text within your personal notes vault",
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "Search term to look up in notes"
}
},
"required": ["query"]
}
}
}
]
# 3. Send the prompt with tools to Ollama
user_prompt = "What time is it now, and what projects are mentioned in my notes?"
messages = [{"role": "user", "content": user_prompt}]
response = ollama.chat(
model="qwen2.5:7b",
messages=messages,
tools=tools_schema
)
# 4. Check if the AI requested a Tool Call
if response.message.tool_calls:
messages.append(response.message) # Preserve AI's conversation context
for tool in response.message.tool_calls:
fn_name = tool.function.name
args = tool.function.arguments
print(f"🤖 AI wants to call: {fn_name} with args: {args}")
# Execute the actual Python function
if fn_name == "get_current_time":
result = get_current_time()
elif fn_name == "search_local_notes":
result = search_local_notes(args.get("query", ""))
else:
result = "Function not found"
# Feed the result back into the conversation history
messages.append({
"role": "tool",
"content": json.dumps(result, ensure_ascii=False)
})
# 5. Let the AI compose the final answer for the user
final_response = ollama.chat(model="qwen2.5:7b", messages=messages)
print("\n💡 Final AI response:")
print(final_response.message.content)
---
### 3 Security Principles for Running Local Function Calling
Even though Function Calling runs entirely offline, guard against model misbehavior with these three rules:
#### 1. Sanitize and Validate All Arguments (Input Sanitization)
Before using any parameter the AI returns — such as file paths or search queries — always validate that it doesn't contain dangerous tokens like ../ to prevent Path Traversal or access to files outside the allowed directory.
#### 2. Separate Read-Only Functions from Mutating Actions
* Read-Only Functions: e.g., reading the clock, searching notes, fetching system status — safe to auto-execute.
* Mutating Functions: e.g., deleting files, sending emails, modifying databases — always add a Human-in-the-Loop step or a confirmation prompt before execution.
#### 3. Keep Temperature Low
When the model is performing Function Calling, set temperature to 0.0 – 0.2 to minimize JSON formatting errors and maximize function-selection stability.
---
### The Bottom Line
Enabling Function Calling on Ollama shatters the old limitation of Local LLMs as mere text responders. It transforms them into a Cognitive Automation Engine that works seamlessly with your local system and personal knowledge base.
At Private Brain OS, pairing Function Calling with your own scripts is the key to building AI Agents that organize tasks, summarize data, and link notes on your machine — automatically, without the cloud, and without a single byte of your data leaving your hands.