# Knowledge Graph Agent — Specification Document
Specification for the Knowledge Graph Agent feature in the Story Studio system.
---
1. Overview & Objectives
### 1.1 Overview
The Knowledge Graph Agent is a personal AI agent that runs entirely on your machine (local/offline). Its job: read prose from notes, documents, or project logs, then automatically extract the relational structure between entities as structured Triplets (Subject–Predicate–Object). It generates a Mermaid graph diagram and Obsidian Canvas links on the fly. Your personal data never leaves your machine — not a single byte.
### 1.2 Problems Solved
- Hundreds of knowledge notes in your Second Brain can't be visually connected to reveal their relationships.
- Reading notes page by page and manually identifying "who/what (Subject)" relates "to what (Object)" is cognitively expensive and time-consuming. Good ideas stay siloed, never linked.
- The current relationship-mapping process is 100% manual human labor.
### 1.3 Primary Objectives
| # | Objective | Target Detail |
|---|---|---|
| 1 | Automated Extraction | Convert prose into structured JSON Triplets with zero-touch (no human intervention) |
| 2 | Offline-First Security | Personal data never leaves the machine — uses a Local LLM via Ollama instead of a cloud API |
| 3 | Instant Visualization | Generate Mermaid graph code you can paste into Obsidian / Notion / any Markdown viewer for an immediate diagram |
| 4 | Bilingual Support | Handles both Thai and English — Qwen2.5 / Llama3 models deliver strong instruction-following performance |
### 1.4 Key Definitions (Glossary)
| Term | Definition |
|---|---|
| Entity | A person, technology, concept, or object mentioned in the text |
| Triplet | A 3-tuple of entities: Subject–Relation–Object, representing a graph edge |
| Mermaid Graph | A DSL for diagrams that renders to SVG (used in Story Studio) |
| Obsidian Canvas | A JSON-based `.canvas` file in Obsidian for arranging notes and links in a multi-view layout |
| Local LLM | A Large Language Model running on your machine via Ollama — not a cloud API |
---
2. System Architecture (Local LLM + Ollama)
### 2.1 High-Level Flow Diagram
+---------------------------+ +---------------------------+
| Story Studio UI | | Knowledge Graph Panel |
| (Markdown Input Area) |-----text--> [Mermaid Preview] |
| | live-rendered SVG |
| Raw Note Text |---send----> |
+---------------------------+ +---------------------------+
|
+----------------------v----------------------------------+
| Knowledge Graph Agent (kg_extractor.py) |
| |
| [Prompt Builder] --> Ollama API (/api/generate) |
| v |
| [JSON Parser / Validate / Extract Triplets] |
| v |
| [Component Router / Renderer] |
| |-- Mermaid Graph Generator |
| |-- Obsidian Canvas Dumper |
| |-- Wikilink Backlink List |
+----------------------------------------------------------+
|
+--------------v---------------+
| Local Runtime |
| Ollama :11434 (Qwen2.5 / |
| Llama3.1) |
| Pydantic JSON Validation |
+------------------------------+
### 2.2 Component Details
| Component | Role | Key Details |
|---|---|---|
| `kg_extractor.py` (Core Agent) | Orchestrates the full pipeline: receives raw text from the user, sends it through Ollama API + Pydantic validation | Sub-functions: `extract_knowledge_graph(text)`, `validate_triplets(data)`, `generate_mermaid_code(triplets)`, `dump_obsidian_canvas(triplets, filepath)` |
| Ollama Local Runtime (LLM Engine) | Performs inference with locally installed models | Endpoint: `localhost:11434/api/generate`; Recommended models: qwen2.5:latest (7B general / 14B precision); uses format constraint `"format":"json"` to enforce JSON output |
| Output Router & Renderer | Splits results into 3 output paths | 1. Mermaid Diagram — string code, paste into any `.md` file 2. Obsidian Canvas — `.canvas` file in the user's vault 3. Wikilinks — a list of `[[ ]]` links for each entity |
### 2.3 Tech Stack
| Layer | Technology | Notes |
|---|---|---|
| Language | Python 3.10+ | Minimal dependencies — only `requests` + `pydantic` |
| API Client | `requests` | HTTP POST to Ollama REST endpoint |
| Validation | `pydantic` | Validates JSON structure before passing output downstream |
| LLM Runtime | Ollama (local) | Install via `brew install ollama` (macOS) or `apt` (Linux) |
| Model Hub | Ollama Library | Qwen 2.5 / Llama 3.x via `ollama pull` |
---
3. Data Pipeline & Schema (Triplets)
### 3.1 Pipeline Flow
[Step 1: Raw Input] --> [Step 2: Ollama Inference] --> [Step 3: Validation] --> [Step 4: Output Generation]
Markdown / Text File Prompt Builder + Pydantic Strict JSON Mermaid Code / Canvas
(.md / .txt) local LLM (Qwen2.5/ Schema Check & + Wikilink Backlinks
Llama3.1 via Ollama API) normalize Triplets JSON
### 3.2 Prompt Engineering — System Prompt & User Prompt Template
System Prompt (hardcoded — does not accept user input, preventing Prompt Injection):
You are an AI specialist in Knowledge Graph Extraction.
Read the provided text and extract entities and their relationships as structured Triplets (Subject -> Relation -> Object).
Response constraints:
1. Respond ONLY with a JSON object containing the key "triplets" — no preamble, no markdown code blocks
2. JSON structure:
{
"triplets": [
{"source":"Entity A","relation":"relationship","target":"Entity B"}
]
}
3. Be concise and clear. Use short phrases for source, relation, and target.
User Prompt Template:
Text for Knowledge Graph extraction:
{raw_text_input_from_user}
### 3.3 JSON Schema — Pydantic Validation Model
| Field | Type | Constraints | Example |
|---|---|---|---|
| `triplets` | list | Length >= 2, no duplicates (unique source+relation+target) | `[ ... ]` |
| `source` | str | Min 1 char, strip whitespace | "Second Brain" |
| `relation` | str | Must not be empty; use a short verb or state | "uses", "builds", "installs" |
| `target` | str | Min 1 char, strip whitespace | "Obsidian" |
### 3.4 Expected Output — Sample Triplets
{
"triplets": [
{"source":"Linn","relation":"builds","target":"Second Brain"},
{"source":"Second Brain","relation":"uses","target":"Obsidian"},
{"source":"Obsidian","relation":"syncs files via","target":"Syncthing"},
{"source":"System","relation":"runs on","target":"Home Server"},
{"source":"Home Server","relation":"enables","target":"Docker"},
{"source":"Home Server","relation":"installs","target":"AdGuard Home"},
{"source":"AdGuard Home","relation":"filters ads in","target":"Network"},
{"source":"Tailscale","relation":"provides remote access to","target":"Home Server"}
]
}
---
4. Mermaid Graph & Obsidian Canvas Integration
### 4.1 Mermaid Diagram — Rules
| Rule | Detail | ||
|---|---|---|---|
| Graph Direction | `graph TD` (Top-Down) for overview diagrams | ||
| Entity in Nodes | Use quotes `"Entity Name"` — supports both Thai and English | ||
| Edge Labels | Use `--> | "relation" | ` for every edge |
| Sanitization | Strip nested quotes, collapse extra whitespace, enforce max 64 chars per node | ||
| Conflict Resolution | Multiple triplets sharing the same source/target → merge into a single node (no duplicates) |
### 4.2 Sample Mermaid Code Output
graph TD
"Linn" -->|"builds"| "Second Brain"
"Second Brain" -->|"uses"| "Obsidian"
"Obsidian" -->|"syncs files via"| "Syncthing"
"System" -->|"runs on"| "Home Server"
"Home Server" -->|"enables"| "Docker"
"Home Server" -->|"installs"| "AdGuard Home"
"AdGuard Home" -->|"filters ads in"| "Network"
"Tailscale" -->|"provides remote access to"| "Home Server"
### 4.3 Obsidian Canvas Integration
| Item | Value / Detail |
|---|---|
| File Extension | `.canvas` |
| Format | JSON object following Obsidian's native structure |
| Output Path | User's vault → `Knowledge-Graphs/ |
Canvas Data Model (abbreviated):
{
"version": 1,
"nodes": [
{"id":"a001","x":0,"y":0,"width":300,"height":40,"text":"Linn"},
{"id":"a002","x":150,"y":80,"width":300,"height":40,"text":"Second Brain"}
],
"edges": [
{"id":"e001","from":"a001","to":"a002","label":"builds"}
]
}
### 4.4 Wikilink Backlink Generator
For every extracted entity, generate Wikilinks:
- [[Linn]]
- [[Second Brain]]
- [[Obsidian]]
- [[Syncthing]]
- [[Home Server]]
- [[Docker]]
- [[AdGuard Home]]
- [[Tailscale]]
---
5. Features & UI/UX Integration in Story Studio
### 5.1 Feature Placement in UI — Knowledge Graph Panel
+------------------------------+ +--------------------------+
| Story Studio Editor | | Knowledge Graph Panel |
| | | |
| [Markdown Input Area] | | [Mermaid Preview] |
| "Linn is setting up..." | | [live rendered SVG] |
| |--> --->| |
+-------------------------------+ | [Triplet Data Table] |
| (editable: source, |
| relation, target) |
| |
| [Actions Bar] |
| [Export .canvas] |
| [Export Wikilinks .md] |
| [Export Raw JSON] |
+--------------------------+
### 5.2 User Flows — Usage Scenarios
| Flow | Steps | Result |
|---|---|---|
| **Extract** | User pastes or uploads a `.md`/`.txt` file → clicks "Generate Graph" | Triplets JSON + Mermaid render, all instantly |
| **Preview** | View the Mermaid diagram in the right panel (live); edit nodes/relations inline | Diagram updates in real-time (300ms debounce) |
| **Export Canvas** | Click Export → Obsidian Canvas | Download `.canvas` file, open directly in Obsidian |
| **Export Backlinks** | Click Export → Wikilink list | Copy the wikilink list and paste into an Obsidian note |
### 5.3 UI Components List
| Component | Location | Detail |
|---|---|---|
| `KnowledgeGraphPanel` | Right panel | Displays Mermaid preview + Actions bar |
| `TripletDataTable` | Inside Panel | Table of triplets (editable source/relation/target — not read-only) |
| `ExportDropdown` | Bottom of Panel | Options: `.canvas`, `.md (wikilinks)`, `.json (raw triplets)` |
| `ModelSelector` | Top toolbar | Select Ollama model: Qwen2.5 / Llama3.1 / Custom model name |
### 5.4 Behavior Specifications
- Real-time Preview: Every time a Triplet changes → Mermaid diagram re-renders instantly (300ms debounce)
- Editable Triplets: Users can drag/edit source/relation/target directly in the DataTable (not read-only)
- Conflict Highlighting: If two triplets share the same relationship between the same node pair → display a yellow warning highlight
- Error Recovery: If Ollama returns non-JSON or times out (>60s) → show toast notification + retry button
- Model Selection: Users can switch models on the fly (toggle Qwen2.5 ↔ Llama3.1 without restarting)
---
6. Security & Privacy Model (100% Offline)
### 6.1 Core Design Philosophy — Offline First
+--------------------------------------------+
| OFFLINE-ONLY DESIGN |
| |
| Personal data / organizational secrets |
| never leaves the machine |
| Your Laptop / Home Server |
| |
| [Your Notes] -> [Local LLM/Ollama] -> |
| [Output .md/.canvas] |
| |
| Zero cloud interaction. Ever. |
+--------------------------------------------+
### 6.2 Data Scope — What's Processed and Stored
| Data Type | Where Stored? | Sent Off-Machine? | Retention |
|---|---|---|---|
| Raw input note (source text) | RAM only during processing | No | Cleared after session ends |
| Ollama API traffic | `localhost:11434` loopback interface | No | — |
| Triplets JSON intermediate | RAM → local filesystem output | No | Until user deletes |
| Mermaid `.md` output files | User's local filesystem (vault) | No | Permanent (user-owned) |
| Model weights | `~/.ollama/models/` local disk | No | Permanent (one-time download) |
### 6.3 Network Boundary — Zero Outbound Traffic Verified
+-------------------------- Home Machine (isolated) ------------------------+
| |
| +-----------+ HTTP +------------+ JSON +-----------+ |