As your Second Brain — the personal knowledge base you maintain in Obsidian — grows from a handful of notes to hundreds or even thousands, something quietly starts to happen: "digital clutter" accumulates without you even noticing.
The most common symptoms include:
* Orphan Notes: Notes with no inbound or outbound [[...]] links, effectively disconnected from your knowledge graph
* Missing Metadata: Notes lacking YAML Frontmatter — no creation date, no status, no category
* Untagged Notes: Notes with zero #tags, making them nearly impossible to filter or retrieve later
Manually scanning every single file to clean up your vault is a massive drain on both time and mental energy. In this article, we'll build a Cognitive Automation pipeline using Python and Ollama to create a Vault Health Check Agent that audits your knowledge base, identifies structural issues, and uses a local LLM to analyze and recommend improvements — all 100% locally, with zero risk of your personal data leaking to a third party.
---
### Why a 100% Local Vault Health Check?
Your personal knowledge vault is, in essence, your "second brain." It holds business ideas, personal reflections, research notes, and life plans. Analyzing the content inside those notes — to categorize, tag, or suggest connections — requires an AI to actually read them.
If you pipe that content through a cloud-based AI service, every thought you've ever written gets transmitted off your machine, potentially used to train external models. By running a local LLM through Ollama, the entire process stays on your hardware: 100% private, fast, no monthly subscription, and it works even when you're offline.
---
### How the Vault Health Check Agent Works
The Python script we're building operates in four main stages:
1. Vault Scanning: Reads every .md file in your Obsidian vault, extracting YAML Frontmatter, #tags, and internal [[WikiLinks]]
2. Health Audit: Identifies orphan notes, files missing metadata, and notes that are too short to be useful
3. AI-Powered Recommendations (Local LLM): Sends the content of flagged notes to Ollama to suggest tags, categories, and related notes that should be linked
4. Health Report Generation: Writes a summary report as a Markdown file directly into your Obsidian vault
---
### Prerequisites
1. Python 3.8+ installed on your machine
2. Ollama installed and running as a service (download from ollama.com)
3. Pull a language model that handles your content well, such as gemma2 or llama3.2:
ollama pull gemma2
---
### The Python Code: Building Your Vault Health Check Agent
Create a file called vault_health_agent.py and paste the script below. It's designed to be readable, uses only Python's standard library, and has zero external dependencies:
import os
import re
import json
import urllib.request
from pathlib import Path
# ================= Configuration =================
OBSIDIAN_VAULT_PATH = "/home/linn/notes" # Change to your Obsidian vault path
REPORT_OUTPUT_DIR = os.path.join(OBSIDIAN_VAULT_PATH, "Reports")
LLM_MODEL = "gemma2" # Ollama model to use
MAX_NOTES_TO_AI_ANALYZE = 5 # Max flagged notes to send to AI per run
# ==================================================
def parse_md_file(file_path):
"""Read and extract the structure of a Markdown file."""
with open(file_path, "r", encoding="utf-8", errors="ignore") as f:
content = f.read()
# Extract YAML Frontmatter
has_frontmatter = bool(re.match(r'^---\s*\n.*?\n---\s*\n', content, re.DOTALL))
# Extract WikiLinks [[...]]
links = set(re.findall(r'\[\[([^\]\|]+)(?:\|[^\]]+)?\]\]', content))
# Extract Tags #tag
tags = set(re.findall(r'(?<!\w)#([a-zA-Z0-9_\-ก-๙]+)', content))
# Rough content length calculation
clean_text = re.sub(r'^---\s*\n.*?\n---\s*\n', '', content, flags=re.DOTALL).strip()
word_count = len(clean_text.split())
return {
"path": file_path,
"name": Path(file_path).stem,
"has_frontmatter": has_frontmatter,
"links": links,
"tags": tags,
"word_count": word_count,
"content_snippet": clean_text[:400] # Short snapshot for AI context
}
def scan_vault(vault_path):
"""Scan all notes in the vault to identify areas needing improvement."""
all_notes = {}
inbound_links = {}
# Walk through every .md file
for root, _, files in os.walk(vault_path):
for file in files:
if file.endswith(".md") and not file.startswith("."):
full_path = os.path.join(root, file)
parsed = parse_md_file(full_path)
note_name = parsed["name"]
all_notes[note_name] = parsed
inbound_links.setdefault(note_name, 0)
# Count cross-note links (inbound count)
for note_name, data in all_notes.items():
for target in data["links"]:
clean_target = Path(target).stem
if clean_target in inbound_links:
inbound_links[clean_target] += 1
else:
inbound_links[clean_target] = 1
# Categorize issues
orphans = []
missing_metadata = []
untagged = []
for note_name, data in all_notes.items():
# Orphan: no outbound links AND no inbound links
if len(data["links"]) == 0 and inbound_links.get(note_name, 0) == 0:
orphans.append(data)
if not data["has_frontmatter"]:
missing_metadata.append(data)
if len(data["tags"]) == 0:
untagged.append(data)
return all_notes, orphans, missing_metadata, untagged
def ask_ollama_for_suggestions(note_data):
"""Send flagged note content to Ollama for tag & category suggestions."""
prompt = f"""You are an expert in Second Brain systems and PKM (Personal Knowledge Management).
Analyze the note below and recommend:
1. One primary Category
2. 3–5 appropriate tags (format: #tag)
3. A one-sentence summary of the core idea
Note name: {note_data['name']}
Content excerpt:
{note_data['content_snippet']}
Respond concisely and clearly."""
payload = {
"model": LLM_MODEL,
"prompt": prompt,
"stream": False
}
try:
req = urllib.request.Request(
"http://localhost:11434/api/generate",
data=json.dumps(payload).encode("utf-8"),
headers={"Content-Type": "application/json"}
)
with urllib.request.urlopen(req, timeout=30) as resp:
res = json.loads(resp.read().decode("utf-8"))
return res.get("response", "").strip()
except Exception as e:
return f"Failed to reach AI: {str(e)}"
def generate_health_report(all_notes, orphans, missing_meta, untagged):
"""Generate a summary report as a Markdown file inside Obsidian."""
os.makedirs(REPORT_OUTPUT_DIR, exist_ok=True)
report_file = os.path.join(REPORT_OUTPUT_DIR, "Vault-Health-Report.md")
total_count = len(all_notes)
orphan_count = len(orphans)
meta_count = len(missing_meta)
untagged_count = len(untagged)
# Calculate vault health score (out of 100)
health_score = max(0, 100 - int((orphan_count * 3 + meta_count * 2 + untagged_count * 1) / (total_count or 1) * 20))
lines = []
lines.append("---")
lines.append("title: 'Vault Health Check Report'")
lines.append("type: 'report'")
lines.append("tags: ['pkm', 'vault-health', 'automation']")
lines.append("---\n")
lines.append("# 🏥 Vault Health Report")
lines.append(f"\n* **Total Notes:** {total_count} files")
lines.append(f"* **Vault Health Score:** `{health_score}/100`")
lines.append("\n---")
lines.append("### 📊 Issue Overview")
lines.append(f"* 🔗 **Orphan Notes (no links in or out):** {orphan_count} files")
lines.append(f"* 📋 **Missing Metadata (no Frontmatter):** {meta_count} files")
lines.append(f"* 🏷️ **Untagged Notes:** {untagged_count} files")
lines.append("\n---\n### 💡 AI Recommendations for Flagged Notes\n")
# Pick problematic notes for AI analysis
problematic_notes = (orphans + untagged)[:MAX_NOTES_TO_AI_ANALYZE]
for idx, note in enumerate(problematic_notes, 1):
print(f"Analyzing suggestions for: {note['name']}...")
ai_suggestion = ask_ollama_for_suggestions(note)
lines.append(f"#### {idx}. [[{note['name']}]]")
lines.append(f"**Issues Found:** {'No links ' if note in orphans else ''}{'No tags' if note in untagged else ''}")
lines.append(f"**AI Suggestion:**\n{ai_suggestion}\n")
lines.append("---\n### 🛠️ Recommended Action Items")
lines.append("1. Open **Orphan Notes** and add `[[...]]` links to related notes or a MOC")
lines.append("2. Add YAML Frontmatter to notes missing basic metadata")
lines.append("3. Apply primary category tags to improve Dataview queryability")
with open(report_file, "w", encoding="utf-8") as f:
f.write("\n".join(lines))
print(f"\n✅ Health report generated at: {report_file}")
if __name__ == "__main__":
print("🔍 Starting Obsidian vault scan...")
all_notes, orphans, missing_meta, untagged = scan_vault(OBSIDIAN_VAULT_PATH)
generate_health_report(all_notes, orphans, missing_meta, untagged)
---
### How to Run It
1. Open your terminal and update the OBSIDIAN_VAULT_PATH variable in the script to match your actual Obsidian vault folder
2. Run the script:
python3 vault_health_agent.py
3. Once the script finishes, open Obsidian and navigate to Reports/Vault-Health-Report.md. You'll find a scored health report with AI-generated tagging and categorization suggestions — all produced by your local model.
---
### Automate It with Cron or Task Scheduler
To keep your vault in good shape without thinking about it, schedule the script to run every Sunday night via a Cron Job on Linux/macOS:
# Open cron editor
crontab -e
# Add this line to run every Sunday at 22:00
0 22 * * 0 /usr/bin/python3 /path/to/vault_health_agent.py
---
### The Bottom Line
Building a Second Brain isn't about hoarding the most notes — it's about maintaining the quality and connectivity of your knowledge system. By layering Cognitive Automation on top with Python and Ollama, you turn a tedious maintenance chore into a self-running health check that keeps your second brain sharp. And the best part? Every byte of your knowledge stays yours, on your machine, 100% of the time.