IntelligentOldMan — agentic email intelligence

IntelligentOldMan is my answer to a question most inbox tools dodge: how much judgment can you safely delegate to an autonomous system? Not “summarize my mail” — classify intent, validate the output, persist it for retrieval, and let agents reason over the whole mailbox later.

Problem

Email is unstructured, sensitive, and threaded. Generic LLM wrappers treat every message like a chat prompt. Production needs idempotent ingest, schema-bound classification, observable failure modes, and retrieval that returns the right thread — not just similar paragraphs.

Architecture

• Ingest: Microsoft Graph change notifications → Event Grid → Service Bus → idempotent processing per message, with subscription lifecycle handling and delta-sync reconciliation for anything missed.
• Classify: Intent classification with structured output: Ollama locally, Azure OpenAI in production.
• Validate: A grounding evaluator checks stored content against the source email, and an operator correction API feeds mistakes back in.
• Leads: Supply offers (dealer, private collection, consignment) are extracted into structured details and queued for human review, with optional SMS alerts for hot leads.
• Store: Chunked content + pgvector embeddings in PostgreSQL for downstream RAG agents.
• Infra: Azure resources provisioned with Terraform — iterate on model behavior without manual drift.

Key decisions

Grounded automation, not autonomous spending. The agent retrieves, classifies, summarizes, and recommends; a person approves anything consequential, like sending an email or buying a collection. Every supply lead lands in a review queue rather than triggering an action.

RAG as a product surface, not a demo. Retrieval quality is evaluated separately from generation. Wrong context is worse than no context — that lesson shaped how chunks, metadata, and thread boundaries are stored.

Stack

• Java, Spring Boot, Spring AI
• PostgreSQL + pgvector
• Microsoft Graph, Event Grid, Service Bus
• Azure OpenAI, Ollama, Dokimos evals
• Azure App Service, Terraform

Status

IntelligentOldMan is live on the Poke Pitch Shop mailbox and powers the shop's email analytics: every incoming message is classified by intent and grounded, supply offers are tracked by type and lead source in a review queue, and the whole inbox is searchable through RAG. Ingest, classification, grounding checks, supply-lead capture, and mailbox RAG are covered by eval suites for intent, grounding, and answer quality. The project is scoped to the Email Manager. Next: a chat API over the mailbox, thread-aware retrieval, and agent tools that act on classification without over-automating replies.