← Back to Journal
2026-07-30

Beyond Chunks: Why I'm Rebuilding My Document AI Stack

Building AI products is humbling.

Every few months, something arrives that makes you question the assumptions your last project was built on.

Last year, I built DocumentGem to explore AI-powered document understanding. It used the standard Agentic RAG approach: parse documents, create embeddings, retrieve relevant chunks, and let users chat with PDFs.

It worked surprisingly well... until I started testing it with real business documents.

Where RAG Started Falling Apart

RAG is excellent when documents are mostly text.

Business documents aren't.

Invoices, contracts and purchase orders are visual. Tables matter. Layout matters. Relationships between documents matter.

The more I tested, the more obvious the limitations became.

Chunking loses context. Split an invoice table across two chunks and suddenly totals, taxes and line items stop making sense.

OCR loses structure. Extracting text isn't the same as understanding a document. The model knows the words but not that this number belongs to that label.

Trust becomes the bottleneck. If an AI extracts a bank account number or payment amount, nobody wants to believe it just because the model says so. Users need to see exactly where the value came from.

Documents don't live in isolation. An invoice should be validated against its purchase order, vendor and contract. Vector search isn't designed for that.

Those weren't implementation bugs. They were architectural limitations.

Enter DocWiser

I'm now rebuilding the idea from scratch as DocWiser.

Instead of treating documents as blocks of text, the goal is to treat them as structured, connected objects.

Three ideas are shaping the architecture.

1. Structured knowledge instead of vector-only retrieval

Google's recently released Open Knowledge Format (OKF) provides a simple, human-readable way to represent structured knowledge using Markdown and YAML.

Policies, business rules and reference data no longer need to live inside embeddings. They become explicit, versioned and verifiable.

That makes retrieval deterministic instead of probabilistic.

2. Visual verification

Extraction is only half the problem.

Every extracted field stores its exact location on the page. When a user clicks a value, the original document highlights that region instantly.

Instead of asking users to trust the model, the product lets them verify every answer in seconds.

3. Relationship-first document intelligence

Documents are connected.

Invoices reference vendors. Vendors have contracts. Contracts reference purchase orders.

Rather than searching isolated chunks, DocWiser builds these relationships into a graph so it can answer questions like:

"Does this invoice match the agreed pricing in the contract and the corresponding purchase order?"

That's a much more useful problem to solve than simply answering questions about a PDF.

Looking Ahead

DocumentGem taught me what Agentic RAG could do.

DocWiser is an attempt to build around what it couldn't.

The goal isn't a better chatbot for documents.

It's a system that understands structure, preserves visual context, and makes every answer verifiable.

I'm particularly excited about what open standards like OKF could mean for document intelligence over the next few years.

Curious to hear how others are thinking about this space.