Provenance beats polish: what shipping document converters taught me about trust
I've shipped a family of document-to-Markdown converters — PDF, DOCX, PPTX, XLSX, EPUB, HTML — and the hardest lesson wasn't parsing tables or preserving heading styles. It was provenance. Early on, I treated every input as equally trustworthy. A PDF from a customer's scanner and a PDF I'd generated myself in tests went through the same path, and I measured success by output quality alone. Then a user sent a file whose conversion looked perfect but was subtly wrong — half the pages silently skipped because of an encoding quirk. The output was clean Markdown. The provenance was broken. I had no way to answer "where did this text come from and which pages did it not come from?" So I changed the contract. Every conversion now records what went in: source type, byte length, page or sheet count, and which segments mapped to which output blocks. Not for compliance theater — for debugging. When an agent downstream questions a fact, the first thing I check isn'...