I wired up a live job search endpoint expecting the hard part to be the search. It wasn't. The hard part was that the market sells the same job five times. My demo query was "python developer" + remote. Thirty results, and four were the same Berlin fintech requisition — indexed by three aggregators plus the company's own board, each with a mangled title ("Sr. Python Dev", "Senior Python Developer (m/f/d)"), different scraped snippets, different dates. Naive fixes failed fast: exact title match: caught nothing company + lowercase title: caught two of four, missed the abbreviations The signal that actually worked was the apply URL. Nearly every listing sits on top of an ATS — Greenhouse, Lever, Ashby — and those URLs carry the canonical requisition ID: boards.greenhouse.io/company/jobs/1234567 Same trailing ID in every aggregator copy means same job. I dedupe on that key first, then fall back to normalized company plus token-sorted title fuzzy ...