Overview
Bid Stats is the data layer behind UK public procurement pipelines. It ingests tenders from two official sources — Contracts Finder and Find a Tender Service — normalises them from OCDS format, and exposes a searchable REST API. The goal is a reliable, queryable feed of live and historical tender data that downstream tools (like Tender Intel) can consume without re-implementing ingestion logic.
How it works
A cron-driven scheduler runs incremental syncs against both sources on configurable intervals. A backfill mode handles historical gaps. All records land in SQLite via better-sqlite3, schema-validated with Zod at the boundary. Duplicate candidates are grouped using a deterministic hashing strategy across key fields. Each record gets quality scores — completeness, freshness lag, classification confidence — computed at ingest time and updated on subsequent revisions.
The API (Fastify) exposes:
- Search — keyword + filter queries across the full tender feed (source, notice type, sector, region, CPV code)
- Tender detail + history — full lifecycle of a notice, including all revisions
- Delta endpoint —
updated-sinceparameter for consumers that need change streams rather than full pulls - Ops endpoints — reconciliation snapshots, SLA breach reporting, and a replay queue for failed ingestions
Status
Phases 1–4 are complete: ingestion pipeline, normalisation, search, and operational monitoring. Phase 5 (API productization — auth, rate limiting, versioning) and Phase 6 (Postgres migration for scale) are planned. Currently used as the data backend for Tender Intel.