Quickstart
ETL pipeline is responsible to transform European public procurement data from Tenders Electronic Daily (TED) into a format that's easier to handle and analyse. Primarily, the TED XMLs and eForms(), are transformed into Open Contracting Data Standard (OCDS) JSON and parquet files to ease importing the data into a:
- Graph database (KuzuDB in our case, but processed data should be generic enough to support any graph database and a
- Search engine (Meilisearch in our case)
Organizations are deduplicated before they are imported into the graph database.