Trino Data Lake
A medallion lakehouse for customer banking data.
An end-to-end data lake built on Trino and MinIO. A silver layer of raw banking data is loaded, then declarative pipelines derive gold-layer customer profiles, spending and high-value segments — all visualised in a workflow UI. Backend infrastructure, so this one is a walkthrough rather than a clickable prototype.
How it works
Docker Compose brings up Trino, MinIO and the pipeline services. A silver layer is loaded — roughly 1,000 customers and ~50,000 transactions — then the pipeline runner derives gold-layer customer profiles, spending and high-value customers. The UI visualises layer sizes, pipeline dependencies, the data catalog, and ingestion into MinIO-backed Parquet files, with runs shown in the workflow UI.
Walkthrough
What it does
- —Medallion architecture: silver (raw) → gold (profiles, spend, high-value)
- —Trino query engine over MinIO object storage (Parquet)
- —Declarative pipeline runner (SQL → transforms) with a dependency graph
- —Workflow UI for executions, data catalog and ingestion visibility