Skip to content

Deep dives into architecture, data engineering, and the tools we build.

From 60 Minutes to 4: Optimizing Spark MERGE INTO on a 2 Billion Row Iceberg Table
· Platform Team

From 60 Minutes to 4: Optimizing Spark MERGE INTO on a 2 Billion Row Iceberg Table

How we cut our daily upsert pipeline from an hour to under 4 minutes using storage partition joins and shuffle hash hints.

Recent Posts

View all →