Load Testing Fabric - What is our hold capacity?
summary
Loads are generated from the TPC-H toolchain into CSVs and copied into a Fabric Lakehouse via Pipelines and Notebooks, then transformed into Delta and joined; an attempted 6 billion rows import runs in about 2h4m on F2 but about 7m46s on F64 (note: only ~4.2B rows survived malformed-drop), showing F64 ≈5× faster overall but at ~32× the cost. Fabric evaluates capacity in 30s blocks (F2=60 CUs/30s, F64=1920 CUs/30s), exposes overuse smoothing and burndown (1h burst → ~3h repay) in the Capacity Metrics app, and Power BI caps external results at 1M rows.
AI-written summary · about
chapters (16)
- 0:00Load testing Fabric
- 3:15Large data amounts
- 6:28Fabric performance challenges
- 9:19Fabric Capacity Metrics app
- 15:02Test data generation
- 19:44Fabric pipelines loading
- 23:18Notebook processing
- 26:536B rows on F2 vs F64
- 29:25Columns and partitioning
- 30:54Joining large datasets
- 33:04Pipeline testing
- 35:56Power BI reports
- 37:43Capacity overuse
- 39:08Scaling capacity
- 40:19Performance vs cost
- 44:43Question and answer session
related talks
30mDataEngBytes 2023 - BRI-T1-04 - Mimoune Djouallah
19mAccelerate your data potential with Fabric
50mMicrosoft Fabric Data Factory Q&A
27mModernize your data integration to enable petabyte scale analytics with Microsoft Fabric
20mFull Lifecycle Data Analysis on a Large-scale and Leadership Supercomputer...
46mETL Confessions
20mManaging Capacity
26mReal Time analytics - Sense analyze generate insights with Fabric