Advanced Power BI refresh with ADF/Synapse

on YouTube ↗50m

" summary

Shows how to trigger Power BI dataset refreshes from ADF/Synapse using the Power BI REST API and a pipeline that uses the factory’s managed identity to call workspace and dataset endpoints. Documents hitting the Pro limit of eight refreshes (48 in Premium) and fixes imprecise UI scheduling by adding SQL metadata tables: dataset rows, refresh policies (times per day), a stored procedure that picks the next datetime, a Lookup+ForEach pipeline that refreshes individually and updates the next-run. Implementation works but needs timezone, throttling and retry hardening.

AI-written summary · about

" chapters (13)

1. 0:00Introduction and agenda
2. 2:03Limitations of Power BI UI
3. 4:41Data freshness challenges
4. 7:37Client case study
5. 9:41Exploring integration options
6. 11:09API authentication setup
7. 13:44Implementing API refresh
8. 18:10Refining the refresh logic
9. 21:46Advanced scheduling concept
10. 25:40Metadata table implementation
11. 31:05Orchestration in ADF
12. 38:54Final demonstration and recap
13. 42:29Q&A and conclusion

" related talks

• 50mPower BI Incremental refresh made simple.Robert French · SQLBits 2023
• 50mPower BI + Azure Synapse Analytics: Beginning your journeyPatrick LeBlanc, Adam Saxton · SQLBits 2023
• 51mSupercharge Power BI with Azure Synapse AnalyticsMathias Halkjaer · SQLBits 2023
• 49mHow to start with Azure Synapse Analytics - Power BI Developer EditionWolfgang Strasser · Power Platform Virtual Conference 2021
• 50mPower BI dataflows: building your own refresh orchestrationBen Watt · SQLBits 2024
• 51mPower BI Composite and Hybrid ModelsAlex Whittles · SQLBits 2023
• 48mGo With The (data)FlowMaria Gabriel · SQLBits 2023
• 40mSelf-Service ETL: The PowerBI Data FlowsDennes Torres · Data Con LA 2020