~/talk

Power BI dataflows: building your own refresh orchestration

Talkat SQLBits 2024 · 2024-03-19..03-23· [data-engineering, developer-experience]
name:Ben Watt
links:none on file
slides:none on file

Shows a working orchestration that runs Power BI dataflows in ordered blocks by calling the Power BI REST APIs from Azure Functions and marshalling execution with Power Automate using JSON metadata and polling for status. Built components: two function endpoints (start + status), a JSON list-of-blocks, retry logic and delays; demo runs six flows correctly. Notes performance change when using Enhanced Compute Engine where a sample run drops from 202s to 48s. Code is on GitHub.

AI-written summary · about

  1. 0:00Power BI dataflows intro
  2. 1:31Live demo: refresh orchestration
  3. 5:32What are dataflows and limits
  4. 8:19Orchestrate with Power Automate & Azure Functions
  5. 12:23Azure Functions code review
  6. 21:46Power Automate refresh flow
  7. 27:26Large-scale orchestration
  8. 36:24Orchestration in action with status
  9. 36:44Linked entities and compute engine
  10. 48:51Fabric pipelines for refresh
  11. 49:35Q&A
~/search
type to search 6,249 conferences, 138,987 speakers and 185,172 talks
~/help
cmdctrl k
search every conference, speaker and series
/
focus the filter search
h
home
c
conferences
s
speakers
e
series
t
light / dark — the status bar button does the same
?
this help
esc
close this, then clear every filter — or just the search, while you are typing in it

Ignored while you are typing. Pages without a filter rail have no search to focus.