How we achieved extreme scale under the hood in Azure Functions

on YouTube ↗01h 04m

" summary

Deep dive into Azure Functions Flex Consumption: rewritten architecture (Legion + placeholder pools) that uses App Insights, Azure Load Test (JMeter engine) and Cosmos DB to hit extreme throughput, e.g. burst to 1000 instances in under a minute and customer tests reaching 1.6 million requests per second. Workloads run in containers on nested VMs, concurrency is a tunable knob (defaults: 2GB → 16 concurrency; Python = 1), placeholder specialization reduces cold starts, and experiments show higher concurrency cuts instance count and cost.

AI-written summary · about

" chapters (8)

1. 0:00Extreme scale in Azure Functions
2. 5:27Testing at scale goals
3. 14:38High-throughput telemetry demo
4. 26:39Architecture deep dive
5. 39:46Concurrency in Flex Consumption
6. 55:35Concurrency settings metrics
7. 1:00:03Placeholder pools cold start
8. 1:01:48Max scale tips

" related talks

• 59mAzure Serverless for JavaScript TypeScript FolksDouglas Starnes · Connect.Tech 2022
• 52mHands-on Workshop: Refactor Node.js and Express APIs to Serverless APIs with Azure Functionschristian mumba · Microsoft Create: Serverless 2020
• 18mScaling Kubernetes-Native Systems with Event-Driven Patterns on AzureAshif Anwar · Conf42 Kube Native 2025
• 56mServerless microservices with Azure Container AppsJakob Ehn · CloudBurst 2022
• 44mAzure and Containers, the tale of the two inseparable friendsYaser Adel Mehraban · NDC Melbourne 2022
• 44mHop on the Serverless Adventure with NodeJSSimona Colin · BuzzJS 3.1 2018
• 28mHop on the serverless adventure with NodeJSSimona Cotin · NodeConf EU 2017
• 31mLessons from Scaling GraphQL to Half a Billion Requests per MinuteTushar Mathur · GraphQLConf 2024