Serverless Stream Processing with Confluent Cloud, Kafka and AWS Lambda
summary
Shows a concrete architecture: Confluent Cloud Kafka topics feed ksqlDB which creates a ksqlDB materialized view (left outer join enriching trades with user data) and persists state to RocksDB change log topics, while Kafka topics trigger AWS Lambda functions that write back to other topics and into DynamoDB. Claims serverless scaling works in practice and cites a load test with 100,000 partitions where Lambda scales out and processes events reliably. Code and load-test results are available on GitHub.
AI-written summary · about
chapters (5)
related talks
42mBuilding real-time serverless data applications
36mLeveraging Serverless architecture with Confluent Kafka
28mBuilding Event Streaming Applications with Serverless Kafka
47mEvent-driven Serverless Architecture with DynamoDB
32mBuilding and scaling a log analytics platform - a serverless approach
47mServerless Kafka Patterns.
01h 07mKafka Streams and KSQLDB
51mServerless architectural patterns