Performance Testing and Big Data
summary
Shows concrete numbers and tools: a 1.6 TB/s fiber example, wind‑turbine sensors that spot one failing unit among 100, and benchmarks that compare a target of 10,000 messages per second to an infrastructure that only handles 5,000. Describes using Apache Storm, Hadoop/Spark, cloud scaling and Docker, and an automated gatekeeper Dr. Elephant that flags slow MapReduce/Storm jobs. Gives failures that happened — wrong database for attachments causing collapse at 20 vs 200 users — and practical mitigation steps.
AI-written summary · about
chapters (18)
- 0:00Intro
- 1:40Big Data
- 6:30Performance Testing
- 11:25Big Data Architecture
- 13:55Lambda Architecture
- 15:55Practical Example
- 17:45Meetups
- 19:35Practical examples
- 20:35Apache Storm
- 22:05Infrastructure
- 23:50Benchmarks
- 25:15Requirements
- 27:10Wrong database
- 29:35Monitoring
- 31:05Dr Elephant
- 33:15Scalability
- 35:15DevOps
- 36:30Questions
related talks
44mPerformance Testing with a "Big Data Tool"
57mLast- und Performancetests in der Cloud?!
38mCharacterizing the performance of Big Data processing platforms
01h 08mPerformance testing in the cloud
21mCrunching Millions of Requests for Breakfast: A Tale of Performance Testing Prowess
42mAutomation Testing in a Real Big Data Project
01h 06mBest practices when accessing Big Data or any other data!
30mAbove the Clouds: Investigating Systems from 32,000 ft.