BigMail: How We Built a DataLake at Russian Post
summary
Describes building a Big Data infrastructure (Hadoop-based Data Lake) for Russian Post: the chosen technologies (Hadoop ecosystem), reasons for those choices, and how the product is evolving inside Russian Post. Targets teams planning Big Data adoption and anyone interested in how a modern postal service organizes its data platform.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 7:07Problem and first release
- 11:19Spark issues and solutions
- 16:20Vertica and Cassandra choice
- 20:15Real-time processing shift
- 25:30Scaling and stability problems
- 32:45Infrastructure optimization
- 37:09Current architecture and components
- 45:23Conclusions and recommendations
- 59:49Q&A
related talks
48mMachine Learning in the MAIL.RU Advertising System
43mData Lake beyond the Buzzword
01h 35mThe Modern Data Stack
41mBig data as we know it @bol.com
47mSolving Real & Big (Data) Problems Using Hadoop
26mBuilding a data lake at a bank #bbuzz
42mSolve Big Data Challenges with Data Lake
19mDemystifying Big Data