Spotify and Python: love at first sight

on YouTube ↗32m

" summary

Spotify describes a concrete backend: many tiny services (about 60) talking HTTP/Thrift with a custom open client protocol, an access point handles authentication and multiplexing, search uses Java Lucene, playlist logic becomes a distributed VCS and is being reimplemented in Java/Scala, core storage uses PostgreSQL plus Cassandra since spring, fast lookups use Tokyo Cabinet and memcache, Hadoop ingests logs, deployment uses Git, 428 repos, Puppet and 10m packages, and the stack moves from Twisted then gevent to scale for 10 million users and 1 million paying.

AI-written summary · about

" chapters (23)

1. 0:00intro
2. 0:30why am I here
3. 2:08Running water or electricity
4. 2:36Native desktop client
5. 3:31Spotify box
6. 4:24Spotify staff
7. 5:24Windows client
8. 7:54Twisted
9. 8:21Square Root
10. 8:39Architecture
11. 14:07Core Services
12. 15:53Tools
13. 16:37Service
14. 17:27Frameworks
15. 18:07Languages
16. 18:27Services
17. 19:22Protocols
18. 20:45Storage
19. 22:18Cassandra
20. 24:01Adam
21. 25:22Deployment
22. 27:30Future problems
23. 27:52Thank you

" related talks

• 33mLessons learned building Python microservicesRichard Jones · PyCon AU 2019
• 35mDiversity and Data: Cases in the Music IndustryDelger Enkhbayar, Mike Sumner · PyData London 2017
• 21mWhen R Met Python: A Meet Cute on Posit ConnectBlake Abbenante · posit::conf(2025)
• 44mPython in the MicroServices worldHari Krishnan · PyCon Singapore 2016
• 26mBuilding A Python Service Using Open Source Components That Connect, Transform & Deliver Data In RTTomas Neubauer · Big Data LDN 2022
• 19mPython for Cloud Services and Infrastructure ManagementBhaumik Shukla · EuroPython 2015
• 40mPython microservices on PaaS done rightMichał Bultrowicz · EuroPython 2015
• 26mPython as a Universal BackendKix Panganiban · Pycon PH 2017