How we increased developer productivity 2x
summary
Pinterest reports 1,600 engineers, 400M ML inferences/sec, 60M events/sec and over an exabyte of data; analytics starts with 300,000 datasets so a metadata platform, Data Mesh and data stewards cut core tables to ~170 tier‑one tables that already serve ~40% of queries with a goal of 80%. Built open‑source tooling including Querybook and DataHub, use GitHub Copilot in a 200‑engineer POC that boosts commits 50–100% for low‑volume coders and ~40% for seniors, plus LLMs for auto‑documentation and 95%‑accurate text‑to‑SQL.
AI-written summary · about
chapters (19)
- 0:00Pinterest engineering scale
- 1:04Improving analytical productivity
- 3:53Data analytics challenges
- 6:22Data mesh approach
- 8:08Self-serve data platforms
- 10:38Standardizing tables
- 12:43Data stewards role
- 15:14Measuring productivity impact
- 15:37GitHub Copilot boost
- 17:00Copilot proof of concept
- 18:26AI docs automation
- 20:13Slack Q&A demo
- 20:34Next steps for analytics
- 21:36Q&A on alignment
- 23:27AI code safeguards
- 24:49Documenting tables with large language models
- 26:34Measuring developer productivity
- 27:56Slack Q&A integration
- 29:21What the speaker enjoys most about their work
related talks
12mEngineering Productivity, Infrastructure
38mKeeping pace with data infrastructure so it doesn't outrun you
28mInformation to Inspiration: Morgan Stanley's Data-Driven Approach to Developer Productivity
22mHow Building with AI Can Double the Throughput of Your Engineering Team
42mEvolution of Observability Tools at Pinterest
22mBuild Smarter, Don't Just Build
38mData Secrets from a Platform Engineer
39mMobile Developer Productivity Panel