Keynote: Experimenting in a Sociotechnical System in Software Engineering
summary
Company value at Honeycomb treats changes as experiments and uses observability and feature flags as instruments: a session DB audit finds 5.99 million of 6 million inserts for a pointless redirect and code change stops them, freeing space; a Lambda A/B splits traffic 50/50 then 99/1 to test a new chip, reveals Go 1.17 and a compression library as bottlenecks and after fixes yields cost wins; a navbar A/B (50% new UI) cuts clicks to “data settings” but yields no statistically significant query lift; ensemble programming increases team knowledge flow.
AI-written summary · about
chapters (9)
- 0:00Introduction and context
- 3:13What is an experiment?
- 9:01Experiments at the code level
- 14:03Testing in production
- 20:02AB testing with users
- 25:20Experiments in teams
- 35:20Action research in teams
- 46:46Scaling experiments and culture
- 51:47Conclusion and call to action
related talks
01h 02mKeynote: The Socio-technical Elements that Make Good Platforms
15mKeynote: The Self Driving Company
36m"Engineering to embrace change" Or "How clumsy engineering can lead to powerful research – Part II"
01h 16mKeynote 3: The Zen of Programming
58mKeynote: Seven Habits of a (Mostly) Successful Team
39mKeynote - The gordian knot
46mClosing Keynote
12mKeynote