Beautiful Python Refactoring

on YouTube ↗30m

" summary

A concrete refactor pipeline takes scraped HTML rows using requests and lxml into a list comprehension and finally into a pandas DataFrame reducing code from 60+ lines to ~10 and then to three to five lines by using pandas.read_html. The code builds a working scraper and an analysis that shows C++ dominates submissions C++ 89% Python 6% Java 4%. Notes mention cuDF as a drop-in acceleration with reported ~100x speedups.

AI-written summary · about

" chapters (13)

1. 0:00Introduction
2. 1:14Why this talk
3. 3:09How I started
4. 5:50Imports
5. 11:30Data Extraction
6. 11:45If Statement
7. 12:23Slicing
8. 14:57Conditional Expressions
9. 17:11Structural Changes
10. 23:39Step 2 HTML Table
11. 24:46Step 3 Code
12. 26:46Performance
13. 28:16Python Bites

" related talks

• 01h 33mFrom Raw Recruit Scripts to Perfect PythonStanley van der Merwe, Petr Wolf · PyData New York City 2019
• 30mSpeed Up Your Data ProcessingChin Hwee Ong · PyData Global 2020
• 25mPandas on Jupyter: Improving Your Data Analysis WorkflowNick Macro · PyOhio 2018
• 01h 55mAll Them Data Engines: Data Munging with Python circa 2023Shaurya Agarwal · PyData Global 2023
• 31mTechnical Lessons Learned from Pythonic RefactoringYenny Cheung · PyCon.DE 2017
• 53mPython strategy panelMiki Tebeka, Orit Wasserman, Michael Czeizler, Barak Peleg, Lior Mizrahi, Oren Nissenbaum, Tal Sayhonov · PyCon Israel 2022
• 19mPython - best practicesRanjan Mohan · Conf42 Python 2021
• 40mHow To (Semi-)Automate JavaScript RefactoringGiles Bowkett · Fluent 2013