MapReduce Programming in Python with Pydoop

on YouTube ↗01h 23m

" summary

Pydoop is a Python library for writing Hadoop MapReduce programs. The talk presents how to use Pydoop to implement MapReduce jobs in Python, covering its API and integration with Hadoop to run distributed data-processing tasks.

AI-written summary · about

" chapters (13)

1. 0:00Parallel programming and MapReduce
2. 11:24What is MapReduce
3. 17:59MapReduce implementations
4. 21:09Hadoop architecture
5. 30:23Hadoop pseudo-distributed setup
6. 37:50Python MapReduce options
7. 46:25Pydoop: Python API for Hadoop
8. 51:44Writing MapReduce with Pydoop
9. 1:00:03Pydoop advanced components
10. 1:05:03Pydoop performance comparison
11. 1:10:46MapReduce in computational biology
12. 1:15:12MapReduce strengths and limits
13. 1:19:31Debugging MapReduce apps

" related talks

• 44mBig Data with Python & HadoopMax Tepkeev · EuroPython 2015
• 50mBig Data Processing and Data Science Environment in PythonRick Bahague · Pycon PH 2017
• 44mUsing Python with Hadoop to create production ready big data applicationsUlrich Zink · PyData London 2016
• 01h 22mIntroduction to Big Data Processing using Spark and PythonRaoul-Gabriel Urma · PyData NYC 2018
• 41mMassively Parallel Processing with Procedural PythonRonert Obst · PyData Berlin 2014
• 37mData Science with Python and RLuciana Tanus · DevDay 2018
• 25mAutomated Scientific Pipelines With DoitSam Hames · PyCon Australia 2015 Science/Data Miniconf
• 39mHow Python Drives Modern Data EngineeringKiana Alessandra Villaera · PyCon Philippines 2023