Processing Hotel Reviews with Python
summary
Pipeline processes text with Python plus Hadoop and Luigi, handling 30 million reviews crawled daily, supporting 25 languages and over 500,000 properties with ~200,000 new reviews per day. Code uses Hadoop streaming, Pig for complex transforms, and Luigi for orchestration. Word2vec via Gensim trains hotel and city vectors from review sentences so similar hotels cluster and power a prototype recommender; demo works unevenly but shows concrete, reproducible gains.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 2:23Problem description
- 8:00Architectural requirements
- 10:51Python and Hadoop
- 13:02Orchestrating with Luigi
- 21:45Pig for transformations
- 24:58Review processing demo
- 28:10Word2Vec for representations
- 36:45t-SNE visualizations
- 38:31Hotel recommendations
- 41:26Key takeaways and conclusion
- 42:32Q&A
related talks
43mDetecting fake reviews at scale using Apache Spark and John Snow Labs
01h 31mModern NLP in Python
31mDeep Learning Techniques for Natural Language Processing
26mReview Analysis: an Approach to Leveraging User-Generated...
34mActionable Insights with Real-time Streaming Analytics of Customer Reviews
01h 22mWhen Rotten Tomatoes Isn’t Enough: Analyzing Twitter Movie Reviews
44mMachine learning @ booking.com
21mFrom 0 to a crowd sourced neural network in a jiffy