Data Acquisition Made Easy
summary
Socialgist supplies an enterprise pipeline for Reddit with three concrete delivery options: an HTTP full feed of every post and comment (20M monthly posts cited), a filtered HTTP stream by rules, and a REST-indexed archive for historical queries (latency under 10s). Additional feeds include compliance deletions and a votes stream for score/time-series. Indexing uses Manticore-like search; a Tableau workbook surfaces top subreddits, subscribers, and an index score. Example analysis finds CGI mentioned 32 times and faces 14 times, shifting sentiment versus top-post NLP.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 4:48Who is Social Geist
- 9:43Data Delivery Options
- 13:18Tools
- 16:34Examples
- 19:26Reddit Info
- 22:28Text
- 26:38Analysis Example
- 29:58Tableau Workbook
- 38:03Questions
related talks
56mSocial Listening, Data Extraction & Discovery
31mHow Do Push Notification Recommendations at Reddit Work?
19mA History of Automatic Aggregations
31mConfessions of a Data Junkie: Fetching, Parsing, and Visualization
14mHarnessing Common Crawl for AI and ML Applications
01h 05mSocial Text, Sentiment, and Tone Analysis
45mData as culture as software
41mOpen Source Social Applications with Apache Streams