Using HBase Coprocessors to Implement Prospective Search
summary
Prototype embeds HBase coprocessors per table, hooks postPut, converts each put into a Lucene in-memory index document, runs all user queries (agents) against it and emits email alerts; uses HBase on HDFS, MapReduce, Apache Solr, Mahout and the HBase Rollag queue. Prototype is not production: naive implementation recreates index per put and must be deployed via classpath/table descriptor requiring restarts; measured write throughput falls from ~1600 ops to ~600 ops with 10 agents — write throughput drops >50%.
AI-written summary · about
chapters (4)
- 0:00Introduction and motivation
- 10:11Prospective search with coprocessors
- 23:55Challenges and lessons learned
- 28:15Additional resources and Q&A
related talks
28mBreakout Session
55mApache HBase: crazy dances on the elephant back
36mSearch on the fly: how to lighten your Big Data
33mBuilding domain specific databases using Python for prototypes during take off
51mHibernate Search talk
41mStream Processor as a Database: Building Online Applications
26mIdentity in a World of Values
43mCaching and Beyond - Smarter Data Processing with Java