Evaluating Text Extraction: Apache Tika's™ New Tika-Eval Module
summary
Apache Tika™ adds a new tika-eval module that runs file-share to file-share, profiles extracts into an H2 DB and emits XLSX reports to detect regressions. It compares two runs (profile vs compare) and flags surprises found on a public Rackspace VM over a terabyte corpus (≈3 million files from Common Crawl and Gov Docs one), including a PDFBox upgrade that produced a substitution-cipher output and a batch where 50% of Excel embedded files threw exceptions. Reports require human inspection but pinpoint specific failing files and stack traces.
AI-written summary · about
chapters (32)
- 0:00Introduction
- 2:10Overview
- 3:15Whats different
- 3:51Content Extraction
- 5:30Metadata
- 6:45Blood on the Highway
- 7:40Search
- 8:31Regression Testing
- 13:48Hidden Problems
- 15:29Example of Missing Text
- 18:30Dream
- 19:57Evaluation Metric
- 20:18TikaEval Overview
- 23:03TikaEval Definitions
- 24:49Why TikaEval
- 26:27TikaEval
- 27:21Profile
- 29:04Compare
- 29:41StartDB
- 29:56Profile Reports
- 31:55Common Words Metric
- 33:18Similarity Metric
- 34:05Common Word Metric
- 36:30Evaluation Metric Public
- 37:29Limitations
- 38:34Human Interpretation
- 39:24Conclusion
- 39:45Resources
- 40:21Thank you
- 40:41Data import handler
- 41:50Metadata normalization
- 42:56Application dependent
related talks
42mExtract information accurately from documents using computer vision
16mSoK: Automated TTP Extraction from CTI Reports – Are We There Yet?
31mTaking AI/ML to the next level: Text!
32mAutomate data extraction & analysis with intelligent document processing
32mAutomate data extraction & analysis with intelligent document processing
01h 00mExtract information using the Queries feature in Amazon Textract
23mText analytics on annual reports
30mProcessing Threat Reports at Scale Using AI and ML: Expectations and Reality