Tika 2.0 -- Robustness and Scale
summary
Apache Tika 2.0 modularizes parsers (Maven modules: standard, extended, ml) and splits server into a tika server core plus optional parser bundles so deployments avoid heavy native libs. A new pipes module isolates parsing into one JVM per parse and adds fetchers (S3, GCS, HTTP, filesystem) and emitters (Solr/OpenSearch/Elasticsearch), plus an emit cache that batches JSON extracts to Solr. The project runs regression corpora (~2 TB / ~2M files) and fuzzing, finds crashes and infinite loops, and reports 4–6 hour full regression runs; the system materially improves robustness and cloud/Kubernetes scaling.
AI-written summary · about
chapters (9)
- 0:00Introduction to Apache Tikka
- 2:56Tikka 2.x and modularization
- 6:26Evaluation with Tikka Eval
- 8:21Robustness and security
- 12:26Mitigation and testing
- 17:52Tikka pipes and async
- 23:07Scaling and emitters
- 27:42Future work and integration
- 30:46Q&A session
related talks
40mWhat's with the 1s and 0s? Making sense of binary data at scale
36mScalably crashing JVMs, or why binary data to content is hard
06mLightning talk - Broadway + Sqs + Apache Tika
37mMeasuring 2.0 – Building the next generation webanalytics ...
59mPolyglot Pipelines with Apache Nifi
44mAtomic Habits in writing high performing distributed code in the cloud
24mStreaming analytics on edge with Kuiper and KubeEdge
35mA Reactive Architecture Approach for a Tax Information System Modernization