Berlin Buzzwords 2023
2023-06-18..2023-06-20
-- online --
[Berlin Buzzwords 2012, Berlin Buzzwords 2015, Berlin Buzzwords 2016, Berlin Buzzwords 2018, Berlin Buzzwords 2019, Berlin Buzzwords 2020, Berlin Buzzwords 2021, Berlin Buzzwords 2022, Berlin Buzzwords 2023, Berlin Buzzwords 2024, Berlin Buzzwords 2025, Berlin Buzzwords 2026]
| title | author | length |
|---|---|---|
| What defines the "open" in "open AI"? | Jennifer Ding | 55m |
Survey of concrete cases and tools: startup Parkit used OpenCV and scikit-learn for parking detection; YOLO’s public-domain release catalyzed industries… | ||
| Vectorize Your Open Source Search Engine | Atita Arora | 19m |
Shows how to add vectors into an existing Lucene/Elasticsearch pipeline using off‑the‑shelf Hugging Face models, a lightweight encoding service, and Chorus… | ||
| When Probably is Good Enough | Savannah Norem | 37m |
Examines Redis Stack’s bloom filter, count min sketch, and top K with concrete numbers and a demo: Pride and Prejudice text and an 813,695-usernames file… | ||
| Kaldb: serverless lucene at petabyte scale | Suman Karumuri | 42m |
An open-source Lucene serving system claims to run petabyte-scale logs on Kubernetes by separating compute, durability and storage: Kafka acts as a write-ahead… | ||
| Boosting Ranking Performance with Minimal Supervision | Jo Kristian Bergum | 41m |
With just three labeled queries and documents, an LLM (including open-source 3B models or GPT-style APIs) generates synthetic questions for every document… | ||
| Declarative Data Collections for Portable Parallelism | Zhibo Li | 18m |
Declarative collection skeletons expose eight property groups (semantic, interface, hybrid) and use C++ template metaprogramming to let programmers declare… | ||
| How to train your general purpose document retriever model | Tom Veasey, Quentin Herreros | 41m |
A learned sparse model based on SPLADE-style learned sparse model architecture is fine-tuned on ~1M queries (Stack Overflow titles + Google Natural Questions)… | ||
| ClickHouse: what is behind the fastest columnar database | Olena Kutsenko | 42m |
Open source ClickHouse claims throughput of billions of rows and tens of gigabytes per second by using columnar storage with per‑column files and heavy… | ||
| Migrate Data, Mesh in mind | Aydan Rende | 21m |
A migration replaces an on‑prem Hadoop warehouse with a cloud pipeline that actually runs: Kafka topics → DataHub data‑sync writing Avro→Delta into S3 →… | ||
| Supercharging your transformers with synthetic query generation and lexical search | Milind Shyani | 43m |
A 1.5B-parameter GPT-style model is fine-tuned on 650k MSMarco/Natural Questions pairs to generate 20 synthetic queries per document, producing a synthetic… | ||
| Apache Airflow in Production - Bad vs Best Practices | Bhavani Ravi | 36m |
Recommends upgrading to modern Airflow (avoid brand‑new .0 releases; prefers 2.5.3 unless a feature forces 2.6), use Postgres not SQLite and front it with PG… | ||
| Privacy-Preserving Web Search | Lara Perinetti | 18m |
Quant operates a privacy-first search stack: no tracking cookies and only hashed IPs, no sessions, clicks logged but de-identified. Traffic is large — about… | ||
| Using TensorFlow in a Solr Query Parser | Radu Gheorghe, Rafał Kuć | 42m |
A Python TensorFlow model does query tagging while BERT tokenization runs outside the model because TensorFlow for Java lacks tokenizer support; the SavedModel… | ||
| Introducing Multi-valued Vector Fields in Apache Lucene | Alessandro Benedetti | 40m |
Adds multi-valued KNN vectors to Lucene by making each HNSW node a vector ID and building an ordinal map to documents so the index can map ordinal vector ID to… | ||
| Towards a decentralized and collaborative search engine | Aline Paponaud, Lucian Precup | 35m |
A collaborative search engine called all.site combines curated, community-run indices with ML question answering, using Hugging Face QA models over… | ||
| Big data in the service of reliable news | Radu Pop, Benjamin Dauvissat | 19m |
Argument: using large, heterogeneous datasets plus curated trustworthy sources makes truth emerge faster and lowers the cost of debunking against Brandolini’s… | ||
| Building On-Ramps for Non-Code Contributors in Open Source | Natali Vlatko, Celeste Horgan | 44m |
Shows concrete practices from Kubernetes and SIG Docs: mandate 5 PRs for org membership, run a twice‑year contributor summit and yearly awards, use enhancement… | ||
| How to not kill people | Lars Albertsson | 42m |
Compares aviation safety to software and AI with concrete takeaways: aviation averages about 270 deaths per year and still treats every incident (Habsheim… | ||
| The Debate Returns (with more vectors): Which Search Engine? | Jo Kristian Bergum, Alessandro Benedetti, Etienne Dilocker, Philipp Krenn, Kacper Łukawski, Charlie Hull | 54m |
Five open-source engines compare concrete tradeoffs: one reports 48 billion vectors indexed in a single cluster and adds GPU-accelerated embedding inference… | ||
| Tip of the Iceberg | Fokko Driesprong | 41m |
Apache Iceberg is an open table specification that lets SQL-like ACID operations run directly on object storage, using manifest lists and manifests to provide… | ||
| Creating chaos in containers | Maish Saidel-Keesing | 37m |
Explains concrete chaos practices for containers: define four health checks — liveness checks, local, dependency and anomaly — and test CPU, memory and… | ||
| Who broke the build? Using Kuttl to test and Release faster | Ram Mohan Rao Chukka | 33m |
Team replaces slow remote CI e2e runs with Kuttl to reproduce production-like Kubernetes environments locally, spinning up a kind cluster per test namespace… | ||
| No Mean Feat: Upgrading a Customized Solr to Upstream Solr | Shikhar Srivastava | 19m |
Bloomberg migrates a heavily patched Apache Solr fork to upstream Solr while keeping 750M documents (126 TB), 22M queries/day (median 90 ms) and 2.2M new… | ||
| Model Fine-tuning For Search: From Algorithms to Infra | Maximilian Werk, Bo Wang | 39m |
Fine-tuning turns pretrained encoders into searchable embeddings by cutting classification heads, attaching pooling layers and training with… | ||
| Ingesting over 4 million rows a second on a single instance | Javier Ramirez | 43m |
Claims ingesting over 4 million rows/sec on a single instance using QuestDB and a 16‑CPU benchmark; a live demo on an SSD Mac shows ~3.2M/sec. Implementation… | ||
| A Kafka Client’s Request: There and Back Again | Danica Fine | 42m |
Follows a Kafka client request from producer.send() through serialization, partitioning, batching and optional compression to the broker, where network threads… | ||
| Joining Dozens of Data Streams in Distributed Stream Processing Systems | Yingjun Wu | 22m |
Claims that decoupling compute and storage by keeping state in S3-backed state lets a distributed SQL streaming database join dozens of streams without… | ||
| Laptop-sized ML for Text, with Open Source | Nick Burch | 47m |
Demonstrates running modern open models locally: LLaMA on a laptop (released in 13–122 GB sizes) and several community forks and tools (OpenLLaMA, MPT… | ||
| ML with Domain-Specific Ontology for IT Security Industry | Qi Wu, Bertram Sändig | 40m |
A pipeline for BSI-style IT security monitoring combines a curated ontology with machine learning: ontology relation vectors (one dimension per related entity)… | ||
| Building Real-Time Applications: Cyclist Crash Detection | Tomas Neubauer | 40m |
A real pipeline processes phone GPS and accelerometer telemetry from a fitness app into Apache Kafka then into a Python microservice that loads a pickled… | ||
| Cooking up a new search system: Recipe search at Cookpad | Matt Kelly Williams | 30m |
Cookpad replaces a brittle legacy search with a new event-driven, Python-centric stack (Kubernetes, Kafka, Python, Elasticsearch, ML) over two years, keeping… | ||
| What's coming next with Apache Lucene? | Uwe Schindler | 22m |
Around Lucene 8 most people thought "There's not much that can be done anymore". In contrast to that, if you look into Apache Lucene's list of new features… | ||
| Building MLOps Infrastructure at Japan's Largest C2C E-Commerce Site | Ryan Ginstrom, Teo Narboneta Zosa | 41m |
Mercari adds learning-to-rank as a re-ranker over Elasticsearch, serving a small DNN (trained with TF‑ranking / PyTorch) via a separate RPC microservice with… | ||
| Synthetic data: when, why, and how | William Benton | 40m |
Argues synthetic tabular data is practical for privacy, load testing, and rare-event simulation, showing concrete techniques: sampling marginals, sampling… | ||
| Search saves lives: solving healthcare problems with search | Chris Hutchinson | 37m |
NHS testing moves from straight‑line distance to travel‑time matching using a driving model + graph traversal that returns driving times for up to 100,000… | ||
| Advanced Search Plays with GraphQL | Stanimira Vlaeva | 01h 04m |
Shows a hands-on build: a football dataset of ~19,000 player documents is indexed with MongoDB Atlas using Apache Lucene analyzers and a visual index, queried… | ||
| How to Implement Online Search Quality Evaluation with Kibana | Ilaria Petreti, Anna Ruggero | 38m |
A reproducible Kibana pipeline that runs A/B comparisons of two rankers against live Elasticsearch logs, indexing interactions with fields for book_id… | ||
| Avoiding Anti-patterns in Technical Communication | Sophie Watson | 21m |
Lists four concrete anti-patterns: keeping it brief (example: “Notebook won’t run” Slack messages that force extra triage), chasing likes (99% accuracy… | ||
| Connect GPT with your data: Retrieval-augmented Generation | Malte Pietsch | 41m |
Shows how to connect LLMs to private company data using retrieval-augmented generation with concrete code in Haystack, a live demo that swaps plain GPT answers… | ||
| A Fresh Start? The Path Toward Apache Solr's v2 API | Jason Gerlowski | 39m |
Apache Solr admits its V2 HTTP API effort started in 2017 as a JSON-based translation layer over V1 that decays when V1 changes, so the project pivots to… | ||
| Rethinking Autoscaling for Apache Solr using Kubernetes | Houston Putman | 43m |
Shows how Kubernetes HPA can drive SolrCloud scaling by extending the Solr Operator to call Solr APIs: it uses the replace node API to safely drain pods on… | ||
| Highly Available Search at Shopify | Khosrow Ebrahimpour | 35m |
Shopify runs ~2 PB of search data across ~114 Elasticsearch clusters (largest ~120 nodes) and sustains a background indexing rate around ~90k/sec with reindex… | ||
| When ms matter: Maximizing query performance in CrateDB | Marija Selakovic | 23m |
Shows concrete CrateDB tactics: use EXPLAIN/ANALYZE to inspect logical plans and see Lucene queries, prefer hash join over nested-loop when possible (linear… | ||
| Deep dive into an Elasticsearch plugin for query-time joins | Stéphane Campinas | 41m |
Siren Federate implements query-time joins on Elasticsearch by hash-partitioning documents across nodes, building an in-memory hash table per shard and… | ||
| From keyword to vector | Byron Voorbach | 25m |
Ten years of e-commerce work leads from managing brittle synonym lists and tokenization bugs (a 52/48 split on "wine glass" spellings) through costly… | ||
| Semantic vs keyword search as context for GPT | Tudor Golubenco | 22m |
Compares semantic/vector search to classic keyword (BM25) search as the step that selects documentation context for GPT and finds accuracy is essentially a… | ||
| Alexa, is The Smart Home vision failing? | Steve Loughran | 39m |
Amazon losing $10 billion on Alexa highlights that the business model fails to cover hardware plus cloud maintenance, and voice features misroute music because… | ||
| Hadoop Vectored IO: your data just got faster! | Steve Loughran | 31m |
Hadoop adds a new Filesystem API called vectored read (targeted for Hadoop 3.4) that accepts lists of byte ranges and memory allocators so object-store clients… | ||
| A Crash Course in Error Handling for Streaming Data Pipeline | Stefan Sprenger | 38m |
Covers concrete patterns for Kafka-based pipelines: use retry with exponential backoff for transient network or broker failures and set retries>0 and… | ||
| Fact Checking Rocks: how to build a fact-checking system | Stefano Fiorucci | 37m |
A baseline fact‑checking web app checks user statements about rock music against a Wikipedia corpus using Haystack + FAISS for vector storage… | ||
| Learning to hybrid search | Roman Grebennikov, Vsevolod Goloviznin | 47m |
Uses the Amazon ESCI dataset and Metarank to compare BM25, LambdaMART LTR, sentence-transformer miniLM embeddings and cross-encoders; BM25/LTR sit around ~76… | ||
| Searching large data sets in (near) constant time | Torsten Bøgh Köster, Dennis Berger | 41m |
Claims a generic Solr-side system that limits expensive searches to a 60k cap and a 400–500 ms budget by estimating hit counts from tiny index buckets, using… | ||
| Platform Engineering is All About Product | Gal Bashan | 44m |
Argues that an internal developer platform must be treated as a product: build around a problem-first mindset, validate with users, ship tiny experiments and… | ||
| Using Dense Vector search at the EU Publications Office | Martin Bayton | 39m |
Proof of concept adds Elasticsearch dense vector support plus Google BERT (Hugging Face) embeddings, distilBERT QA and Pureinsights Discovery Platform to a… | ||
| Cross Data Center Replication in Solr - A new approach | Anshum Gupta | 41m |
New Solr XDC module implements scalable cross‑data‑center replication by pushing mirrored update requests into Kafka and consuming them into remote clusters… | ||
| Tiny Flink — Minimizing the memory footprint of Apache Flink | Robert Metzger | 20m |
Explains how to run Apache Flink inside one JVM using Flink’s MiniCluster, trimming JVM native memory by lowering thread stacks, switching GC, and tuning Flink… | ||
| Column-level lineage is coming to the rescue | Paweł Leszczyński, Maciej Obuchowski | 34m |
OpenLineage adds a column-level lineage facet and demonstrates end-to-end extraction: a Spark listener inspects Spark logical plans at runtime and emits column… | ||
| Catch the fraud — with observability and analytics | Philipp Krenn | 22m |
Elastic’s contributor program runs on Kubernetes, Spring Boot, Spring Data and Elasticsearch and gets swamped after MacBook prizes lead to ~7 million requests… | ||
| ChatGPT is lying, how can we fix it? | Kacper Łukawski | 35m |
Claims that ChatGPT produces hallucination—confident but false facts—because pretraining stops at a cutoff (September 2021) and models only predict next… | ||
| Scalable distributed messaging&streaming with Apache Pulsar | Julien Jakubowski | 45m |
Apache Pulsar is a cloud‑native messaging and streaming platform that separates stateless brokers from durable storage (Apache BookKeeper), storing topics as… | ||
| nothing matches | ||