Berlin Buzzwords 2024
2024-06-09..2024-06-11
-- online --
[Berlin Buzzwords 2012, Berlin Buzzwords 2015, Berlin Buzzwords 2016, Berlin Buzzwords 2018, Berlin Buzzwords 2019, Berlin Buzzwords 2020, Berlin Buzzwords 2021, Berlin Buzzwords 2022, Berlin Buzzwords 2023, Berlin Buzzwords 2024, Berlin Buzzwords 2025, Berlin Buzzwords 2026]
| title | author | length |
|---|---|---|
| The Paradox of Open: Can Digital Commons Offer a Way Forward? | Zuzanna Warso | 49m |
Argues that openness both enables and is co-opted into concentrated power, citing concrete cases: 400 million CC photos from Flickr get scraped into… | ||
| Improving Search @scale with efficient query experimentation | Andreas Wagner | 42m |
A production “Query Testing” platform routes traffic to experiment on a query-level randomization unit (not users/sessions), auto-identifies candidates… | ||
| Brick-by-Brick: Exploring the Elements of Apache Kafka® | Danica Fine | 46m |
Covers concrete Kafka mechanics: events as immutable records, topics split into partitions with offsets, a 3-node cluster example with replication factor 3 and… | ||
| Heap sizing and GC tuning for Solr and friends | Radu Gheorghe, Rafał Kuć | 21m |
Solr/Elasticsearch-style nodes need heap sized from measured peaks not rules of thumb: take highest peak and lowest peak and use twice the low as a baseline… | ||
| Hybrid Search with Apache Solr Reciprocal Rank Fusion | Alessandro Benedetti | 45m |
Implements Reciprocal Rank Fusion inside Apache Solr 9.7 as a QueryComponent that accepts multiple labeled JSON queries, pulls top-K lists (configurable… | ||
| Accelerating TopK Queries | Juliane Waack | 20m |
Snowflake adds runtime pruning for TopK SQL by taking the boundary from sort-with-limit during execution and comparing it to per-partition minmax + nulls… | ||
| End-to-end pipeline agility | Lars Albertsson | 42m |
Quantitative claim: adapting downstream pipelines ranges from median 6 months to some teams doing it in less than a day, a gap of 100x–1000x. The setup uses… | ||
| Moving from Offline to Online Machine Learning with River | Tun Shwe | 23m |
River brings online ML to Python by training models on streaming data one event at a time and updating state incrementally, which cuts memory/CPU compared with… | ||
| Flink's SQL Engine: Let's open the engine room! | Timo Walther | 32m |
Walks the SQL string through parsing, validation, rule-based and cost-based rewrites, physical planning and mapping to Flink stream execution nodes, showing… | ||
| End-to-End Encryption for Streaming Data Pipelines | Hans-Peter Grahsl | 27m |
Kryptonite for Kafka provides client-side field-level cryptography for Apache Kafka, implemented as a custom SMT for Kafka Connect that encrypts selected JSON… | ||
| Fixin the Hard Bits of Event Processing with Restate & Kafka | Stephan Ewen | 41m |
Claim: durable execution is the missing piece for transactional event-driven apps, playing the role stream processing plays for analytics. Uses Kafka as… | ||
| Learning to Rank for Reddit Search: A Project Retro | Doug Turnbull, Charles Njoroge | 43m |
Reddit integrates Learning to Rank into its existing Solr stack using the Solr LTR plugin and trains XGBoost-style tree models on click+dwell labels, adding… | ||
| Synergy of Signals: Traffic Logs Meet LLM Labels | Stefana Serban | 22m |
E‑commerce search relevance is built by merging user traffic logs with LLM labels to improve learning‑to‑rank targets: unknown traffic scores get LLM checks… | ||
| Evolving Yelp's Streaming at Scale | Ashish Khatkar | 37m |
Yelp runs streaming on Flink with a custom control plane, schema registry and CLIs, and currently operates 50 Flink services across 250 clusters running… | ||
| `New` Workflow Orchestrator in town: "Apache Airflow 2.x" | Jarek Potiuk | 42m |
Apache Airflow 2.8+ and 2.9 are presented as a modern, evolving orchestrator with concrete results: 2,900 contributors, regular quarterly releases, and… | ||
| From Natural Language to Structured Solr Queries using LLMs | Ilaria Petreti, Anna Ruggero | 41m |
Prototype converts natural-language questions into structured Solr queries by using an instruction-tuned LLM to do two steps: JSON field-extraction from a Solr… | ||
| Practical introduction to OpenTelemetry tracing | Nicolas Fränkel | 40m |
OpenTelemetry gets the request ID across a polyglot e‑commerce demo (Kotlin/Spring Boot, Rust inventory, Go and Ruby warehouses, Python pricing, Redis cache… | ||
| Shattering the Limits of Search with Domain Specific Computing | Ohad Levi | 21m |
Claims a purpose‑built search chip and SPU architecture (FPGA instances on AWS) that runs hybrid lexical+vector search, rebuilding indexing… | ||
| cuVS and Lucene: GPU-based Vector Search | Corey J. Nolet, Vivek Narang | 43m |
NVIDIA cuVS provides CUDA implementations for ANN and clustering and delivers a novel KAGRA graph built on GPU that batch-constructs indices far faster than… | ||
| S3 as the state store for stream processing systems | Yingjun Wu | 40m |
RisingWave is a production stream processor that uses S3 as state store with a storage/compute separation to support large states without local disk limits… | ||
| Better search relevance using Learning to Rank at mobile.de | Manish Saraswat | 25m |
mobile.de replaces the Elasticsearch LTR plugin with a Python ranking microservice (FastAPI) that reads Kafka events, writes to BigQuery, uses Airflow to sync… | ||
| Back to the Future! Time Travel with Bitemporal Databases | Tim Zöller | 42m |
A logistics app records bills of lading and must reproduce exactly what data was shown to authorities, so a handcrafted Postgres solution uses a vessel_name… | ||
| Can ChatGPT build a Data Platform faster than a developer? | Chloe Caron | 25m |
A timed race reconstructs an energy generation dashboard from ElectricityMap data and shows concrete differences: one approach ingests with Airbyte → BigQuery… | ||
| Monitoring your home, with DevOps observability tools | Nick Burch | 21m |
Cheap Wi‑Fi and Arduino devices cost about “a beer” each (three for a beer-era price on Arduinos) and battery sensors wake ~every 5 minutes, so data ingestion… | ||
| Let's Do Data Lineage in Kafka, Flink and Druid! | Hellmar Becker | 33m |
Demonstrates a working pipeline that collects ADS‑B aircraft messages from Raspberry Pi ADS‑B feeders, sends them into Kafka with Kafka headers carrying… | ||
| How we isolate streaming ingest from search using RocksDB | Igor Canadi | 39m |
Disaggregates streaming ingest and query by turning each JSON update into RocksDB key/value physical deltas on dedicated ingestion workers and streaming those… | ||
| Large language models are not a paradigm shift | William Benton | 45m |
Argues that LLMs are not a paradigm shift for computing but a new, powerful instance of familiar ML systems, showing concrete analogies: fourth‑order Markov… | ||
| Improve LLM-based Applications with Fallback Mechanisms | Bilge Yücel | 38m |
Shows how to harden RAG apps by wiring fallback branches in Haystack: build pipelines of embedder → retriever → prompt builder (Jinja-style templates) →… | ||
| Remote work is here to stay: and what's next? | Radovan Bacovic | 36m |
Claims that all-remote scales: a company with 1600 colleagues from 56 countries and 30+ million customers operates with 0 offices by treating remote as a… | ||
| Build your 8-bit computer from scratch | Olivier Huber | 33m |
A homemade 8‑bit computer on breadboards executes a 4-line binary program that loads 5 (101) and 3 (011), runs a hand‑built adder/ALU and registers, and… | ||
| Rediscover your keyword search: Expand, Enrich and Rewrite | Praveen Mohan Prasad, Hajer Bouafif | 44m |
Shows practical steps to get semantic ranking without full vector search by adding three concrete pieces: neural sparse (SPLADE/OpenSearch) expansion to… | ||
| Can Apache Pinot replace your OLTP database? | Chinmay Soman | 37m |
Compares Apache Pinot to OLTP using a live Docker demo that wires MySQL → Debezium → Kafka → Pinot and shows CDC-driven real‑time upserts and backfill with the… | ||
| Advancements in Evaluating Large Language Model Applications | Petr Polezhaev | 42m |
Explains concrete evaluation recipes: end-to-end business metrics for customer‑support summaries and reply suggestions, plus task‑level metrics… | ||
| Sequential Testing Simplified with Basemath | Alexander Weiss, Konrad Richter | 33m |
GetYourGuide reports 40% reduction in experiment timelines by using Basemath, a Python package (pip install basemath-analysis) that implements sequential… | ||
| Open Web Search: A platform for a free European Web Index | Michael Dinzinger | 22m |
EU-funded project builds an open web index as public infrastructure, using a distributed crawler (OWLer) based on StormCrawler and URLFrontier, storing raw… | ||
| Streaming DataFrames: A New Way to Process Streaming Data | Tomas Neubauer | 20m |
An open source Python library Quix Streams exposes a Pandas DataFrame API for streaming, handling checkpointing, state, and PVC-backed storage so stateful… | ||
| Kafka on the Fly: A Serverless Approach to Data Streaming | Desmond Obisi | 38m |
Serverless Kafka using Upstash and KafkaJS demonstrates a concrete pipeline: a Docker Compose microservice produces to a topic and a consumer stores JSON user… | ||
| Comparing vector implementations in generic databases | Tudor Golubenco | 22m |
Compares pgvector on Postgres versus Lucene-based vectors in OpenSearch/Elasticsearch using a real RAG case: 5 million vectors, ~1,645 dimensions, ~1,000… | ||
| The “C” in CI/CD is not for “Closed” | Josh Reed | 33m |
Argues that closed-source CI/CD (GitHub Actions, Buildkite, CircleCI) now runs much of open source and that this creates vendor lock‑in; proposes building a… | ||
| A Practical Approach To Semantic Search | Kentaro Takiguchi | 33m |
Compares lexical vs semantic on Amazon-ESCI product data and shows that when both are fine-tuned they reach similar NDCG but semantic search has much lower… | ||
| Text to Context: How We Introduced Hybrid Search | Ansgar Gruene, Dharin Shah, Alexander Butenko | 41m |
Claim: a hybrid system on top of OpenSearch + KNN plugin combines BM25, vector cosine and a product popularity score via a learned linear combiner and yields… | ||
| Harnessing Spare Cores to Breeze Through Cloud Compute | Gergely Daroczi | 22m |
Spare Cores is a Python open‑source ecosystem that inventories cloud offerings, runs hardware discovery and benchmarks (stress‑ng, Geekbench 6, IO/compression… | ||
| The Unsung Hero of Vector Database: Metric Learning | Sonam Pankaj | 18m |
Metric learning is presented as the fix behind better vector search: using Siamese and triplet architectures with contrastive and triplet loss to stop… | ||
| Applications of Tries in Apache Cassandra | Branimir Lambov | 36m |
Apache Cassandra replaces comparison-based memtables and SSTable indexes with tries and reports concrete wins: about 2x faster random SSTable reads and over 2x… | ||
| Hardcoding airpods (and other stories from NLP in insurance) | Murhaf Fares, Emanuele Lapponi | 30m |
Worked with real Norwegian claims data and ships concrete solutions: built a “Netflix of unhappy customers” Streamlit app that ranks messages using classifiers… | ||
| Advanced Retrieval-Augmented Generation Techniques | Zain Hasan | 43m |
Applies RAG to a medical-chatbot pipeline using a 170,000 patient-summaries plus 1.4M-abstracts PMC patients dataset, indexes them into an open-source vector… | ||
| Learning to Apply Generative AI at Enterprise Level | Sebastian Arnold | 22m |
Claims practical enterprise rollout: a private myGenesis platform integrates ChatGPT and hostable open‑source LLMs, supports 85 proposals distilled into six… | ||
| Under the hood of vector search with JVector | Joel Knighton | 40m |
JVector is a Java embedded library that implements graph-based ANN with PQ rerank to run indexes in memory and on SSD, aiming for high recall and low latency… | ||
| Lessons learned writing 10+ Kubernetes Operators | Lars Francke, Jannik Heyl | 43m |
Built and running over 10+ operators for on‑prem data platforms, the talk lists concrete tradeoffs: authors extract shared logic with a Rust framework (Rust… | ||
| Streaming doesn’t have to be hard | Chloe He | 39m |
Shows a working stream-batch unified Python dataframe API that keeps transformation logic separate from execution, parses Python dataframe calls and compiles… | ||
| Deep Learning plays Handball | Pere Urbón-Bayes | 38m |
Uses only player coordinates and shows what actually works: feature engineering with centroid distances plus possession time reaches about 0.76 F1… | ||
| Jina Embeddings V2: From Raw Data to Bilingual Hybrid Search | Bo Wang, Isabelle Mohr | 42m |
German‑English embedding model trained on ~400–500M web documents with ~50/50 English and German data, using a BERT/ALiBi backbone for long context, mined… | ||
| Standing on the Shoulders of Giants | Bryan Burkholder, Varun Thacker | 33m |
Slack builds Astra, an open source log search engine that ingests 5 million messages a second (≈2 trillion messages over 7 days) to replace 25 Elasticsearch… | ||
| Recognizing the driving forces of impactful OS projects | Maryblessing Okolie | 34m |
Six out of ten contributors say open source is code‑centric, so CHAOSS builds visible paths for no‑code work: the tour guide initiative moves newcomers to… | ||
| Apache Lucene: From Text Indexing to Artificial Intelligence | Lucian Precup | 40m |
Apache Lucene traces 22 years of concrete changes: inverted index plus text analysis, FSTs that made fuzzy search 100x faster, doc values that turn the index… | ||
| Elevating AI Applications with OpenSearch's Flow Framework and RAG Tool | Mingshi Liu, Owais Kazi | 27m |
OpenSearch 2.13 adds a Flow Framework that automates the full RAG pipeline: it creates connectors to embedding and LLM endpoints, registers models, builds… | ||
| Cracking the Code: Deciphering Evaluation Essentials for RAG | Atita Arora | 22m |
RAG accounts for 65–70% of generative AI use cases and evaluation focuses on precision, relevance, faithfulness, groundedness, coherence, summarization, safety… | ||
| Open-Source Generative AI: A Product Manager’s Blueprint | Saahil Ognawala | 37m |
Roadmap for building open source generative AI products that actually sell and scale with specifics on models tools and outcomes including Gina embeddings V2… | ||
| Blazing-Fast Serverless MapReduce Indexer for Apache Solr | Daniele Antuzi | 26m |
An indexer built as a serverless MapReduce pipeline uses AWS Lambda + Step Functions to fetch SQL query shards, write per-song intermediate files to S3, run… | ||
| Robust AI Search Ranking for Radical C2C Marketplace Growth | Teo Narboneta Zosa, Chingis Oinar | 44m |
Mercari deploys a decoupled post-retrieval ML reranker (ElasticSearch retrieval + MLP learning-to-rank) serving tens of thousands QPS with p99 latency in the… | ||
| A journey in geospatial timeseries | Nils Larsgård | 18m |
An application ingests AIS from the Norwegian Coastal Administration and navigational maps from the Norwegian Mapping Authority into PostgreSQL using jsonb for… | ||
| Search in the Cloud: separation of compute and storage | Ilan Ginzburg | 46m |
Modified SolrCloud stores one copy of each shard on shared object storage (S3) and makes JVM nodes stateless caches, so nodes can be added or removed without… | ||
| Enhancing RAG with Neo4j Knowledge Graph | Djordje Benn-Maksimovic | 39m |
Claims that coupling Neo4j graph DB with vector+graph retrieval lets RAG answer cross‑document and “global” queries. A multilingual news graph is built by… | ||
| Taming the cost of Kafka workloads in the cloud | Stefan Sprenger | 36m |
Cloud Kafka workloads show network as the biggest surprise: a simple app at 100 MB/s in/out can incur >$3,000/month just for cross‑AZ traffic. Enable follower… | ||
| The Power of the Bonus Card: Road to Personalised Search | Luuk Kaandorp, Vincent Peijnenburg | 38m |
Albert Heijn uses Bonus Card purchase histories to move from popularity to personalized product search, training a listwise Learning-to-Rank model with XGBoost… | ||
| nothing matches | ||