Berlin Buzzwords 2019
2019-06-16..2019-06-18
Berlin, Germany
[Berlin Buzzwords 2012, Berlin Buzzwords 2015, Berlin Buzzwords 2016, Berlin Buzzwords 2018, Berlin Buzzwords 2019, Berlin Buzzwords 2020, Berlin Buzzwords 2021, Berlin Buzzwords 2022, Berlin Buzzwords 2023, Berlin Buzzwords 2024, Berlin Buzzwords 2025, Berlin Buzzwords 2026]
| title | author | length |
|---|---|---|
| A user's perspective on open source | Isabel Drost-Fromm | 41m |
Ten years of Apache-based experience argues that downstream users must evaluate governance and process, not just code: check public archives and mailing lists… | ||
| E-Commerce search at scale on Apache Lucene | Michael Sokolov, Mike McCandless | 41m |
Amazon ports product search from a legacy C++ engine to Apache Lucene and proves it works in production, serving millions of shoppers; architecture uses… | ||
| From zero to hero with Apache Kafka's connect API | Robin Moffatt | 42m |
Live demo wires a MySQL binlog source into Kafka via Kafka Connect and pushes the same topic to two sinks: Elasticsearch and Neo4j, using Avro serialization… | ||
| Introducing Apache Iceberg: Tables Designed for Object Stores | Owen O'Malley | 39m |
Apache Iceberg replaces Hive's directory-based tables with a metadata layer of immutable snapshots and manifest files so writers produce atomic commits and… | ||
| Large-scale Data Quality Verification (...) | Philipp Schmidt | 21m |
Open-source library deequ lets teams unit-test your data by computing column-level metrics (completeness, uniqueness, histograms) on Apache Spark and… | ||
| What's evolving in Elasticsearch and Lucene? | Simon Willnauer | 40m |
Elasticsearch 7 and Lucene 8 deliver concrete changes: a “magic wand” precompute of per-term max scores boosts single-term throughput to 1,160 qps, VKD trees… | ||
| Bias in NLP 101 | Sakshi Deo Shukla | 35m |
Uses an Indian movie corpus of about 400 films (1970–2017) with roughly 5,858 female vs ~9,380 male cast entries to measure gendered descriptions and… | ||
| Better Than Deep Learning: Gradient Boosting Machines (GBM) | Szilard Pafka | 41m |
Gradient boosting routinely outperforms neural nets on relational tabular data: use xgboost, lightgbm, h2o and expect to tune learning rate, tree depth and… | ||
| Running Solr within Kubernetes at Scale | Houston Putman | 40m |
Bloomberg runs over a thousand Solr clouds and indexes tens of billions of documents and the team ports Solr onto Kubernetes by building a Solr Operator that… | ||
| Building an enterprise NLSE ... | Debmalya Biswas, Louis Baligand | 35m |
Built an enterprise QA pipeline that plugs Facebook’s DrQA reader into an ElasticSearch retriever over Office/PDF docs (ingested with Apache Tika), producing a… | ||
| Location Analytics - Real-Time Geofencing using Kafka | Guido Schmutz | 42m |
Prototype implements real-time geofencing by ingesting GPS streams into Kafka, enriching with KSQL UDFs with GeoTools (Java), and emitting enter/exit events… | ||
| Evolution of Yelp search to a ranking platform | Umesh Dangat | 21m |
Yelp migrates a custom distributed Lucene system to Elasticsearch to decouple infrastructure from relevance, moving from heap-stored maps to document_values… | ||
| Eventually, time will kill your data pipeline | Lars Albertsson | 38m |
Catalogues concrete failure modes (daylight saving causing silent hourly data loss, leap second spikes, time-zone offsets) and shows reproducible fixes: using… | ||
| 10 (funny) years of Apache Lucene hacking | Uwe Schindler | 40m |
Ten years of Lucene/Solr hacking traces concrete changes: replacing millions of short Token objects with tokenstream attributes to cut allocation and speed… | ||
| Not automating Tinder in the Cloud (...) | Sirar Salih | 33m |
A project automates Tinder via its undocumented API using an Azure Function deployed to the cloud, sending 40 likes/sec for eight days, harvesting hidden… | ||
| Reindexing in record time (...) | Conor Landry | 20m |
Shopify moves mapping changes from MySQL into Elasticsearch by spawning coordinator jobs per pod that create worker jobs per shop, running up to 30,000–50,000… | ||
| Dealing with pain points of recommenders in the real world | Sophie Watson | 39m |
Alternating least squares shows practical gaps: it handles explicit or implicit data but not both, is sensitive to anomalies, and ignores temporal drift. The… | ||
| Evolution of Slack Search | Varun Thacker | 34m |
Slack runs search on Apache Solr with a hybrid pipeline: an offline MapReduce/EMR indexer that emits Solr cores to S3 plus seven daily live collections layered… | ||
| Visualizing the Output of Your Search AI | Marcus Eagan | 53m |
A search app combines Apache Solr/Fusion with a TensorFlow convolutional model applied in the index pipeline, using transfer learning from e‑commerce review… | ||
| How to handle 100k events per second | Niels Basjes | 38m |
Bol.com measures at the source (server or owning service) to capture full attributes per event so joins and cache races can't change cause/effect, handling… | ||
| Reaching Zen in ElasticSearch's Coordination | Philipp Krenn | 23m |
Elasticsearch rewrites its coordination (Zen) for 7.0 from a formal TLA+ model and model checking to fix repeated network-partition bugs; master elections… | ||
| Querying ElasticSearch with Deep Learning | Hans-Peter Zorn, Sebastian Blank | 33m |
Sequence-to-sequence LSTM with pointer-attention converts natural-language questions into Elasticsearch queries, trained with reinforcement learning to bypass… | ||
| From User to Contributor | Aizhamal Nurmamat Kyzy, Riona MacNamara | 40m |
Google funds a Q1 2019 user research study (18 interviews) that identifies five contributor archetypes and maps their journeys, showing documentation drives… | ||
| Managing Distributed Workflows at Scale | Abhishek Kumar Singh | 22m |
Shows a real Workflow Orchestration Engine built for Unbxd PIM that runs nodes as Docker images on Kubernetes Jobs and scheduled runs via Kubernetes CronJobs… | ||
| Towards Flink 2.0: Rethinking the stack and APIs (...) | Stephan Ewen | 42m |
Explains why DataStream currently runs batch workloads much less efficiently than DataSet and lays out a concrete redesign: a new Flink stack with explicit… | ||
| Chatting with Solr | Erik Hatcher | 41m |
Demonstrates using the Solr Tagger as a first-stage query interpreter: a JSON /tag endpoint plus a collection built with catenateGraphFilterFactory tags known… | ||
| Integrate your Search Engine (...) | Margaux Wagner, Lucian Precup | 45m |
An enterprise search engine is adapted to Google Assistant by placing a proxy that maps speech-to-text hypotheses plus intent/slot templates into rewritten… | ||
| DeepCS: a Code Search Tool Powered by Deep Learning | Robert Rodger | 27m |
DeepCS builds a joint embedding that maps English queries and Java methods into the same vector space using a CODE-NN architecture, API‑call and name token… | ||
| Multilingual Search System - How to Build and Improve!? | Paresh Paradkar | 20m |
Company handling ~287 billion emails across >40 PB builds a pipeline using Apache Lucene/Elasticsearch, a logistic regression language detector trained on… | ||
| Band-Aids don’t fix bullet holes | William Benton | 36m |
Argues that production ML pain comes from pipeline entanglement and vendor-driven point solutions, not lack of models: deep learning excels at vision/speech… | ||
| Analysing Real time Activity Streams for Security | Rashmi Singh | 16m |
Rashmi Singh talking about "Analysing Real time Activity Streams for Security: Flink ML to the Rescue".
Flink ML can be used to apply Machine Learning models… | ||
| Accelerate big data analytics with Apache Kylin | Shaofeng Shi | 39m |
Apache Kylin builds precomputed OLAP cubes on Hadoop with MapReduce/Spark cube build and persists cuboids to HBase persistence, exposing JDBC/ODBC/REST and… | ||
| Architecture of relevancy search at mobile.de | Richard Knox | 20m |
mobile.de runs ~1.6 million active adverts and collects 300 million interactions per day into Hadoop, aggregates 30-day profiles in Cassandra, and serves a… | ||
| Apache Beam pipelines at 100TB+ scale using Apache Spark | David Moravek | 19m |
Seznam.cz runs Apache Beam on the Spark runner to process 100TB+ daily inputs (pipeline handles 50–100 TB each run, 300M pages/day) and solves severe… | ||
| State of diversity in open source communities | Holden Karau | 39m |
Data from 50 projects (about 30 GB of commits plus 11 GB for geographic data) and roughly $300 of human review uses GitHub extraction, Spark notebooks… | ||
| Apache Zeppelin | Eric Pugh | 42m |
Apache Zeppelin presents a team-first data notebook that connects Spark, Python/pandas, SQL and supports 20+ languages and data stores (Solr, Elasticsearch… | ||
| Precise graph-based phrase query with SpanNearQuery | Michael Gibney | 21m |
A patch that makes SpanNearQuery matching precise and complete while keeping performance comparable to the current implementation, presented as a… | ||
| Scalable machine-learned model serving | Jon Bratseth | 40m |
Shows an architecture that pushes computation to data shards, copying TensorFlow and ONNX models to C++ content nodes so each node evaluates models locally in… | ||
| Streaming your shared ride | Thomas Weise | 37m |
Lyft migrates real-time pipelines from Kinesis cron jobs to Kafka + Apache Flink, adds a Python path via Apache Beam so data scientists can reuse model code… | ||
| Building Analytics Applications (...) | Amrit Sarkar | 29m |
Amrit Sarkar talking about "Building Analytics Applications with Streaming Expressions in Apache Solr".
Effective real-time analysis and visualization of… | ||
| Building an AI/ML powered text search system | Nick Burch | 48m |
Takes 294 Berlin Buzzwords talks, extracts text with Apache Tika, turns it into tf-idf vectors (≈30,000 distinct terms), trains a multinomial Naive Bayes… | ||
| Why data-driven methods will shape the future (...) | Pedro Balage | 46m |
Argues that replacing brittle synonyms, taxonomies and rules with trained models scales better and faster: implements language modeling for autocompletion… | ||
| Containers, Kubernetes and Practical DevSecOps | Thomas Fricke | 40m |
Kubernetes contains the primitives to secure microservices but default deployments leave gaps: audits find misconfigured RBAC, exposed sockets, and databases… | ||
| Scaling ONNX and TensorFlow model evaluation in search | Lester Solbakken | 22m |
Vespa evaluates ONNX and TensorFlow models on content nodes so data stays local, not sent over the network, and a benchmark with a ~200k‑parameter neural… | ||
| Monitoring Solr at Scale | Ken LaPorte | 41m |
Bloomberg’s Search Infrastructure describes a monitoring stack that scales from hundreds to thousands of Solr services by separating collection of state from… | ||
| Miles or Kilometers? Why your data schema should include units | Erik Erlandson | 20m |
Shows a Scala library, Coulomb, that encodes units as types, compiles conversions, and rejects incompatible units. Recounts a Kubernetes/Spark memory bug where… | ||
| Playing fair in the Open Source World | Griselda Cuevas | 32m |
Describes Google’s concrete lessons from donating Apache Beam: mismatched tooling and release cadence caused friction, so Google builds an Apache Google… | ||
| Building applications with Serverless Framework ... | Fredrik Vraalsen | 01h 18m |
Uses Serverless Framework with AWS Lambda, deploying via simple YAML and SLS commands that package to S3 and create CloudFormation stacks; demonstrates REST… | ||
| 7 reasons to use Apache Flink | Jakub Piasecki, Fabian Hueske | 38m |
Shows concrete use: Apache Flink ingests IoT streams from Kafka, keeps state locally (RocksDB), and uses exactly-once checkpoints persisted to S3/HDFS so state… | ||
| Fast log management for your apps | Nicolas Fränkel | 20m |
Shows concrete changes that cut log overhead: replace eager messages with a lazy logger with Supplier to avoid expensive computations, use async appenders with… | ||
| Hops in the Cloud | Jim Dowling | 39m |
HopsFS becomes a multi–data‑center, POSIX‑like filesystem with strong metadata consistency, storing small files in metadata on NVMe disks and pushing a… | ||
| Writing a Distributed Ray Tracer with Apache Beam, Abridged | Robert Burke | 17m |
Maps a Monte Carlo ray tracer onto the Apache Beam Go SDK, using Beam primitives: PCollections, DoFns, side inputs, combines and reshuffles to generate and… | ||
| Taming the language border (...) | Uwe L. Korn | 39m |
Apache Arrow provides a language‑neutral columnar in‑memory format and libraries that let Java produce off‑heap Arrow record batches, hand the memory pointers… | ||
| Doing Data: The Critical Process of Data Preparation (...) | Ellen Friedman | 39m |
A practical playbook for making ML projects work: preserve raw inputs (one lab kept blood samples for 30 years and later found genetic signals), verify labels… | ||
| 50 Shades of Legacy | Gregorio Kusowski | 21m |
Company is a used‑car marketplace (heycar). Claim: many “new” systems become legacy fast — example: a Swift microservice + Kafka pushed to production generates… | ||
| Python, Java, or Go: It's Your Choice ... | Maximilian Michels, Ismaël Mejía | 41m |
Apache Beam exposes a portable pipeline model that lets Python, Java, Go and Scala run on engines like Apache Flink and Google Cloud Dataflow via a job server… | ||
| Open standards for machine learning deployment | Nick Pentreath | 41m |
Overview of practical deployment standards: PMML (1997) is XML, supported by 30+ vendors and handles classic models but limits deep nets and forces extensions… | ||
| Non-Code contributions (...) | Griselda Cuevas | 31m |
Non-code work proves measurable: documentation and ticket triage speed development cycles and reduce duplicate issues, while meetups, release announcements and… | ||
| Java on ARM. Theory, Applications, and Workloads | Dmitry Chuyko | 39m |
Shows that OpenJDK on ARM is production-ready: AArch64 intrinsics are added to hotspot and Java 11+ builds run on servers and Pi-class devices, with minimal VM… | ||
| How we use Reinforcement Learning (...) | Juantomás Garcia Molina | 47m |
Recreates the 1987 Abbey of the Crime by disassembling the original tape, reimplementing the engine in C++ and exposing it through an OpenAI Gym–style API so… | ||
| Measuring Vegetation Health to Predict Natural Hazards | Suneel Marthi | 32m |
With the abundance of Remote Sensing satellite imagery available on public clouds(AWS), the possibilities are endless as to the kind of insights that can be… | ||
| Dynamic Big Data in the Cloud with Kubernetes | Frank Conrad | 24m |
Shows how to run Apache Spark, Flink and HDFS on Kubernetes using Terraform/Helm, operator patterns and node pools, and claims clusters can scale to bis zu… | ||
| nothing matches | ||