Berlin Buzzwords 2015
2015-05-31..2015-06-03
Berlin, Germany
[Berlin Buzzwords 2012, Berlin Buzzwords 2015, Berlin Buzzwords 2016, Berlin Buzzwords 2018, Berlin Buzzwords 2019, Berlin Buzzwords 2020, Berlin Buzzwords 2021, Berlin Buzzwords 2022, Berlin Buzzwords 2023, Berlin Buzzwords 2024, Berlin Buzzwords 2025, Berlin Buzzwords 2026]
| title | author | length |
|---|---|---|
| Algorithms & data-structures that power Lucene & ElasticSearch | Adrien Grand | 37m |
Lucene stores a sorted term dictionary plus posting lists on disk and answers queries by iterating with next and advance, using a leapfrog algorithm (lead list… | ||
| Practical t-digest Applications #bbuzz | Ted Dunning | 15m |
Algorithm computes approximate quantiles with constant relative accuracy (very tight at tails, looser near median) by merging sorted points into centroids with… | ||
| Talk the Talk: How to Communicate with the Non-Coder #bbuzz | Ellen Friedman | 17m |
Clear claim: technical teams must translate core ideas into concrete, testable messages for non-coders to get resources and domain input. Practical techniques… | ||
| Computing recommendations at extreme scale with Apache Flink | Till Rohrmann | 33m |
Implementation uses Apache Flink to run three ALS variants and a blocked ALS partitioning that cuts redundant network IO and removes one shuffle by… | ||
| Predictive Insights for IT Operations #bbuzz | Omer Trajman | 13m |
Describes a system that applies machine learning to datacenter event streams to surface correlated anomaly detection across thousands of time series, using… | ||
| What and Why and How: Apache Drill 1.0 #bbuzz | Ted Dunning | 36m |
Apache Drill 1.0 claims to run SQL directly over JSON and files without ETL by using late-binding SQL with any type that observes an empirical schema per row… | ||
| The Dos & Don'ts of ElasticSearch Scalability & Performance | Patrick Peschlow | 40m |
ElasticSearch mappings must be chosen up front: dynamic mapping explodes into a 40 MB mapping at one site, and changing analyzers forces full reindex… | ||
| Application performance management w/ open... | Tudor Golubenco, Monica Sarbu | 26m |
A methodology that turns per-transaction JSON (response time, error codes, URL path, DB query, rows, size) into event analytics by shipping Packetbeat ->… | ||
| Real-time analytics with Apache Cassandra & Apache Spark | Christopher Batey | 38m |
Demo shows Apache Cassandra for raw time-series storage, Kafka for ingestion and Apache Spark (including the Spark Cassandra connector) for both batch and… | ||
| Detecting Events on the Web with Java, Kafka and ZooKeeper | James Stanier | 32m |
Built a JVM cluster that ingests matched mentions into Apache Kafka and runs per-query in-memory spike detection for ~130,000 active queries (about 18M… | ||
| Real Time Big Data Analytics with Kafka, Storm, HBase #bbuzz | Ameya Kanitkar | 36m |
Groupon builds a realtime pipeline using Kafka-Storm-HBase-Redis that ingests and processes about 3 million updates/sec, claiming millions in extra revenue… | ||
| Beyond Significant Terms #bbuzz | Andre Lynum | 21m |
An open‑source intelligence pipeline using RabbitMQ, Elasticsearch and a graph DB ingests noisy multilingual public data and applies significant terms… | ||
| What's with the 1s and 0s? Making sense of binary data at scale | Nick Burch | 40m |
Uses Apache Tika to auto-detect file types, extract metadata and plaintext, and chain Tesseract OCR for images; the distribution ships ~70 format parsers and… | ||
| Apache Flink deep-dive #bbuzz | Stephan Ewen | 42m |
Apache Flink is a streaming dataflow engine that runs batch jobs as finite streams, implements barrier checkpoints (a Chandy‑Lamport variant) to get… | ||
| Hive on Spark #bbuzz | Szehon Ho | 34m |
Ports Hive to run on Spark as an alternative execution engine, keeping existing Hive operators and SQL dialect while replacing MapReduce jobs with Spark… | ||
| Fast Decompression Lucene Codec #bbuzz | Ivan Mamontov | 37m |
Shows that replacing Java decoding with a SIMD C decoder (C99 vector code from Daniel Lemire) called via HotSpot’s critical native interface halves block… | ||
| Apache Lucene 5 - New Features & Improvements for Apache Solr | Uwe Schindler | 41m |
Lucene 5 focuses on data safety and operational stability: it moves to Java 7/NIO.2 (recommended Update 55), adds checksums during merging and per-segment… | ||
| Data Challenges with 3D Computer Vision #bbuzz | Eugen Funk, Martin Scholl | 17m |
Automated 3D capture from Kinect/Project Tango, stereo rigs and LIDAR produces huge raw volumes: a 9 MP 8‑bit color + 16‑bit depth frame is ~43 MB, 10 fps →… | ||
| Online learning, Vowpal Wabbit and Hadoop #bbuzz | Heloise Nonne | 34m |
Claim: Vowpal Wabbit runs distributed online learning on Hadoop and speeds training; implemented by combining Vowpal Wabbit mappers with an all-reduce spanning… | ||
| Compression in Lucene #bbuzz | Ryan Ernst | 27m |
Explains concrete encodings added and tuned for Lucene 5: a prefix FST maps terms to block offsets, postings switch from VInt vs packed ints (delta +… | ||
| Analytics in the age of the Internet of Things #bbuzz | Ludwine Probst | 33m |
A pipeline collects CSV accelerometer data sampled at 50 ms from 47 users into Cassandra, windows data into 5–6 s segments, extracts features (mean, variance… | ||
| Solr Sparse Faceting #bbuzz | Toke Eskildsen | 29m |
Netarchive.dk indexes 500 TB into Solr and faces faceting on fields with hundreds of millions to billions of unique strings. The code changes claim about 2x… | ||
| Using Random Projections to Make Sense of High-Dimensional ... | Stefan Savev | 26m |
Random projections (aka locality sensitive hashing) become a fast nearest-neighbor index for dense vectors by building many random-partition trees and… | ||
| You've got questions. We've got answers! #bbuzz | Grant Ingersoll | 35m |
A working question‑answering pipeline is described that indexes a ~10,000‑record Wikipedia snapshot and uses Apache Solr and Lucene plus OpenNLP to detect… | ||
| From Machine Learning Startup to Big Data Company | Christoph Tavan | 36m |
Startup builds an RTB engine that grows from ~500 requests/sec and ~10 GB/day to 10x per stage while keeping 100 ms round trips, and learns the hard way… | ||
| Change Data Capture: The Magic Wand We Forgot #bbuzz | Martin Kleppmann | 35m |
Change Data Capture extracts every committed row-level change from PostgreSQL 9.4 via logical decoding and streams it into Kafka using a small in-server plugin… | ||
| Low latency scalable web crawling on Apache Storm #bbuzz | Julien Nioche | 20m |
Storm-Crawler is an Apache Storm SDK that implements a multi-threaded fetcher bolt with per-host politeness queues, a parser pipeline (Tika for non‑HTML, HTML… | ||
| Cassandra at Yammer #bbuzz | Michal Rutkowski | 32m |
Yammer migrates its inbox index from a Berkeley DB backend to Cassandra, using shadow deploys, double dispatch and integration tests to validate semantics. The… | ||
| Approaching Join Index for Lucene #bbuzz | Mikhail Khludnev | 32m |
Compares three concrete Lucene approaches and shows a working prototype: a join index prototype built with updatable doc values that stores Lucene docIDs to… | ||
| Real-Time Monitoring of Distributed Systems | Tobias Kuhn, Nakul Selvaraj | 18m |
Trademob runs up to 125k requests/sec across 700+ instances and was drowning in 2k–5k metrics/sec, so engineers build Anam, a Python pipeline that multiplexes… | ||
| It's All Fun And Games Until: A Tale of Repetitive Stress Injury | Eric Evans | 39m |
Startup-era long uninterrupted hours produce numbness, tingling and neck and shoulder pain that progressive diagnostics pin down with a nerve conduction study… | ||
| Signatures, patterns and trends: Timeseries data mining @ Etsy | Andrew Clegg | 35m |
Etsy processes about one million metrics in real time and builds Kale v1 as two components, Skyline (anomaly detection) and Oculus (similarity search), but the… | ||
| How to automate the deployment of ... | Isabelle Robin, Matthieu Vautrot | 29m |
Automates deploying R models by directly shipping RDS-serialized models to production and calling predict via an embedded R session using the rJava / JRI… | ||
| Side by Side with Elasticsearch & Solr part 2 | Radu Gheorghe, Rafał Kuć | 32m |
Benchmarks use JMeter hammering three m3.large-like VMs with Spanish Wikipedia, a 10M-video dataset and Apache logs; product search yields ~500 qps on… | ||
| A complete Tweet index on Apache Lucene #bbuzz | Michael Busch | 28m |
Claims a complete searchable index of every tweet ever (hundreds of billions, ~500M new tweets/day, ~2B queries/day) built on Apache Lucene variants with an… | ||
| Going deep with Spark Streaming #bbuzz | Andrew Psaltis | 34m |
Spark Streaming exposes micro-batches of RDDs where input happens via receivers (must be serializable) that call store and replicate blocks so sources can be… | ||
| In-memory data pipeline and warehouse at scale | Ema Iancuta, Radu Chilom | 36m |
Live demo builds an in-memory pipeline that ingests millions of financial transactions from S3 → Tachyon (Tachyon backed by HDFS), transforms and balances… | ||
| An introduction to Apache Kylin Business Intelligence ... | Fabian Wilckens | 14m |
Apache Kylin is an eBay‑origin OLAP engine that builds key-value cubes with Hive pre-joins and MapReduce jobs, stores them in HBase/HDFS, and exposes a SQL… | ||
| "What's in the news?" #bbuzz | Andrej Rosenheinrich | 21m |
Bluekiwi.de implements a news pipeline that fetches thousands of articles daily, indexes them in Elasticsearch and uses significant terms aggregation together… | ||
| Understanding Databases for Distributed Docker Applications | Chris Ward | 36m |
An open source distributed DB built on Lucene and Elasticsearch with a custom SQL planner uses Docker plus Weave to spin clusters across hosts; a live demo… | ||
| Scale with NSQ: a realtime distributed messaging platform | Georgi Knox | 16m |
An open-source, Go-built messaging platform created at Bitly runs in production for over three years and handles the load behind 8 to 10 billion clicks per… | ||
| Diving into ElasticSearch Discovery #bbuzz | Shikhar Bhushan | 20m |
ElasticSearch discovery actually spans node discovery, leader election, state publishing and failure detection; the uploader-built ESCA (an Akka-cluster… | ||
| Designing Concurrent Distributed Sequence Numbers ... | Boaz Leskes | 22m |
Explains why sequence numbers for ordering let ElasticSearch replace heavyweight consensus with a practical master‑backup design, evaluates Raft/ZAB and finds… | ||
| nothing matches | ||