Berlin Buzzwords 2018
2018-06-11..2018-06-13
Berlin, Germany
[Berlin Buzzwords 2012, Berlin Buzzwords 2015, Berlin Buzzwords 2016, Berlin Buzzwords 2018, Berlin Buzzwords 2019, Berlin Buzzwords 2020, Berlin Buzzwords 2021, Berlin Buzzwords 2022, Berlin Buzzwords 2023, Berlin Buzzwords 2024, Berlin Buzzwords 2025, Berlin Buzzwords 2026]
| title | author | length |
|---|---|---|
| Keynote: Blockchain In Wildlife – Use Cases in the Field | Marta Piekarska | 35m |
Hyperledger describes concrete enterprise stacks: Hyperledger Fabric (Kafka consensus, chaincode in Go/Java/JS, Solidity support), Sawtooth with PoET… | ||
| Deploying Large Spark Models to production #bbuzz | Subhojit Banerjee | 43m |
Shows how a PySpark pipeline becomes a portable, production scorer by serializing the Spark pipeline to an MLeap bundle (JSON + protobuf), packaging that into… | ||
| Beyond the Algorithm: What Makes Machine Learning Work? | Ellen Friedman | 41m |
Anecdotes and case studies show the difference between a lab model and production value: a Raspberry Pi running a TensorFlow+Inception v3 classifier on… | ||
| Big News for Big Data in Kubernetes #bbuzz | Ted Dunning | 41m |
Kubernetes now supports running a full software-defined storage layer as a cluster-managed service via a new plugin so containers can hold large state without… | ||
| From 5 PB of Daily Logs to Thousands of ML Models #bbuzz | Fabian Höring | 41m |
Criteo's pipeline reads 5 PB of daily logs to train and deploy several thousand machine-learning models per day that predict which ads to show, supporting 3.5… | ||
| Next generation plug & play IT-systems for public transport | Christoffer Vig | 36m |
Ruter runs public transport for 317 million passengers a year and builds a streaming back end around Apache Kafka and Kafka Streams to fuse GPS, APC sensors… | ||
| From boolean towards semantic ... | Arpan Gupta, Seinjuti Chatterjee | 21m |
A production pipeline identifies weighted query tokens as Must-have Tokens via dependency parses, trains local embeddings from catalog plus clickstream on… | ||
| Your Search Service as a Composable Function #bbuzz | Stefanie Schirmer | 40m |
A Scala-based proxy models the search pipeline as a composition of a Reader monad and Future to keep server state minimal and treat requests as pure functions… | ||
| Getting Started with Query Understanding #bbuzz | Giovanni Fernandez-Kincade | 41m |
Argues that focusing on intent instead of raw keywords prevents “poisoned” results and lists concrete fixes: add entity exceptions to stop aggressive stemming… | ||
| Building a search platform 101 #bbuzz | Anshum Gupta | 40m |
Central recommendation: build a managed platform around Solr (or Elasticsearch) that cuts setup time from months to hours and supports multi‑tenant teams. Use… | ||
| Working with Tensorflow from the JVM #bbuzz | Holden Karau | 34m |
Shows how Java jobs start TensorFlow per partition using TF on Spark and pandas UDF wrappers, forcing a Java→pickle→socket→Python→C++ chase that doubles… | ||
| Deep Learning Recommender Architecture: Master Thesis to ... | Igor Mazor | 42m |
A deep learning recommender evolves from a master thesis into a production pipeline that embeds users and items as 77‑float vectors, trains three nets (user… | ||
| Apache Lucene and Java 9+ #bbuzz | Uwe Schindler | 41m |
Explains concrete changes Lucene/Solr/Elasticsearch make for Java 9+: compatibility work with the Java 9 module system access restrictions (fixes for… | ||
| Top 10 data engineering mistakes #bbuzz | Lars Albertsson | 40m |
Pick a proper workflow orchestrator rather than ad hoc scripts—use Luigi or Airflow—so pipelines are reproducible and recoverable (typical rollback time with… | ||
| Search and Recommendations: 3 Sides of the Same Coin #bbuzz | Nick Pentreath | 37m |
Compares three concrete ways to serve recommendations with search: score then search (separate scorer and searcher, flexible models but round‑trip latency)… | ||
| Stateful Stream Processing with Apache Flink 1.5 and beyond | Nico Kruber | 37m |
Apache Flink 1.5 adds a new deployment architecture with a ResourceManager, Dispatcher and per-job JobManagers enabling dynamic resource allocation and native… | ||
| What they didn’t teach you in Grad School #bbuzz | Sophie Watson | 15m |
Implements a well‑cited ALS recommender (4416 citations) on the 26 million ratings MovieLens set using an off‑the‑shelf Apache Spark implementation, and finds… | ||
| Scalable OCR pipelines using Python,Tensorflow & Tesseract | Mark Keinhorster | 20m |
Production OCR pipeline that combines OpenCV pre-processing (binarization, Hough-line rotation, dilation/erosion for text segmentation) with Tesseract OCR at… | ||
| The Neural Search Frontier #bbuzz | Doug Turnbull, Tommaso Teofili | 47m |
Claims that neural search is a silver bullet give way to measured results: experiments on an ACM corpus of 3,000 documents compare Lucene/BM25 and TF‑IDF to… | ||
| Spark and Flink Running Scalable in Kubernetes #bbuzz | Frank Conrad | 37m |
Shows that replacing shared multi-tenant clusters with per-job temporary clusters and an operator-based deployment reduces noisy-neighbor interference but… | ||
| Machine Learning for Apache Solr, Elasticsearch & Vespa #bbuzz | Mitali Jha | 23m |
Compares LTR support across Elasticsearch (third‑party LTR plugin using RankLib/XGBoost), Solr (built‑in LTR contrib module) and Vespa (native tensors +… | ||
| Event stream processing using Kafka Streams | Fredrik Vraalsen | 01h 29m |
Hands-on workshop showing how Kafka Streams becomes the core of a production pipeline that processes 800 million events per day from collectors through Amazon… | ||
| Continuous Monitoring of Machine Learning Models | Patrick Baier, Lorand Dali | 36m |
Zalando Payments monitors deployed fraud models because labels arrive with a two-week label delay, so accuracy alerts are too late. The system captures live… | ||
| Docker to kubernetes: running Hadoop in a cloud native way | Marton Elek | 41m |
Shows concrete recipes for running Hadoop in containers using docker host network for a Hadoop-native setup and Consul + Nomad for a DIY, service-discovery… | ||
| Learning to Rank journey at GetYourGuide: Our Logbook #bbuzz | Felipe Besson | 38m |
GetYourGuide builds an LTR pipeline on Databricks + Spark, collects 30,000 judgments from internal experts (0–3 scale), engineers query‑document, business… | ||
| TensorFlow: Hotdog or Not with Toppings #bbuzz | Jim Dowling | 01h 24m |
Workshop builds a Hotdog-or-Not pipeline that actually trains a CNN from JPEGs, converts them to TFRecords in HDFS for fast GPU feeding, and deploys the… | ||
| Search suggestions as killer feature | Joshua Bacher, Christine Bellstedt | 25m |
Auto retailer reports 1.9 million visits/day and up to 10 orders/sec and finds poor autocomplete can hurt revenue; A/B tests confirm negative impact and later… | ||
| ElasticSearch (R)Evolution — You Know, for Search ... #bbuzz | Philipp Krenn | 41m |
History of ElasticSearch adoption is brief; the rest focuses on concrete changes: shows rolling upgrades with zero downtime by mixing 5.6 and 6.3 nodes and… | ||
| Fast Access To Your Complex Data #bbuzz | Owen O'Malley | 40m |
Benchmarks run on Apache Spark compare Avro JSON ORC and Parquet across real datasets: 22 million taxi rows, 25 million synthetic sales rows, and 10 million… | ||
| Practical Use-Cases of Solr's AutoScaling Framework #bbuzz | Varun Thacker | 40m |
Solr’s AutoScaling framework aims to let clusters grow to a trillion documents with minimal ops effort by using declarative policies (hard rules) and… | ||
| Calculating recommendations based on product images #bbuzz | Vlad Dolezal | 17m |
A one-day prototype proves image colors boost recommendations: extract ten centroids per product by k-means on pixels (background removed by border variance)… | ||
| Profiling and optimizing a Spark job with Babar #bbuzz | Benoit Hanotte | 20m |
Criteo’s open‑source profiler Babar attaches a jar to executor JVMs, samples stacks and OS metrics, and emits an HTML report with six tabs and flame graphs… | ||
| Learned Indexes: a New Idea for Efficient Data Access #bbuzz | Robert Rodger | 37m |
Proposes replacing B‑trees with trained models and shows concrete designs: a neural-network index trained on 200 million logs performs poorly in TensorFlow… | ||
| Ship your Machine Learning Application #bbuzz | Michael Stockerl | 20m |
A practical deployment pipeline is described that uses Apache Kafka topics named by preprocessor git commit and Kubernetes to promote models through build… | ||
| Building a Personalised Home Feed #bbuzz | Christiane Lemke, Domenic Vossen | 20m |
eBay Kleinanzeigen builds an on‑demand personalized home feed that scores and mixes results from a 28 million‑item Elasticsearch index using user features… | ||
| distributed stream processing for everyone with KSQL #bbuzz | Michael Noll | 42m |
KSQL is a streaming SQL engine that turns what would be dozens of lines of Java or Scala into a single SQL statement that runs against Apache Kafka. It runs as… | ||
| DataFrames in Apache Spark - the analysts perspective | Marcin Szymaniuk | 35m |
Practical debugging of Spark DataFrame failures with concrete fixes and numbers: shows partitionBy can produce thousands of tiny HDFS files when 8,000 input… | ||
| CrateDB: A Search Engine or a Database? Both! #bbuzz | Maximilian Michels | 36m |
CrateDB runs a full SQL layer SQL-99 compatible compiled into Lucene queries and routed over Elasticsearch so queries execute across a cluster with a… | ||
| Head-N-Tail Analysis to Increase Engagement #bbuzz | Amrit Sarkar | 22m |
Head‑n‑tail analysis claims to lift engagement by surfacing at least one relevant result for low‑traffic queries and gives concrete numbers: 100 unique queries… | ||
| New Replica Types in Solr 7 #bbuzz | Tomás Fernández Löbbe | 36m |
Solr 7 adds three replica types so clusters can trade immediacy for availability: NRT replicas index and write the tlog and support soft commits and real‑time… | ||
| 800 mio. IP addresses to coordinates using Kafka Streams #bbuzz | Håkon Åmdal | 43m |
A Kafka Streams pipeline enriches roughly 800 million events/day by batching IP-to-coordinate HTTP calls with the Processor API transformer and punctuate… | ||
| Leverage the power and simplicity of SQL on Apache Flink | Fabian Hueske | 43m |
Argues that Apache Flink exposes full SQL and a Table API that deliver unified batch/stream semantics, with production deployments at Alibaba, Huawei, Uber and… | ||
| Relevant Data Analysis: Apache Solr Analytics #bbuzz | Houston Putman | 33m |
Search engine–native analytics inside Apache Solr 7 implements a MapReduce-like pipeline that maps per-document fields, reduces across shards, and merges… | ||
| Blueprints for common stream processing use cases with Flink | Kostas Kloudas | 31m |
Kostas Kloudas talking about "Stream Processing for the Practitioner: Blueprints for common stream processing use cases with Apache Flink"
When working with… | ||
| Schema Learning in Apache Solr #bbuzz | Abhishek Kumar Singh | 21m |
Apache Solr has come a long way from being used for simple full-text search to modern day analytics, geospatial, media and multi-tenant search applications… | ||
| Classification of Satellite Imagery | Suneel Marthi, Jose Luis Contreras | 36m |
Project uses Sentinel-2 10‑m RGB tiles plus OpenStreetMap/GeoPedia labels to classify agricultural vs non‑agricultural land, focusing on tulip fields. Workflow… | ||
| Technical Consequences of the Data Subject's Rights #bbuzz | Aurélie Pols | 55m |
GDPR reintroduces enforceable data subject rights that create concrete engineering work: one‑month reply deadlines, identity verification flows, deletion that… | ||
| Elasticsearch index management for paas style logging | Jaeik Lee, Qin Tang | 41m |
NELO ingests over 3 billion logs per day (1.6 PB stored) and hits a mapping explosion where per-index mappings grow and block cluster state updates; 3,000… | ||
| Kafka Security - A brief overview of its history ... #bbuzz | Sönke Liebau | 41m |
Kafka adds real security milestones: SSL and Kerberos in 0.9 provide authentication and encryption, PLAIN then SCRAM avoid cleartext, and delegation tokens in… | ||
| @Dissidentbot: dissent will be automated! #bbuzz | Steve Loughran | 16m |
A 500‑line Ruby bot running on a Raspberry Pi hooks into the Twitter firehose, listens for tweets, and replies by picking sentences at random; a DM control… | ||
| Behind the scenes of building a doctor in your pocket | Domenico Corapi | 21m |
Babylon describes a production pipeline that links a medical knowledge base (≈8 million tokens) to live patient events via a custom Event Store, Kafka and CDC… | ||
| Large Scale Graph Solutions: Use-cases and Lessons Learnt | Rekha Joshi | 30m |
When 1000+ different products and services, with millions of customers and more than 25% of US depend and use your platforms, an intelligent efficient data… | ||
| Building a data lake at a bank #bbuzz | Dominic Breuker | 26m |
SolarisBank builds a cloud data lake that ingests via Kinesis → Avro → S3, runs batch analytics with Luigi, and centralizes access and audit trails using… | ||
| When DataFrames fail, resort to mapPartitions #bbuzz | Matija Gobec | 34m |
DataFrame APIs hit nested-schema and lineage limits: a generated schema spans ~2,000 lines with nine-deep, 11-wide nesting and exploding 10 GB of source into… | ||
| Connecting the data infrastructure with the DataFlow | Pere Urbón-Bayes | 42m |
Shows concrete integration: deploys Apache NiFi to consume MySQL binlog changes via Maxwell -> Kafka -> NiFi -> Elasticsearch, routes by JSONPath attributes… | ||
| Lifting AntiSpam to the cloud and beyond #bbuzz | Juan De Dios Santos Rivera | 20m |
Throughout the last months, our AntiSpam platform underwent many significant changes. The biggest of them, the migration to the cloud, and the removal of our… | ||
| Building your own Distributed Streaming Recommendation Engine | Rui Vieira | 40m |
A distributed streaming ALS implemented with stochastic gradient descent updates per incoming rating, using Apache RDDs and Apache Spark DStreams and Kafka for… | ||
| Lector in Codigo #bbuzz | Alvaro Videla | 42m |
Argues that code is primarily communication and that clarity requires explicit paratext: up-to-date comments, tests, package and folder names, and… | ||
| Protecting sensitive data in huge datasets #bbuzz | Felipe Hoffa | 40m |
Shows concrete workflows using Google DLP API, BigQuery and Data Prep to scan and transform massive public tables, detect PII in text and images, do… | ||
| Noise: a search library in Rust #bbuzz | Volker Mische | 19m |
Noise is an open-source search library implemented in Rust that stores JSON documents and auto-indexes them on insert: string fields are indexed for full-text… | ||
| Intelligent applications in containers | William Benton, Michael McCune | 01h 23m |
Shows a concrete workflow: prototypes run interactively as Jupyter + Spark notebooks (k‑means, logistic regression, Word2Vec) then get lifted into a… | ||
| Scalably crashing JVMs, or why binary data to content is hard | Nick Burch | 36m |
Finding what a blob of bytes actually is breaks at scale: retries and speculative re-execution can spin a single bad PDF into a crashed cluster, container… | ||
| Meeting complex data load and data preparation challenges | Johannes Peter | 21m |
Shows a concrete pipeline built with Apache NiFi that extracts from mixed sources, runs XML-to-JSON transforms, enriches records via lookups, and pushes… | ||
| nothing matches | ||