DataXDay 2019
who it's for
data engineers, machine learning engineers, platform engineers, site reliability engineers, data scientists, mlops engineers, database engineers
topics
Kafka-based streaming pipelines; BigQuery ELT rollups; Flink stream hydration; Kafka Connect and KSQL deployment; OpenTelemetry and Jaeger tracing; PyTorch GAN training with DALI; neural architecture search on Kubernetes; RabbitMQ for distributed NAS; Apache Cassandra cloud architectures; Spark Structured Streaming internals; Kubeflow TensorFlow on Kubernetes; production ML deployment from Jupyter to Spark
| title | author | length |
|---|---|---|
| Give meaning to 100 billion analytics events a day, analytics at Teads | Par Alban Perillat-Merceroz | 24m |
Claims a pipeline that handles 100 billion events per day (≈1M events/s) by writing immediately into Kafka → Dataflow → BigQuery ELT, splitting tracking into a… | ||
| How to leverage the Apache Kafka Ecosystem to productionize Machine Learning | Par Kai Waehner | 40m |
Shows how to run trained models at scale by streaming sensor data from cars through an MQTT proxy into Kafka, replicating to an analytics cluster, and using… | ||
| The Incredible Unification of Logs | Par Jonathan Winandy | 45m |
The talk explains how to build a unified logging and debugging platform for microservices using Kafka and Jaeger. It covers unifying and compressing… | ||
| Harnessing the power of Generative Adversarial Networks (GANS) for supervised learning | Par Olga Petrova | 37m |
Builds a face-frontalization pipeline: an encoder-decoder generator with a 512-dimensional bottleneck and ~5M trainable parameters, trained on about 700,000… | ||
| Deep Learning in Production Seen by a Data Engineer | Par Romain Sagean | 16m |
This talk explains how to industrialize a chosen deep learning model for production: parallelizing prediction workloads, options for deploying without Python… | ||
| Keynote - Database Infrastructure at Instagram Scale | Par Michaël Figuière | 21m |
Instagram runs Apache Cassandra on several thousand servers and peaks at tens of millions QPS, backed by a custom storage engine. A central storage API gateway… | ||
| Keynote - Trust Between Humans and Data | Par Caroline Goulard | 22m |
Caroline Goulard, CEO of Dataveyes, discusses trust between humans and data, focusing on how organizations can build reliable data practices and… | ||
| How to scale Neural Network Architecture Search with RabbitMQ and Kubernetes | Par Guillaume Michel | 13m |
Dvolver runs a population‑based genetic NAS optimizing speed and accuracy on ImageNet, generating independent training jobs so evaluation scales horizontally… | ||
| Kafka Streams, profiling, framegraphs, Oh My! | Par Xavier Leaute | 21m |
Shows how async-profiler plus flame graphs and Java Flight Recorder expose where a Kafka Streams app actually spends time — e.g., 15–20% in GC, ~10–20% in… | ||
| Apache Cassandra and Cloud Architectures | Par David Leconte | 18m |
Apache Cassandra, a masterless, peer-to-peer distributed database designed for resilience, scalability and replication, is presented as a fit for hybrid- and… | ||
| Natural Language Processing for Email Routing | Par Tom Stringer, Par Antoine Isnardy | 37m |
A production system used at MAIF that preprocesses and routes ~10,000 customer emails per day using natural language processing. It combines statistical… | ||
| How to Implement a Data Architecture | Par Nicolas Brigitte-Alphonsine | 17m |
Practical guidance for implementing a data architecture that makes data easier to access by building industrialized data models closely aligned with existing… | ||
| GCP Serverless Updates | Par Guillaume Laforge | 38m |
Overview of Google Cloud's serverless offerings beyond App Engine and Cloud Functions, explaining how Google App Engine and Cloud infrastructure enable apps… | ||
| Confident Data Migration: automatic regression testing | Par Thomas Franquelin | 19m |
ContentSquare migrates Redshift → Elasticsearch → ClickHouse while traffic grows tenfold and avoids overnight outages by replay production load to legacy and… | ||
| From Jupyter to Spark: Stages of Machine Learning Productionization | Par Pauline Nicolas, Par Théo Bontempelli | 17m |
This presentation describes how Deezer moved machine-learning work from Jupyter notebooks into production on Apache Spark to provide real-time model… | ||
| Kubeflow: Tensorflow on Kubernetes | Par Laurent Grangeau, Par Sylvain Lequeux | 36m |
Kubeflow on Kubernetes to run TensorFlow: explains deploying Kubeflow to make TensorFlow model training portable, repeatable and scalable on Kubernetes. Covers… | ||
| Incremental Data Architecture | Par Walid Haouari | 15m |
Incremental Data Architecture presents Incremental Software Architecture as a method to design data systems that avoid mismatches with requirements, poor… | ||
| Multilingual sentiment analysis & emotion detection based on social media data | Par Alexis Dutot, Par Jade Copet | 37m |
Linkfluence converts emoji signals into a language-agnostic pipeline: a large English-trained emoji predictor outputs emoji-to-sentence-probabilities, then a… | ||
| What Testing Strategy for Your Machine Learning Applications? | Par Victor Landeau | 38m |
Oui.sncf describes their testing strategy for production machine-learning applications, addressing non-deterministic outputs when expected results aren’t… | ||
| The internals of stateful stream processing in Spark Structured Streaming | Par Jacek Laskowski | 48m |
Covers state management in Spark Structured Streaming 2.4.3 with concrete APIs and demos: use of groupBy/rollup/cube, flatMapGroupsWithState for arbitrary… | ||
| nothing matches | ||