Flink Forward Berlin 2016
| title | author | length |
|---|---|---|
| Joining Infinity: Windowless Stream Processing with Flink | Sanjar Akhmedov | 36m |
ResearchGate converts change data capture Kafka topics into a continuous join using Flink (coFlatMap, keyed operator state, checkpoints and savepoints) to map… | ||
| Robust Stream Processing with Apache Flink | Jamie Grier | 40m |
Live demo builds a Flink job that generates sawtooth/sine/square sensor streams, writes to InfluxDB and Grafana, windows by key and sums values, and shows… | ||
| The Future of Apache Flink | Aljoscha Krettek | 37m |
Current Flink road map lists working features and rough designs: production-ready Dynamic Scaling lets jobs change parallelism at runtime, a Window Trigger DSL… | ||
| Beaming Flink to the Cloud @ Netflix | Monal Daxini | 34m |
Netflix runs a Stream Processing as a Service that uses Kafka, Flink and Apache Beam to move and transform live events at massive scale, claiming 700 billion… | ||
| Blink: Improvements to Flink & its application in Alibaba Search | Xiaowei Jiang | 30m |
Blink is a Flink-compatible runtime that Alibaba deploys on a Hadoop cluster of more than 1,000 servers to serve mission-critical search and recommendation… | ||
| Running Apache Flink everywhere | Stephan Ewen | 41m |
Claims a reworked process model called Flink Improvement Proposal 6 that splits Flink into resource manager, dispatcher, job manager, task manager so… | ||
| A brief history of time with Apache Flink | Mohamed ABDESSEMED, T. Lamirault | 29m |
Bouygues Telecom deploys a Flink streaming pipeline reading from Kafka, enriching and grouping events, and writing KPIs to HBase with historical Parquet… | ||
| AMIDST Toolbox: Scalable probabil. machine learning w/ Flink | Ana M. Martinez | 28m |
AMIDST toolbox is a Java open‑source library integrated with Flink and AWS that runs distributed Bayesian parameter learning using variational message passing… | ||
| Beyond the Watermark: On-Demand Backfilling in Flink | Maxim Fateev | 35m |
Uber runs Flink to compute driver incentives but watermarks are global so retroactive incentives break streaming. The solution creates an on-demand source… | ||
| Evaluating Streaming Framework Performance for a Large-Scale... | Ron Crocker | 39m |
Three implementations run against the same Kafka stream: a baseline Java consumer, Apache Flink, and Apache Spark Streaming. Spark produces incorrect… | ||
| Large Scale Social Network Data Collection & Analysis... | Steve Blackmon | 37m |
Pipeline description: People Pattern normalizes multi-network JSON into ActivityStreams using Apache Streams (incubating), runs distributed ingestion and… | ||
| Flinkspector: Taming the squirrel | Alexander Kolb | 36m |
Flinkspector is a small testing framework for Apache Flink that builds reproducible event traces, synthesizes watermarks and timestamps, and runs jobs in a… | ||
| Closing Session | Kostas Tzoumas | 08m |
Conference recap lists 350 participants and 42 technical sessions, plus social stats like 460 tweets and 177 sausages grilled. Sponsors named include Alibaba… | ||
| Keynote | Kostas Tzoumas, Stephan Ewen | 50m |
Flink claims it eliminates trade-offs between volume, latency and accuracy and now runs at production scale: more than 200 contributors, an Alibaba deployment… | ||
| Keynote: How Can We Take Flink Forward? | Ted Dunning | 40m |
Flink is already a unified streaming-and-batch engine with technical advantages, but the pivotal need is production adoption not more features. A prototype for… | ||
| Panel: Large Scale Streaming in Production | Maxim Fateev, Xiaowei Jiang, Monal Daxini, Ted Dunning, Jamie Grier | 50m |
Companies report running Flink at production scale: millions of messages per second ingested, Netflix processes 1.4 petabyte/day, and teams at Alibaba and Uber… | ||
| Streaming SQL | Julian Hyde | 41m |
Apache Calcite extends SQL with a stream-table duality so the same object can be queried as a growing table or as a continuous stream; watermarks and an omit… | ||
| Streaming ML with Flink | Marton Balassi | 40m |
FlinkML gets a Scala extension that provides streaming predictors so a batch-trained model (P and Q factor matrices from ALS) is serialized (prototype write… | ||
| Apache Zeppelin: A friendlier way to Flink | Trevor Grant | 44m |
Shows a Zeppelin notebook workflow running Flink examples end to end: a quick word count from local mini-cluster, loading jars to call FlinkML and Mahout for… | ||
| Faster and Furiouser: Flink Drift | Ted Dunning | 38m |
Engineers push Apache Flink on a MapR streaming transport against the Yahoo streaming benchmark (impressions → enrich with campaign ID → keyed windowed… | ||
| Building a real time... | Matthias Kricke, Martin Grimmer, Michael Schmeißer | 36m |
Proof-of-concept OSTMap collects live geotagged tweets, ingests them with Apache Flink, stores compact records in Apache Accumulo and supports full-text term… | ||
| Advanced visualization of Flink and Spark jobs | Zoltán Zvara | 34m |
A real time visualizer for Apache Flink and Spark exposes the physical plan by instrumenting metrics, extending REST APIs, and sampling key distributions with… | ||
| Flink in genomics... | Francesco Versaci | 42m |
http://flink-forward.org/kb_sessions/flink-in-genomics-efficient-and-scalable-processing-of-raw-illumina-bcl-data/
A single run in genome sequencing can… | ||
| No shard left behind: Dynamic work rebalancing in Apache Beam | Malo Denielou | 43m |
Apache Beam implements dynamic work rebalancing in Google Cloud Dataflow by splitting running tasks and reassigning chunks to idle workers, reducing idle time… | ||
| Flink and Beam: Current State & Roadmap | Maximilian Michels | 33m |
Flink’s Beam Runner implements most Beam API features for both batch and streaming, claiming parity with Google Cloud Dataflow on the Beam capability matrix… | ||
| Apache Beam: A Unified Model for... | Kenneth Knowles | 42m |
A programming model that unifies batch and streaming, expressing pipelines (example: WordCount and per-user hourly sums written to BigQuery) and running them… | ||
| Flink in Zalando's World of Microservices | Javier Lopez, Mihail Vieru | 40m |
Zalando builds Saiki, a Kafka-backed data integration layer on AWS that funnels thousands of event types into Flink jobs running on EC2/Docker, materializing… | ||
| RBEA- Scalable Real-Time Analytics at King | Gyula Fora | 38m |
RBEA (Rule-Based Event Aggregator) is a Groovy DSL platform that compiles user scripts into one heavy Apache Flink job per game, injects scripts into the… | ||
| Dynamic Scaling - How Apache Flink adapts to changing workloads | Till Rohrmann | 43m |
Apache Flink implements dynamic scaling by splitting keyed state into key groups and non-key state into small units so the system can reassign state when… | ||
| The Stream Processor as a Database | Jamie Grier | 43m |
Argues that a stream processor can act as the database by answering live key lookups from its local state instead of writing every event to a KV store; uses… | ||
| Declarative stream processing... | Fabian Hueske, Till Rohrmann | 39m |
Apache Flink claims to run both low-latency pattern detection and high-throughput analytics on the same engine by adding a CEP library and a StreamSQL / Table… | ||
| Taking a look under the hood of Apache Flink’s relational APIs | Fabian Hueske | 47m |
Flink exposes a Table API and SQL that both translate into the same Calcite logical plan, run through a Volcano-style cost-based optimizer and then into… | ||
| Introducing Flink on Mesos | Eron Wright | 41m |
Flink gains native Mesos deployment in the Flink 1.2 line with a working demo that launches a Mesos session producing a jobmanager and taskmanager pair and… | ||
| Flink Security Enhancements | Eron Wright | 30m |
Recent additions add native Kerberos keytab support so a Flink cluster can authenticate to HDFS, ZooKeeper and Kafka (no more short-lived delegation tokens)… | ||
| Code Generation in Serializers and Comparators of Apache Flink | Gabor Horvath | 35m |
Runtime code generation produces specialized POJO serializers and comparators that remove reflection, avoid boxing, and unroll field loops, delivering up to 6x… | ||
| Multi-tenant Flink-as-a-Service on YARN | Jim Dowling | 45m |
Flink runs on YARN inside HopsWorks as a UI-driven, open-source service deployed since June 2016 at www.hops.site, offered on a 54‑machine cluster (~10 TB RAM… | ||
| To Petascale and Beyond: Apache Flink in the Clouds | Greg Hogan | 43m |
Shows how Apache Flink plus Gelly generates R-MAT and worst-case graphs at petabyte scale (trillions to quadrillions of triplets) and implements Gelly graph… | ||
| Connecting Apache Flink to the World... | Aljoscha Krettek | 37m |
Survey of Flink connectors, with concrete internals: Kafka and Kinesis offer exactly-once for sources by integrating consumer offsets with Flink checkpoints… | ||
| Interview with Maxim Fateev | Maxim Fateev | 05m |
Describes building a new messaging system similar to Amazon SQS (to be open sourced) and introducing Apache Flink to get exactly-once semantics for payments… | ||
| Interview with Ted Dunning | Ted Dunning | 06m |
MapR helps bring Stratosphere into Apache and the project becomes Flink as it shifts from research into enterprise use. Key engineering changes include… | ||
| Interview with Gyula Fóra | Gyula Fora | 04m |
At King a streaming platform team builds applications, data infrastructure and tooling that make real-time analytics available to 22 game teams and developers… | ||
| Interview with Márton Balassi | Marton Balassi | 03m |
Contributes to Apache Flink for about three years, helped develop the first streaming API at the Hungarian Academy of Sciences and became a Flink PMC… | ||
| Interview with Jamie Grier | Jamie Grier | 08m |
Director at data Artisans describes building production Flink use at Twitter and then joining the company to push Flink forward, helping teams like Netflix and… | ||
| A taste of Flink Forward | 03m | |
Two days of conference and one day of training bring more than 350 attendees and over 40 international speakers onto three stages for concentrated Flink… | ||
| nothing matches | ||