Big Data & RPA Conference Europe 2024
2024-11-19..2024-11-22
-- online --
as the organisers put it
Explore the 2024 conference archive, including event information, speakers and technical sessions across big data, AI, cloud and data engineering.
who it's for
data engineers, machine learning engineers, data platform engineers, automation engineers, data scientists, engineering managers, data governance leads
topics
data vault data modelling; real-time sentiment pipelines; Milvus vector search for RAG; SQL in frontend applications; cost optimisation on AWS Athena; data governance for AI; Spring AI integration in Java; open source entity resolution; Debezium and Apache Flink data contracts; real-time ML inference with GRIS; LLMs for self-serve analytics; integrating AI with RPA and web automation
| title | author | length |
|---|---|---|
| Data Modelling: Navigating the Data Seas with a Data Vault Compass | Luca Pescatore | 42m |
Claim: Data Vault yields scalability, auditability and flexibility for messy lakes. Example: a boutique shows a false 10x revenue spike caused by per‑meter… | ||
| How Decentralized Data Products are Changing GEMA from Inside Out | Martin Zuern, Markus Zachai | 45m |
GEMA adopts a lakehouse and data mesh approach, builds a self‑service platform on Databricks on GCP with Terraform, and claims rapid adoption — over 250 data… | ||
| Sentiment Analysis in Action: Building Your Real-time Pipeline | Olena Kutsenko | 43m |
Real-time pipeline ingests social media records into Apache Kafka, processes them with Apache Flink calling AI models, then indexes results into OpenSearch for… | ||
| Leveraging Azure PaaS for Real-time Social Media Analysis | Raghav Matta | 43m |
Claims an end to end real time social media sentiment pipeline built on Azure PaaS using Azure Databricks, Event Hub, Stream Analytics, Azure Language Service… | ||
| Supercharge Spark: Unleashing Big Data Potential with Milvus for RAG systems | Stephen Batifol | 43m |
Concrete claims and runnable tools: Milvus integrates with Spark via a spark-milvus connector that supports iterative and bulk inserts (Parquet/JSON →… | ||
| Building Knowledge Graphs | Sumit Pal | 46m |
Claims that knowledge graphs power data catalogs/fabric and MDM, cut 30% of time lost to finding data (McKinsey), and already link ~31 billion triples in… | ||
| SQL from the Frontend: Viable or Not? | David Gomes | 45m |
Claim: exposing SQL to the client massively speeds early development but is not production-ready alone. Demonstrated tools and patterns: Neon Author, Postgres… | ||
| Fail-Safe BigQuery: Disaster Recovery Automation Techniques | Karim Wadie | 34m |
An open‑source framework called BQ Backup Manager automates table backups and policies at scale and is already generalized from large customer deployments… | ||
| When the CFO Comes Knocking: Our Journey to a Cost Efficient Data Platform | Josef Goldstein | 46m |
Claims a >50% cloud cost cut in under two years and shows how: add tags across AWS resources, pipeline billing into an Iceberg table, and attribute cost to… | ||
| Optimising Apache Spark and SQL for Improved Performance | Marcin Szymaniuk | 41m |
Shows concrete failures and fixes: Spark 3 still leaves cases where a single partition swallows most data — 100 GB input becomes a 67 GB partition across 1,000… | ||
| Artificial Intelligence is Nothing without Data Governance | Philippe Nieuwbourg | 43m |
Claims a concrete gap and road map: only 5% of companies have AI governance while ~85% plan to start by summer 2025, cites the EU AI Act (May 2024) and NIST… | ||
| Exploring Large Graphs at the Heart of the French National Audiovisual Ins. | Pietro Mele, Radu Pop | 45m |
French INA metadata model uses a cultural‑heritage graph and powers ~60 million documents stored as relational rows, MongoDB fragments and Elastic indexes… | ||
| How (Not) to Spend $15000 per Month on AWS Athena | Ivica Kolenkas | 31m |
A data pipeline built on AWS Athena and S3 runs queries that scan entire buckets because queries lack a WHERE clause and partitions are unused, producing bills… | ||
| Insights into Your Cloud Database: How Storage Engines Actually Work | Jan Mensch | 40m |
Explains how Hummock (RisingWave’s engine) actually writes and reads: MemTables flushed to SSTs, L0 sublevels and flush every second epochs create rolling… | ||
| Leveraging Open Source Databases for Strategic Advantage | Peter Farkas | 45m |
Concrete takeaways and outcomes not buzzwords: practical migration notes, two repeated case studies, exact numbers and where things broke. An airline moves a… | ||
| Applying Data Quality Monitoring | Martin Pohjakivi | 35m |
Claims that data observability reduces time to discovery and resolution are backed by concrete steps: map metadata and assign ownership, run profiling to… | ||
| Strategic Importance of GenAI Platforms for Business Competitiveness | Anindita Mahapatra | 50m |
Argument: every company becomes a data and AI company and winning requires platform-level work not point tools. The talk shows a concrete stack — Databricks… | ||
| How Well do LLMs Detect Anomalies in Your Data? | Chloe Caron | 41m |
Baseline tests use OpenAI, BigQuery and Mistral and show concrete numbers: GPT‑4 detects about 32% → 68% of intended anomalies after prompt engineering… | ||
| Spring AI: Integrating Generative AI in Java Enterprise | Christian Tzolov | 48m |
Spring AI claims to make generative AI an enterprise integration problem and shows concrete results: a working flight‑booking demo that wires a portable client… | ||
| Unleashing the Potential of Cloud-Native Vector Databases | Jiang Chen | 51m |
Claims Milvus is an open source, cloud‑native vector database that actually scales—tens of billions of vectors—and runs distributed on Kubernetes with… | ||
| Open Source Entity Resolution - Needs and Challenges | Sonal Goyal | 41m |
Zingg claims an open source entity resolution framework that runs natively where data lives and learns matching rules instead of hardcoded heuristics. It uses… | ||
| Apache Pulsar: Finally an Alternative to Kafka? | Julien Jakubowski | 45m |
Claim: Apache Pulsar unifies messaging and streaming on one broker and separates compute from storage via Apache BookKeeper. It uses partitionless storage and… | ||
| GDPR Design for Websites, Apps and (Ro)Bots | Kateryna Dubas | 48m |
Explains concrete compliance steps: which technical and organisational measures reduce risk, why regulators issue fines for request-to-delete failure and… | ||
| AI Development Lifecycle: Learnings of What Changed with LLMs | Noe Achache | 43m |
Claim: proof‑of‑concepts with LLMs are trivial but evaluation becomes the bottleneck; neglecting evaluation blocks production. Built a RAG pipeline for hotel… | ||
| Roadmap to Responsible AI | Amardeep Lidder | 37m |
A practical roadmap that claims an integrated data and AI governance framework with measurable KPIs and a maturity assessment and a start-small-scale-fast… | ||
| Build a Modern Data Platform on AWS | Artsiom Yudovin | 37m |
Builds a Lakehouse on AWS using Delta Lake on S3, streaming via MSK + Kafka Connect into Apache Spark on EMR, with compaction twice daily and vacuum every… | ||
| Lean Process Automation | Dragan Metikos | 46m |
Argues that combining Lean and Six Sigma before automation produces sustainable bots rather than fragile scripts; uses Lean Process Automation plus tooling… | ||
| Technology is Necessary, But Not Sufficient | Simon Copsey | 44m |
Asserts that technology alone never delivers bottom‑line gains unless it removes the real bottleneck and changes managerial assumptions; demonstrates with… | ||
| Real-Time AI with Open Source | Timothy J Spann | 47m |
Builds a working RAG pipeline: pulls OpenAQ air‑quality REST data, ingests ~1,000 records at a time into a Milvus vector store, creates 384‑dim embeddings, and… | ||
| Data Engineers: Respect the Resiliency of SQL | Rui Machado | 41m |
Claim: SQL still powers real systems and now runs ML and streaming pipelines end to end. Examples: BigQuery lets users create model with hyperparameters train… | ||
| Generative AI: Get a Grip … and a Look Under the Hood | Andreas Buhlmeier | 26m |
Generative models rely on injected noise to create new outputs and the breakthrough is Transformer (2017) "Attention is All You Need", not vague hype. Training… | ||
| Autonomous Enterprise | Andrzej Lassak | 50m |
Autonomous enterprise is defined as a self‑optimizing business that lets AI and automation make independent decisions; concrete signals include an expected… | ||
| Using Transformers.js to Bridge the Gap Between AI Research and | Artem Pochechuev, William de Pretre | 33m |
Claims a trained Transformer for dyslexia-aware word prediction achieves almost complete success at detecting and correcting typical errors, and a browser… | ||
| Data Intelligence Platform on AWS | Artsiom Yudovin | 40m |
A practical data-catalog assistant is deployed that uses a retrieval-augmented pipeline: metadata from S3/Delta Lake feeds a vector index (tested with… | ||
| Impact of Vector Search | Atita Arora | 35m |
Claims that purpose-built vector databases beat bolt-on adapters for scale and latency, with concrete numbers and tools: HNSW graph index and libraries like… | ||
| (How) Are We Staying Smarter Than AI? | Boris Nikolai Konrad | 01h 07m |
A live demo memorizes 52 playing cards in 77 seconds and illustrates memory techniques built from images and a memory palace; MRI studies map distributed… | ||
| The Shitposting AI - An Ironic Solution to Fruitless Online Debate | Markus Schuttler | 43m |
Claims an automated system can replace humans in comment threads by training GPT2 variants on scraped Reddit and Twitter data and running continuous… | ||
| Insights and Key Learnings on Our AI Journey | Milad Abbaszadeh, Leonie Behrens | 40m |
Company converts five years of pilots into an enterprise data platform with Databricks feeding Google Cloud Storage BigQuery Tableau CA, builds a reusable… | ||
| Building a Unified Viewing Experience with Multi-Provider TV Recommendation | Natalija Tumanova | 42m |
An aggregator TV recommender that unifies on‑demand, partner catalogs and ~100 linear channels is built by consolidating messy multilingual metadata into a… | ||
| Anatomy of Inter-Agentic APIs | Swanand Rao | 38m |
Claims an offline inference at source platform that runs air-gapped on-device and a VS Code co‑pilot called M9 developer, demonstrated generating code, fixes… | ||
| Putting AI in Production | Bilge Ince | 27m |
Claims that Postgres becomes an AI database by combining PGVector for HNSW/IVF similarity search with an EDB Postgres AI extension called AIDB that… | ||
| A Journey in Space with Apache Kafka data streams from NASA | Frank Munz | 50m |
Reads live NASA GCN Kafka topics with an Apache Kafka client, ingests ~3,000 historical messages, and shows a reproducible GitHub notebook pipeline on the… | ||
| The State of Data Engineering | Jesse Anderson | 53m |
Claim: about 85% fail 15% succeed on data projects (Gartner) and measured team value trends are sliding down not up. Evidence: surveys show 80% of data… | ||
| Creating a Generative AI Adoption Roadmap | Jim Sterne | 47m |
A clear roadmap emerges: assess current usage with anonymous surveys, score executives on capability and willingness, form an AI Council and a frontier task… | ||
| GenAI Beyond Prototyping: The Path to Production with AI-Native Databases | JP Hwang | 40m |
Demo shows an AI‑native database called W8 powering a Streamlit RAG prototype that runs on a local model or API and returns searches in ~300 ms and claims a… | ||
| How to Build an Innovation Funnel That Lets You Constantly Create Strategic Value? | Matt Kurleto | 39m |
Contends that there is no AI strategy as a product but a need to build an environment for continuous innovation, and gives a concrete four‑stage innovation… | ||
| The Hidden Jewels to Solve Confabulation Challenges in Generative AI’s RAG Promise | Moustafa Eshra | 35m |
Claims concrete fixes: DataStax AI Platform packages Langflow and vetted libs and reports 50 companies moved rag into production serving 15 million users daily… | ||
| Empowering Business Users to Apply AI to SAP Data | Philipp Kiencke | 39m |
Shows concrete pipelines: SAP tables are turned into data products (single flat tables) fed to Fabric and Power Platform, clustering cuts 3,500… | ||
| LLMs for Self-serve Analytics - Lessons Learned | Thomas Schmidt | 38m |
Prototype connects Metabase to a real warehouse and shows that text-to-SQL fails in the wild: context windows, noisy schemas and undocumented temporary tables… | ||
| From Quick Wins to Revolutionising Productivity & CX with GenAI | Anna Semjen | 29m |
Claims a pipeline that vectorize entire videos with timestamps, stores vectors in DataStax Astra DB, and answers natural language queries by jumping the UI to… | ||
| Data Contracts In Practice With Debezium and Apache Flink | Gunnar Morling | 37m |
Log‑based CDC via Debezium plus Apache Flink converts table change streams into stable, evolvable contracts: the outbox pattern gives transactionally safe… | ||
| Future Data Architecture Enabled by New AI Technology | Pedram Birounvand | 36m |
Claims that AI plus vector search will cut dependence on data specialists and automate data catalogs, SQL generation and answers from warehouse-backed… | ||
| The State of Apache Airflow | Ricardo Sueiras | 01h 08m |
Claims heavy on community and numbers: 165 million downloads and ~2,000 contributors, Airflow 2.10 is current, demos show operator-based orchestration, Amazon… | ||
| Large-Scale Logging Made Easy | Aliaksandr Valialkin | 40m |
Claims an open source system named Victoria Logs that runs on a laptop and in production, stores terabytes by using 50× compression on typical Kubernetes logs… | ||
| Data-Driven Transformation: Organizational Changes and Leadership Insights | Alexey Novakov, Artsiom Yudovin, Marcell Igneczi, Oleks Maistrenko, Swanand Rao | 46m |
Panel reports concrete wins and failures: a city platform ingests 10 million camera events/day and 2 million IoT events to power transport models that reuse… | ||
| Beyond the Hype: Realistic Expectations of AI | Andreas Buhlmeier, Atita Arora, Stephen Batifol, Peter Farkas, Thomas Schmidt, Zamina Ahmad | 40m |
Panelists report concrete wins and limits: Perplexity shopping curates product lists, Copali vision LLMs finally extract structure from PDFs, and voice cloning… | ||
| Building a Resilient, Safe and Smart City in Challenging Times: Kyiv Case Study | Oleks Maistrenko | 42m |
A Situation Awareness Center for Kyiv collects live feeds and integrates 10 million detections per day from CCTV plus 2 million IoT events per day, 60+ data… | ||
| An Evolution of Data Architectures - Lambda, Kappa, Delta, Mesh & Fabric | Paul Andrew | 42m |
Claims that Delta Lake adds ACID transactions to data lakes (Databricks released Delta in February 2019 and open‑sourced it the same year), that Lambda→Kappa… | ||
| Empowering Real-Time ML Inference and Training with GRIS | Takahiko Saito | 43m |
GRIS delivers a real system: a feature store and gRPC inference API built to support real-time inference and offline training with Iceberg historical datasets… | ||
| Streamhouse Architecture with Flink and Paimon | Alexey Novakov | 42m |
Claims a workable streaming lakehouse built with Flink + Apache Paimon + Flink CDC, not just hype: Paimon is a table format tuned for streaming with upserts… | ||
| Shaping the Future of Real-time Data Pipeline | Bobur Umurzokov | 44m |
Claims a simple path to run batch code as streaming from day one and demonstrates a working serverless pipeline: Python transformation code deployed to glass… | ||
| Tips on Schema, Queries, Parallel Access, and Reactive Programming | Hartmut Armbruster | 42m |
Designs a backend for an authenticated social feed with concrete CQL schemas and query patterns, chooses ULID / UUIDv7 for time-sortable IDs, keeps counters in… | ||
| Building Effective Data Teams: Strategies for Success | Gunnar Morling, Martin Zuern, Olena Kutsenko, Paul Andrew | 38m |
Forty‑five minute panel reports concrete tactics: build a decentralized internal data platform that lets business teams own ingestion and produce bronze silver… | ||
| Three Practical Ways to Use AI in Your Company | Andrzej Kinastowski | 45m |
Three concrete implementations and how they work today: using GenAI to write complex Excel formulas and code, producing a 60‑page report in three days… | ||
| The Road to Intent-Driven Automation | Johannes Dienst | 43m |
Claims that domain experts can type an intent and the system executes it; demonstration of an early intent pilot that uses GPT plus Computer Vision to detect… | ||
| RPA Implementation: Doing It Differently | Justina Kasiuleviciute | 44m |
Claims that one size does not fit all and proves it with three concrete company case studies using Blue Prism, UiPath, Automation Anywhere, showing federated… | ||
| Democratizing Digitalization: Empowering Every Layer of Your Organization | Marcell Igneczi | 26m |
Claims that digital skills can be made universal, backed by concrete practices: run a start small pilot (one dashboard or one user story), host regular Q&A… | ||
| AI Chats: Challenges in Business Integration | Marcin Szymaniuk, Bartek Sadlej | 45m |
Shows runnable code using LangChain and vector databases to do sentiment tagging and RAG retrieval, measures retrieval with precision and recall on a golden… | ||
| Transforming Financial Services with Generative AI | Antonio Zarauz Moreno | 27m |
Credicorp describes building Retrieval‑Augmented Generation (RAG) systems for knowledge search, hyper‑personalized marketing that uses image… | ||
| Using Your Digital Twin to Understand the Effect of Future Changes | Henrik Ekstam | 36m |
Claims that a digital twin speeds the OODA loop and focuses portfolio decisions by mapping value streams to business capabilities and tech, and that a… | ||
| Unlocking Profit: How AI-Driven Budget Optimization Elevates Marketing Performance | Mauro Fusco | 41m |
Claims a production system that combines hierarchical time‑series marketing‑mix models with incremental optimization algorithms and AI creative scoring, and… | ||
| Marketing Mastery, Systems & Processes, 10X Sales Process | Niks Jansons | 45m |
Claims 2,100 European clients helped and a seven‑figure business built in three years; an LVYouTube channel doing 100–200k views/month and viral examples… | ||
| AI Agents in Action: Transforming Enterprise Processes | Karyna Mihalevich | 42m |
Claims an end-to-end quote-to-cash agent that extracts fields from uploaded PDFs, validates against SAP master data, and creates SAP sales orders after human… | ||
| From Quick Wins to Revolutionising Productivity & CX with GenAI | Anna Semjen | 29m |
Claims a working pipeline that vectorizes entire videos with specific timestamps into DataStax Astra DB, using ViT‑B/32 for frames (about 1,536 sampled frames)… | ||
| Integrating AI with RPA | Ana Marija Barisic, Andrzej Kinatowski, Pedram Birounvand, Swanand Rao, Alius Petraska | 43m |
Panel claims show practical wins not hype: an LLM reformats Polish addresses 100% accurate for pennies, call‑center tooling that autofills 10–15 minute… | ||
| Can AI Face Your (Potential) Customers? | Alius Petraska | 51m |
Claims concrete wins: a custom speech‑to‑text pipeline trained with freelancers beat Azure and AWS in two weeks for Baltic languages, handles multi‑language… | ||
| Automating Web Workflows with LLMs | Jiri Moravcik | 47m |
Claims that Apify turns natural-language prompts into repeatable web workflows, using Apify's AI web agent with Python/JS SDKs and open source libraries like… | ||
| nothing matches | ||