Scale By The Bay 2018
2018-11-15..2018-11-17
San Francisco, United States
[Scale By the Bay 2017, Scale By The Bay 2018, Scale By The Bay 2019, Scale By the Bay 2020, Scale By the Bay 2021, Scale By the Bay 2023]
as the organisers put it
Scale By the Bay is the 5th year of the flagship By the Bay conference, the proven data engineering and data science community event laser-focused on learning, open-source excellence, and industry-centric focus on what works.
| title | author | length |
|---|---|---|
| Deploying Kafka Streams Applications... | Gwen Shapira, Matthias Sax | 40m |
Kafka Streams runs as partitioned tasks with a one-to-one task-to-partition mapping and local state backed by a compacted changelog topic, so scaling requires… | ||
| Quantum Computing Modeling in Scala | Constantin Gonciulea | 24m |
This talk video is a node in devreal.ai, the developer community graph.
It connects to:
Talk: Quantum Computing Modeling in Scala —… | ||
| Journey of Building a Modern Data Prep Tool... | Jianneng Li | 20m |
Workday Prism Analytics builds a data‑prep engine on Apache Spark that runs two apps: always-on interactive for sampled, low‑latency queries and on-demand… | ||
| Reactive Microservice framework for... | Phani Srikar Ganti, Erin Kavanaugh | 30m |
Capital One builds Quantum Real Time, a Scala/Akka framework packaged with Docker that starts REST or stream endpoints and runs language‑agnostic plugins… | ||
| Simplicity for Programmable Money | Russell O’Connor | 39m |
Simplicity is a typed, combinator-based language with no recursion or function types that targets crypto scripts; it defines programs as functions between… | ||
| Street fighting techniques for multi tenant machine learning... | Sky Thomas | 40m |
Deploys a MapR CSI storage cluster inside Kubernetes, installs Kubeflow and JupyterHub, and launches training and Spark jobs so the demo actually runs… | ||
| Structure and Interpretation of Stream Processing | Adil Akhter | 43m |
Compares concrete systems (Flink, Spark Streaming, Storm, Heron, Samza) and shows how choices change guarantees: Flink and Spark offer exactly‑once semantics… | ||
| Leveraging Scala to Build Hardware at Scale | Yunsup Lee | 43m |
SiFive describes a full chip stack written in Scala: Chisel as the hardware construction DSL, the Diplomacy parameter framework and FIRRTL as the compiler that… | ||
| Quantum Computing and You | John Azariah | 47m |
Explains why quantum machines aim to simulate chemistry problems that need 2^N complex amplitudes — the featured nitrogenase-like molecule has ~170 electrons… | ||
| Cloud, Edge, and Silver Lining | 45m | |
Panelists debate concrete tradeoffs: enterprises keep primary data on-premises and use the cloud for burst compute while Alluxio offers Alluxio global… | ||
| Nelson - Functional programming in system design | Adelbert Chang | 40m |
Nelson is a deployment orchestration system that enforces immutable deployments by tracking explicit service name+version dependencies and publishing that… | ||
| Scala the Cloud Native Way... | Dennis Adjei Baah | 23m |
Linkerd is a Scala/Finagle/Netty service mesh used in production at Salesforce, CreditKarma and FOX that struggles with per-sidecar memory. Engineers switch… | ||
| Fireside Chat with Richard Socher, Chief Scientist, Salesforce | Alexy Khrabrov, Richard Socher | 32m |
A research group integrates published neural techniques into products, improves accuracy by 20% on a sequence-tagging task for contact/email matching, and… | ||
| Swimming in the stream... | Gabriel Claramunt | 24m |
Live-coded pipeline uses Akka Streams to ingest a random request generator, enrich each tuple with customer/product lookups (fake ODB returning Futures), call… | ||
| Orchestrating Microservices with GraphQL | Greg Kesler | 22m |
Intuit is mid‑journey decomposing QuickBooks Online into microservices and uses GraphQL front-end parsing only to accept and validate requests while a custom… | ||
| Using Akka Streams for Web scale Data Ingestion | Jie Ren, Greg Methvin | 27m |
Iterable moves from RabbitMQ to Kafka to fix single‑point failures and spiky flow control, and wires Akka Streams + Alpakka Kafka to consume partitioned topics… | ||
| 5 tips to build long lasting Scala OSS | Kazuhiro Sera | 21m |
Seven years of work on ScalikeJDBC yields five concrete rules: pick a project worth lifelong maintenance, prefer Java over Scala dependencies to avoid… | ||
| Distinguishing features of production quality pipelines | Nimbus Goehausen | 24m |
Practical checklist for taking Scala/Spark proofs of concept to production: prefer append-only data sources or versioned snapshots to avoid lost records… | ||
| Scaling Bayesian Experimentation | Robert J. Neal, Paul Cho | 25m |
Argues that Bayesian A/B testing uses priors on sample means (normal + inverse-gamma) so posteriors have closed form and avoid slow MCMC, and shows a… | ||
| Fast Data pipelines with Akka Streams and Alpakka... | Sean Glover | 41m |
Overview of building low‑latency, stateful Kafka pipelines using Akka Streams with the Alpakka Kafka connector, showing a concrete consume‑transform‑produce… | ||
| Data Consistency Patterns in Cloud Native Applications | Ryan Knight | 30m |
Compares concrete patterns: Distributed Saga (AP) requires idempotent workers and compensating actions and often leaves unsafe intermediate states; Akka… | ||
| Optimizing network topologies... | Wyatt Meldman Floch, Ryle Goehausen | 22m |
Claim: a DAG-based consensus replaces proof-of-work by layering signature-backed units and dynamic reputation to scale confirmation without heavy mining… | ||
| Privacy aware data science in Scala with monads... | David Andrzejewski | 38m |
Proposes a Scala library that enforces information-flow policies at compile time by translating a Haskell seclib design into a monadic security wrapper with… | ||
| FP for Data Science For Loops Considered Harmful | Aris Vlasakakis | 21m |
Argues that using static types and immutability prevents whole classes of bugs (matrix-dimension and unit mismatches become compiler errors), replaces brittle… | ||
| Data Engineering and AI | 53m | |
Panel explains concrete choices and failure modes: teams use Kubeflow for model orchestration and Argo/Airflow for workflows, prefer Python stacks (PyTorch… | ||
| Reactive Java Programming: a new Asynchronous Database... | Kuassi Mensah | 26m |
Proposal for a new Java DBA API promises truly non‑blocking SQL access with no user thread ever blocked, built on Java SE 8/9 using CompletionStage and… | ||
| Practical Reactive Streams with Monix | Jacek Kunicki | 35m |
Processes gzipped CSV lines with Monix Observables, parses each line into a domain Reading Task, groups readings by ID in pairs, computes averages (invalid… | ||
| The Danger of Implicit Blocking in Finagle | Michael Armella | 15m |
Finagle services block when long computations or explicit waits run on Netty event-loop threads, causing worker threads (default 2× logical CPUs) to exhaust… | ||
| Monitoring Reactive Streams | Stefano Bonetti | 26m |
Measures element throughput checkpoints at chosen graph points and records a per-checkpoint rate (elements/sec) so source, filter and sink rates can be… | ||
| H2O internals | Cliff Click | 33m |
H2O is a pure-Java cluster runtime that stores compressed columnar data in-heap, runs most analytics at memory-bandwidth speeds and uses a MapReduce-style… | ||
| Thoughtful Software Engineering | Bryan Cantrill | 46m |
Panel argues concrete trade-offs and tools: microservices trade autonomy for lost leverage and added serialization/operational complexity; Scala trims features… | ||
| Making Spark ML Models Portable Know Your Options | Matthew Tovbin | 24m |
Compares ways to export Spark ML pipelines: use Spark runtime, DataBricks/local jars, the PFA standard, or build a custom runtime. Describes a custom library… | ||
| Creating a Data Fabric for IoT | Karthik Ramasamy | 30m |
Apache Pulsar is presented as a single system that combines messaging, storage and stream processing with real deployments: runs on Raspberry Pi and… | ||
| A Reactive Fraud Monitoring Engine for Instant Payments | David Massart | 20m |
Reactive pipeline takes channel events, fans them into parallel predictive modules and aggregates partial scores into an instruction: allow / stop / step‑up… | ||
| Consensus Algorithms in Distributed Systems | Yifan Xing | 40m |
Explains how Paxos: prepare/accept with proposal numbers and Raft: leader election + log replication actually operate, including Paxos proposers/acceptors… | ||
| 2 Fast 2 Furious: migrating Medium's architecture... | Sasha Solomon | 24m |
Medium migrates its frontend to a React client backed by a Scala GraphQL server (Sangria) that derives schema from protobufs as schemas, uses fetchers that map… | ||
| Programming the worldwide elastic supercomputer with Unison | Paul Chiusano | 22m |
Unison claims a single program can express an elastic distributed system and shows the concrete primitives: spawn / at / force primitives produce nodes, start… | ||
| Introduction to Apache Spark with Frameless | Bryan Clapper | 46m |
Compares Spark DataFrame/Dataset APIs to Frameless by running live code on ~50,000 tweet samples and building typed pipelines that keep Catalyst optimizations… | ||
| Applied Machine Learning: a Netflix production | Julie Pitt | 27m |
Netflix describes Metaflow as a Python-first workflow layer that actually ships: it provides a DAG decorator API, resume-any-step reproducibility backed by an… | ||
| Motivating Probabilistic Programming | Rahul Chitturi | 29m |
Built proml, a purely functional Scala library that makes model parameters distributions instead of points, using Metropolis–Hastings MCMC to sample posterior… | ||
| Continuous ML Applications in Production | Chris Fregly | 25m |
PipelineAI shows a production runtime that continuously trains, optimizes, validates and explains models using Continuous VOTE techniques, feeding online… | ||
| Monitoring AI with AI | Stepan Pushkarev | 25m |
Model deployments include an immutable package that bundles training data statistics (min/max, quantiles, autoencoder snapshots) and run-time checks like… | ||
| Video Access-Log Processing with Apache Flink | Scott Kidder | 22m |
Mux processes CDN access logs to meter per-account, per-asset video usage and power no-commit utility billing by enriching logs, windowing them by event time… | ||
| MLflow: An open platform to simplify... | Aaron Davidson | 24m |
MLflow is an open source Databricks platform (launched June, ~57 contributors) that actually implements tracking, projects, models: a logging API that records… | ||
| Hadoop Future in AI World | Milind Bhandarkar | 40m |
This talk video is a node in devreal.ai, the developer community graph.
It connects to:
Talk: Hadoop Future in AI World —… | ||
| Understanding World food economy with satellite | Aleksandra Kudriashova | 21m |
Pipeline monitors corn, soy, rice and wheat daily by combining Sentinel‑2, Landsat and Planet imagery, extracts NDVI and many indices, and trains ML (boosting)… | ||
| Edge ain't your gramp's IoT... | Roman Shaposhnik | 34m |
Claims a working Edge Virtualization Engine called Project EVE (open source under Linux Foundation) that runs on $500 industrial PCs down to $7 ARM boards… | ||
| Building a Contacts Graph from activity data | Alexis Roos, Brad Powley | 46m |
Salesforce teams build a contact graph from email and calendar metadata ingested via Kafka and stored in S3, update the raw per-org graph roughly weekly… | ||
| Graph-First Services Using GraphQL | Adam Crane | 25m |
GraphQL schema drives service design, used at scale inside Twitter to unify many back ends and power a Network Health Visualizer. Implementation uses Sangria +… | ||
| Inside NVIDIA’s AI infrastructure for self-driving cars | Clement Farabet | 32m |
NVIDIA describes Project MagLev as an internal AI platform that ingests multi‑PB road data from ~30 cars with 12 cameras each, runs active‑learning selection… | ||
| Distributed Deep Learning with Horovod | Alex Sergeev | 29m |
Horovod is a library that runs on top of TensorFlow and PyTorch and uses MPI plus NVIDIA’s NCCL for fast collectives, implementing ring-allreduce and a… | ||
| Scio data processing nirvana at Spotify | Bram Leenders, Julien Tournay | 24m |
Scio is Spotify’s Scala API for Dataflow and after two years in production it replaces Crunch for most pipelines, handling >200 TB/day with ~3,000 jobs. A… | ||
| Structured Deep Learning with Probabilistic Neural Prog | Jayant Krishnamurthy | 32m |
Claims an open source Scala library called Probabilistic Neural Programs that combines a probability monad with neural nets to let a model define its output… | ||
| Labels to Inference: A Continuous Sentiment Pipeline | Jeff Fenchel | 21m |
Company rebuilds sentiment as a continuous pipeline that labels, trains and deploys daily using Mechanical Turk, Elasticsearch, Kafka/Storm, and Amazon… | ||
| FiloDB: Real-time, In-Memory Time Series at Massive SMACK Scale | Evan Chan | 45m |
FiloDB is an open-source, in-memory time-series database built on Scala, Akka, Kafka, Cassandra and Mesos that targets high-cardinality monitoring and… | ||
| High-performance functional bayesian inference in Scala | Avi Bryant | 55m |
Rainier is an open source Scala library that compiles compute graphs for probabilistic models into zero-allocation JVM bytecode using ASM, auto-derives… | ||
| Enabling Big Data and Machine Learning for the Masses... | Tim Kral | 38m |
Medium replaces mixin-heavy Scala Spark jobs with a dependency-injected platform that treats extraction and loading as services and leaves product teams to… | ||
| Towards Typesafe Deep Learning in Scala | Tongfei Chen | 35m |
Nexus is a prototype Scala deep‑learning engine that encodes tensor shapes and axis semantics into types so axes of tensors are typed statically and operations… | ||
| Machine Learning on Source Code | Francesc Campoy Flores | 37m |
Provides concrete tooling and datasets: a public archive (3 TB, repositories with 50+ stars) pulls entire git blobs for training, parsing into a universal AST… | ||
| Down the Wabbit Hole | Chris McKinlay | 42m |
Formation builds polymorphic model handles in Haskell that abstract TensorFlow sessions and Vowpal Wabbit pointers so the same pipeline runs both frameworks… | ||
| Effects types in Scala - how to choose one | Alexandre Bergeron | 23m |
Explains why Scala standard futures are already running (non-cancelable) while Twitter futures are cancelable, and compares Scalaz Task, Monix Task… | ||
| FP Scala Meat & Potatoes... | Paul Snively | 37m |
Demonstrates a working CRUD JSON REST API built with http4s, JSON codecs auto‑derived via Circe, and SQL via Doobie that parses POSTed JSON, inserts rows, and… | ||
| Scala.js in production | Ramnivas Laddad | 40m |
Learn Raga launches last week: a full frontend+backend music app with ~3,000 commits, ~90% Scala, and a 780 KB JS bundle. Uses Scala.js isomorphism to share… | ||
| Radix Trees: How IntMap Works | Tikhon Jelvis | 31m |
Haskell's IntMap is a radix tree (Patricia trie) that stores integer keys bitwise with path compression so lookups cost key-length, not map-size; worst case… | ||
| Fork It Harder Make It Better | Justin Kaeser | 37m |
Overview of concrete tooling and outcomes: interactive REPLs and Ammonite/worksheets and notebooks speed experimentation; a live demo shows Cargo publish in ~2… | ||
| Duality and How to Delete Half (minus ε) of Your Code | Greg Pfeil | 23m |
Claims that 96% minus ε of duplicated recursion-schemes code disappears by abstracting folds/unfolds over comonads and arrow categories, collapsing monadic… | ||
| Shapeless Party Tricks in the Enterprise | Cody Allen | 26m |
Practical examples of Shapeless show how compile-time literal-backed types let a combinations function return Tuple2 or Tuple3 instead of List, preserving type… | ||
| Effective Scala | Bill Venners, Frank Sommers | 41m |
Argues Scala works best as a third way — pragmatic functional object‑oriented code — and gives three concrete rules: pick a project opinion and stick to it… | ||
| Keynote III: Mind Your State for Your State of Mind | Pat Helland | 46m |
Argues that transactional semantics break at scale and that pragmatic choices win: use careful replacement when you must coordinate updates across identities… | ||
| Declarative distributed concurrency in Scala | Sergei Winitzki | 42m |
Claims a Scala DSL (chemist/Chemical Machine) expresses unrestricted concurrency as data-carrying “molecules” and reactions so many tasks fit in 15 lines of… | ||
| Adopting GraalVM | Petr Zapletal | 23m |
Evaluation of GraalVM covers concrete steps and pain points: SBT can use the Graal plugin and JVMCI flags to compile Scala projects and yields about 10%… | ||
| Rust and Other Interesting Things | Bryan Cantrill | 37m |
Recent experience shows Rust can beat hand‑optimized C on real system code by switching data structures and letting the compiler optimize memory layout… | ||
| You Are a Scala Contributor | Seth Tisue | 35m |
Scala work splits across three GitHub orgs (scala, scala-center, EPFL/dotty); the main repo holds the compiler, standard library and spec, and a majority of… | ||
| Keynote II: Kafka and the Rise of Event-Driven Microservices | Neha Narkehede | 27m |
Argues that Kafka turns all business activity into durable event streams so applications and analytics share a single source of truth, enabling real-time joins… | ||
| Getting started with EitherT | Julie Laver | 16m |
Claims that wrapping Future[Either[E,A]] with EitherT[Future, E, A] (from Cats/Scalaz) makes code readable by avoiding double-unwrapping and enables fast-fail… | ||
| How Twitter teaches Scala | Ivan Corneillet | 24m |
Twitter migrated from Rails to a Scala stack around 2009 and now runs Finagle/Finatra microservices; onboarding is a four‑day engineering intro that walks new… | ||
| Play on Dotty: Design Patterns unlocked by Dotty in a Play... | Jeff May | 39m |
Dotty (Scala 3, using 0.10.0‑RC1 with SBT 1.2.3) unlocks implicit functions and trait parameters to thread request context without thread‑locals. A Play‑like… | ||
| Scalaz Stream: Rebirth | John A De Goes, Itamar Ravid | 44m |
A next-generation scalaz-stream reimplementation targets the Scalaz 8 / ZIO effect model, exposing only Stream and Sink and abandoning spine deconstruction and… | ||
| ArrayDeques and How to Contribute to Scala 2.13 | Pathikrit Bhowmick | 23m |
mutable.ArrayDeque is a circular-buffer implementation that replaces many mutable Scala collections and claims faster common operations than Lists, Buffers… | ||
| Immutable APIs and mutable internals: a Scala design case study | Oscar Boykin | 25m |
Builds a parser combinator library in Scala with an immutable API but a hidden mutable backend that keeps a pointer into an immutable input string, uses a… | ||
| Transpiling GraphQL instead of writing customized.... | Michael Solomon | 29m |
Claim: GraphQL queries are transpiled at runtime into JavaScript-flavored functions that explicitly fetch and compose data across services and databases, so… | ||
| From Scala to ByteCode... | Alon Muchnick | 36m |
Shows exactly what scalac and javap produce: scalac phases: 25 steps culminating in a JVM generation phase, class files hold a constant pool, field/method… | ||
| Production Haskell Demystified | Joe Kachmar | 21m |
Haskell code models produce concrete artifacts automatically: using GHC Generics plus libraries like generic-random and Aeson the compiler derives QuickCheck… | ||
| Towards Parallelizing Scala Compilations | Eugene Burmako | 26m |
Rsc computes public type signatures called outlines, dumps them in the Scala compiler format, and then launches multiple scalac instances to compile partitions… | ||
| Rage Against the Ecosystem | John Pretty | 51m |
Claim: Fury is a source-based dependency manager that defines builds as data not code and replaces binary publishing pain with source-based dependencies. It… | ||
| Graal: How to use the new JVM JIT compiler in real life | Chris Thalinger | 41m |
Graal is the Java-written JIT now available via JDK 9’s JVMCI and JDK 10’s JEP 317 and can be enabled with -XX:+UnlockExperimentalVMOptions -XX:+EnableJVMCI… | ||
| Your Type System Working For You! | Richard Wall | 31m |
Shows how Scala turns CRS errors into compile-time guarantees by baking CRS definitions and transforms into types and implicits, using GeoTools to load EPSG… | ||
| Channeling the Inner Complexity | Jakob Odersky | 33m |
Compares Scala concurrency options and claims any scalable program needs a good model, then shows a library-only implementation called Eskow that emulates… | ||
| Opening Keynote: New Functional Constructs in Scala 3 | Martin Odersky | 51m |
Scala 3 adds concrete, tested constructs: enums as ADTs/GADTs replace verbose sealed-trait + case-class boilerplate and support Java interop (implementation… | ||
| Classical Category Theory in Plain Scala | Vlad Patryshev | 32m |
Scala code implements a full implementation of small categories (objects, arrows, identities, composition), graphs, functors, natural transformations, cones… | ||
| Interview | Bram Leenders | 02m |
Builds and operates event delivery pipelines that ingest client events and make them available to data engineers, describing concrete production work at… | ||
| Interview | Brad Powley | 03m |
A platform constructs an activity graph from emails and meetings and wires it into Einstein to provide contextual services for other machine learning features… | ||
| Interview | Alexis Roos | 04m |
Salesforce teams deploy Email Insights to spot real‑time signals (pricing talks, competitor mentions, even weather) from hundreds of TB and billions of… | ||
| Interview | Adelbert Chang | 03m |
A node in Target’s data infrastructure graph describes Nelson, an automated deployment system developed over several years for the Data Science and… | ||
| Interview | Adil Akhter | 05m |
Building a streaming data platform and a machine learning platform at ING to deploy models, run training jobs and monitor production performance, with Scala… | ||
| Interview | Avi Bryant | 06m |
Built a system called Rainier, a Bayesian inference engine for Scala that generates JVM bytecode, requiring work across statistics, compilers and runtime… | ||
| Interview | Adam Crane | 06m |
Built a data pipeline called Network Health Visualizer that collects latency, connectivity and config-change metrics from internal networks, exposes them… | ||
| Interview | Bill Venners | 08m |
Runs Artima and maintains ScalaTest and Scalactic, open source libraries used for Scala and Java testing and quality utilities; authorship credit on… | ||
| Interview | Frank Sommers | 05m |
Autospaces builds workflow and decision support systems for auto finance and adopts Scala from its early releases, evolving the codebase as the language grows… | ||
| Interview | Erin Kavanaugh | 04m |
Describes a configuration-driven framework that launches real-time microservices to execute credit models, with a concrete claim: a microservice built, wired… | ||
| Interview | Dennis Adjei Baah | 04m |
Building a production service mesh proxy in Rust and Go that integrates with Linkerd L7 proxy to give microservices measurable reliability, visibility, and… | ||
| Interview | David Winters | 05m |
Building GoPro's data platform for four years, creates a Spark-Kafka streaming ingestion pipeline that pushes device events into storage and near‑real‑time… | ||
| Interview | Brian M Clapper | 06m |
Compares and contrasts Spark’s three APIs — DataFrame, Dataset, Frameless — and argues frameless gives the strongest compile-time type safety for schema-heavy… | ||
| Interview | Cody Allen | 05m |
Building intelligence insights on activity data like emails and meetings to surface where deals sit in the sales pipeline and help managers make decisions… | ||
| Interview | Chris Fregly | 04m |
Claims a production platform that turns batch ML into continuous training pipelines and serves models with live streaming updates. Implements end-to-end pieces… | ||
| Interview | Brian Cantrill | 05m |
Builds a production cloud: a Mantle storage substrate with S3-like semantics plus a compute substrate for cloud workloads, engineered for extraordinary scale… | ||
| Interview | Jeff Fenchel | 04m |
Builds a production system that maps a real-time media landscape into actionable signals using streaming pipelines and NLP, aiming for continuous measurement… | ||
| Interview | Ivan Corneillet | 04m |
An internal training program at Twitter runs hands-on courses to level up engineers, teaching Scala with practical functional programming patterns and… | ||
| Interview | Julien Delange | 06m |
Building a metrics platform that collects network telemetry across Twitter to diagnose outages and track month‑scale trends for capacity planning, implemented… | ||
| Interview | John Azariah | 04m |
Lead of Microsoft’s Q# effort describes building the actual programming language for a working quantum computer and also contributing to quantum hardware and… | ||
| Interview | Jie Ren | 04m |
Describes building a new type of data store at Iterable that solves historically naggy scaling problems by using clever architecture and a Scala stack… | ||
| Interview | Jakob Odersky | 05m |
Describes building a treatment matching pipeline that connects Pharma and hospitals by gathering a database of treatments and automatically matching cancer… | ||
| Interview | Greg Methvin | 06m |
Working on a data ingestion pipeline at Iterable that uses Akka Streams and Kafka to process user events into a customer insights platform, with emphasis on… | ||
| Interview | Greg Kesler | 05m |
Intuit describes being an early GraphQL adopter since 2015 and building concrete developer tooling: SDKs and code-generation, developer infrastructure and Java… | ||
| Interview | Julien Tournay | 03m |
Spotify uses Scio as the default Scala-based data processing framework for batch and streaming pipelines, built on Apache Beam and adopted across teams. The… | ||
| Interview | Kazuhiro Sera | 04m |
An interview with Kazuhiro Sera from Salesforce about his work in the Scala ecosystem: his projects ScalikeJDBC and Skinny Framework, and his contributions to… | ||
| Interview | Lukas Biewald | 03m |
Weights & Biases builds experiment tracking tools for machine learning teams and watches hundreds of models across industries; the concrete recommendation is… | ||
| Interview | Matthias Sax | 03m |
Kafka Streams as a Java library on top of Kafka makes stream-processing applications fault-tolerant and scalable by embedding state and processing logic in the… | ||
| Interview | Micah Wylde | 04m |
Lyft is building a stream processing platform that runs real-time pipelines so researchers, service engineers and analysts can process data without deep… | ||
| Interview | Michelle Casbon | 05m |
Description of an open source platform that runs ML on Kubernetes called Kubeflow, positioned as a shortcut to avoid building startup-grade infra from scratch… | ||
| Interview | Nimbus Goehausen | 05m |
Builds and runs data pipelines that maintain a dataset of companies indexed by website and generates B2B targeting audiences for customers; focuses on… | ||
| Interview | Petr Zapletal | 06m |
Building a global real-time streaming platform that serves thousands of concurrent viewers for products like ESPN+ and MLB and is being extended toward… | ||
| Interview | Phani Srikar Ganti | 04m |
Capital One builds a framework called corn that provides out‑of‑the‑box capabilities for real-time execution of models and microservices so credit‑card… | ||
| Interview | Ramnivas Laddad | 05m |
A full web application that teaches Indian classical music uses full-stack Scala with Scala.js and React on the front end, processes live audio to give… | ||
| Interview | Robert J. Neal | 04m |
A Bayesian experimentation platform at Udemy handles Bayesian experimentation at scale by implementing bootstrapping with Spark and Scala using algorithmic… | ||
| Interview | Russell Alexander Spitzer | 06m |
Maintains the Spark Cassandra connector, a production connector used to run Spark on Cassandra data, and describes new work that uses the Catalyst optimization… | ||
| Interview | Francesc Campoy Flores | 07m |
A startup applies neural networks to source code using a 3 TB dataset of GitHub to train models that predict bugs, suggest better names, and detect when two… | ||
| Interview | Julie Pitt | 03m |
Built a production machine learning platform at Netflix that gives data scientists on-demand large-scale compute early in development and removes concerns… | ||
| Interview | Rúnar Bjarnason | 05m |
Description of work and claim: building Unison as a distributed programming language that makes highly distributed, elastic computation as easy to write as a… | ||
| Interview | Sean Glover | 06m |
Describes building production streaming platforms using Alpakka Kafka connector to wire Kafka into processors like Spark, Kafka Streams and Akka Streams, and… | ||
| Interview | Sergei Winitzki | 04m |
Building a reusable streaming pipeline for multiple machine learning applications at Workday, aiming to apply the same pipeline across future projects and… | ||
| Interview | Tim Kral | 05m |
Built a full curation workflow for Medium from soup to nuts, inventing processes to meet the scale required and integrating Scala pipelines and some ML to… | ||
| Interview | Tongfei Chen | 04m |
Describes a completely typesafe tensor operation library in Scala that implements deep learning primitives to eliminate runtime tensor-dimension and type… | ||
| Interview | Wyatt Meldman Floch | 05m |
Building a cryptocurrency-like system that replaces proof-of-work and proof-of-stake with a topology-optimizing game that lets nodes compete to be the best… | ||
| Interview | Justin Kaeser | 03m |
JetBrains maintains an IntelliJ IDEA Scala plugin that focuses on build tool integration and moves from tight SBT support toward using BSP (Build Server… | ||
| Interview | Aaron Davidson | 03m |
Building MLflow as an open source machine learning pipeline tool that companies actually use, with real bug reports coming in that validate production use and… | ||
| Interview | Aleksandra Kudriashova | 04m |
Astro Digital builds small-sat missions with multispectral cameras capturing red and near‑infrared bands at up to daily revisit, enabling per-field tracking of… | ||
| Interview | Alexandre Bergeron | 07m |
Building a data-platform at Frisco Data that lets analysts craft SQL-driven dashboards and run them fast and scale, with responsibility for a load-testing… | ||
| Interview | Brett Koonce | 04m |
CTO at Quarkworks building client mobile apps with offices in San Francisco and Columbia Missouri describes seeing apps used in the wild and a hands‑on… | ||
| Interview | Haoyuan Li | 04m |
An open source layer called Alluxio virtualizes cloud and on‑prem storage, giving data apps a global namespace and unified APIs so they only talk to Alluxio… | ||
| Interview | Holden Karau | 03m |
Builds and contributes to Apache Spark at Google and works across Airflow, SystemML, and Mahout, focusing on practical data pipelines that run in cloud… | ||
| Interview | Jianneng Li | 04m |
Prism Analytics uses Apache Spark to build end-to-end data pipelines that mix Workday and external data so customers move from raw tables to analysis-ready… | ||
| Interview | Paul Snively | 07m |
Building a personalized ad technology platform for several clients that drives loyalty programs through a multi-step engagement flow that increases customer… | ||
| Interview | Russell O Conner | 04m |
Building a new blockchain language that targets smart contracts and gives statically verifiable costs, with the goal of proving correctness because contracts… | ||
| Interview | Sky Thomas | 05m |
MapR is building an ML and AI environment for Kubernetes that claims to make reproducible experimentation practical for data scientists who previously… | ||
| Interview | Vlad Patryshev | 04m |
Built an interoperability layer connecting Salesforce core storage to ML pipelines that maps and preserves every field type and representation for downstream… | ||
| Interview | Yunsup Lee | 05m |
Builds RISC-V processors and RISC-V-based SoCs while using Scala as the hardware construction language, claiming it makes hardware design far more productive… | ||
| Interview | Tikhon Jelvis | 02m |
Lead data scientist at Target describes a production simulation platform that models real Target distribution centers and runs at warehouse scale, built with… | ||
| Interview | Joe Kachmar | 03m |
Works on developer and client tooling in Haskell and contributes to Haskell package infrastructure, so expect concrete examples of tooling and package fixes… | ||
| Scale By The Bay 2018 Unconference: Gabriel Claramunt | Gabriel Claramunt | 22m |
Library implements a generic fold over regular types in Scala by encoding constructors as small functors and a Fix-like wrapper, translating Haskell… | ||
| Scale By The Bay 2018 Unconference: Greg Pfeil | Greg Pfeil | 37m |
Dhall is a total, non-Turing-complete configuration language that still expresses functor, monad and category abstractions, implements kind polymorphism and… | ||
| Scale By The Bay 2018 Unconference: Runar Bjarnason | Rúnar Bjarnason | 46m |
Starts from plain functions and builds category theory step by step, formalizing composition and identity then instantiating them: Scala subtype category… | ||
| Scale By The Bay 2018 Unconference: Sergei Winitzki | Sergei Winitzki | 42m |
A Scala macro library generates concrete implementations from polymorphic type signatures by translating types into logic, running an LJT proof search to build… | ||
| Scale By The Bay 2018 Unconference: Joe Kachmar | Joe Kachmar | 25m |
Builds a Haskell pattern that restricts effects by encoding interfaces as type classes and wiring concrete implementations into a ReaderT ReaderT app newtype… | ||
| Scale By The Bay 2018 Unconference: Gabriel Volpe | Gabriel Volpe | 16m |
Live demo shows wiring PureConfig, cats-effect IO and MTL-style typeclasses to avoid transformer stacks by deriving classy optics lenses and meow-mtl instances… | ||
| Scale By The Bay 2018 Unconference: Devon Stewart | Devon Stewart | 36m |
Generates strongly typed Scala server and client code from OpenAPI/Swagger using a compile‑time sbt plugin so generated files are never committed; works with… | ||
| Scale By The Bay 2018 Unconference: Taro L Saito | Taro L Saito | 31m |
Lightweight Scala library airframe delivers dependency injection plus 18 modules for config, launchers, logging, HTTP, serialization and lifecycle management… | ||
| Scale By The Bay 2018 Unconference: Omer Van Kloeten | Omer Van Kloeten | 22m |
Claim: move validations from runtime to compile time using Scala libraries so that things like a port are a refined type (Int in 0–65535) and petting APIs… | ||
| Scale By The Bay 2018 Unconference: Rob Norris | Rob Norris | 36m |
Skunk is a Scala Postgres client that talks the Postgres wire protocol — no JDBC — and composes with cats-effect, fs2 and nio to give resource-safe sessions… | ||
| Highlights | 02m | |
ai.bythebay.io Nov 2025, Oakland, full-stack AI conference Scale By the Bay 2019 is held on November 13-15 in sunny Oakland, California, on the shores of Lake… | ||
| Stream Processing at Lyft with Flink and Beam | Sherin Thomas, Micah Wylde | 32m |
Lyft runs real-time pipelines using Apache Flink as the runner and is adopting Apache Beam for Python portability, operating 50+ Flink jobs processing tens of… | ||
| Concurrency with Cats-effect | Michael Pilquist | 36m |
Cats Effect 1.0 provides fiber-based concurrency and synchronization primitives and shows concrete implementations: a correct parTuple2 that uses bracket to… | ||
| nothing matches | ||