Scale By The Bay 2019
2019-11-13..2019-11-15
Oakland, United States
[Scale By the Bay 2017, Scale By The Bay 2018, Scale By The Bay 2019, Scale By the Bay 2020, Scale By the Bay 2021, Scale By the Bay 2023]
| title | author | length |
|---|---|---|
| Quill + Doobie = Better Together | Alexander loffe | 31m |
Shows a concrete integration: Quill handles composable, type-driven SQL fragments and macro-expands them at compile time so you can see the SQL before it runs… | ||
| Solving the Scala Notebook Experience | Jeremy Smith, Jonathan Indig | 33m |
PolyNote is a polyglot, server-backed notebook that goes from POC to usable MVP in under ten months and dozens of users. It runs Scala, Python and Spark SQL… | ||
| The Times Are A-Changin' | Heather Miller | 35m |
U.S. demand for developers is exploding with about 660,000 graduates now and 7.4% annual degree growth plus bootcamps growing ~14%, and the Bureau of Labor… | ||
| Metals - building rich IDE features beyond the Language Server Protocol | Olafur Pall Geirsson | 29m |
Metals, a Scala language server, extends LSP with Debug Adapter Protocol, Tree View Protocol and Build Server Protocol (BSP) to provide run/test/debug and… | ||
| Re-programming the programmer, from Actors to FP | Paul Cleary | 30m |
A DNS management service evolves through three rewrites from event sourcing with Akka to an actor-based CQRS on DynamoDB to a functional design using FS2 +… | ||
| Thank you, next: Iterators | Umayah Abdennabi | 32m |
Company runs an internal analytics platform called NAR that feeds Spark using Kinesis shard iterator instances forced into synchronous batches; Spark’s core… | ||
| Hacking F# in JS ecosystem | Krzysztof Cieślak | 31m |
F# code compiles to modern JavaScript by translating F# AST → Fable AST → Babel AST and letting Babel emit ES6 modules/iterables with no added runtime… | ||
| Speedy Scala Builds at Databricks | Ahir Reddy, Li Haoyi | 32m |
Databricks migrates large Scala mono-repos onto Bazel-style builds, yielding concrete wins: test runs drop from ~150 minutes to ~25 minutes via sandboxed… | ||
| Functional Electromagnetism | James Earl Douglas | 30m |
Builds a working Morse-code transmitter in GNU Radio Companion sending over a HackRF One and received on a cheap DVB‑T dongle and handheld radio, so the signal… | ||
| Scoring ONNX ML Models with Scala | Oscar Boykin | 31m |
Agate is a pure-Scala library that scores ONNX models on the JVM, used at Stripe for batch scoring with Spark and Scalding and for low-latency real-time… | ||
| A brief introduction to systems programming, with Scala Native | Richard Whaling | 29m |
Scala Native’s unsafe API gives Scala direct C-level control while staying in Scala: Scala Native 0.4.0-M2 compiles with LLVM, exposes pointers (8-byte… | ||
| Who Needs Serverless? | Vitaly Gordon, Steve Newman, James Ward, Rose Toomey, George Mathew | 40m |
Motion: ten years from now most newly created billion-dollar companies are entirely serverless. The panel debates concrete tradeoffs: AWS Lambda and Fargate… | ||
| Recursion schemes with Higherkindness | Oli Makhasoeva, Andy Scott | 32m |
Live refactor of list fold/unfold into generic recursion schemes using Scala, Cats and the Droste ideas, extracting a project functor (option of tuple)… | ||
| Rsc: Scala Outlining for Distributed Compilation | Win Wang | 31m |
Rsc produces Scala “outlines” (Scala header files) that let compilation start on dependents earlier, enabling parallel and distributed builds using Rsc or… | ||
| A Gentle Introduction to Comonads | Justin Heyes-Jones | 24m |
Claims that comonads let pure code inspect and transform values with their surrounding context; implements a Comonad instance for a 2D FocusGrid where extract… | ||
| Runtime Types at Crunchbase | Themba Fletcher | 21m |
Crunchbase replaces per-entity Avro/OpenAPI specs with a single metadata-driven entity definitions service, moving type checks to request time and storing… | ||
| GDPR Data Cleaner: Mutating Immutable Data | David Winters | 27m |
GoPro builds a configuration-driven GDPR data cleaner that lets teams delete user records from big data without replacing the platform: it uses Spark SQL… | ||
| What is Functional Reactive Programming? | Tikhon Jelvis | 29m |
Defines FRP as a time-focused, functional-style abstraction that represents continuous state as behaviors and discrete occurrences as events, with concrete… | ||
| Unison, and why the codebase of the future is a purely functional... | Paul Chiusano | 32m |
Unison identifies every definition by a content-addressed hash (real system uses 512-bit SHA-3), stores a serialized AST per hash plus name metadata, and keeps… | ||
| In Types We Trust | Bill Venners | 37m |
Proposes expressing semantic contracts as predicate-logic statements that feed both property-based tests and machine-checked proofs and shows concrete support… | ||
| Rust and Scala, Sitting in a Tree…. | Evan Chan | 28m |
A Scala in-memory time-series DB stores 100 billion data points and ingests 1M samples/sec but hits ~20s GC pauses; moving off-heap was painful. Rust’s… | ||
| Kubernetes is a Platform Platform | Joe Beda | 31m |
Kubernetes is presented as a control plane built on etcd and API server with scheduler, kubelet and controllers that reconcile desired vs actual state… | ||
| Netty 5: Lessons Learned | Norman Maurer | 39m |
Netty 5 removes costly handler traversal with a new skip annotation that reduces per-read call sites and, in a five-handler microbenchmark, raises operations… | ||
| Taming complex webapps with Scala and React | Kavita Laddad | 32m |
This talk video is a node in devreal.ai, the developer community graph.
It connects to:
Talk: Scale By The Bay 2019: Kavita Laddad, Taming complex webapps with… | ||
| Deploy end to end ML pipeline using Apache spark ... | Prashant Sharma, Nick Pentreath | 30m |
Deploys a full streaming ML pipeline that uses Apache Spark Structured Streaming, Kubernetes and Kafka to ingest simulated tweets, batch them for throughput… | ||
| Dagster: a Framework for Data Processing Applications | Nick Shrock | 29m |
An open-source Python framework called Dagster models pipelines as graphs of typed functions and ships a local IDE, auto-completing type-checked config, live… | ||
| High Performance Serverless Functions in Scala | Jason Swartz | 24m |
Compiles vanilla Scala fat jars with SBT assembly into native Linux executables via GraalVM native-image (built in Docker), deploys those to AWS Lambda custom… | ||
| Integrating Developer Experiences - Build Server Protocol and beyond | Justin Kaiser | 28m |
Build Server Protocol (BSP) standardizes IDE↔build-tool communication so IntelliJ can import any BSP-enabled build by reading a dropped config and launching a… | ||
| To Spark or Not to Spark | Russell Alexander Spitzer | 33m |
If you need joins, cross-source aggregation or a JDBC endpoint over Cassandra, Spark plus the Cassandra connector gives that: use spark thrift server for a… | ||
| Growing the Scala Community | Yifan Xing | 30m |
ScalaBridge runs one-day workshops and a 12-week program in more than 15 cities and needs mentors and front-end developers to scale; one London graduate lands… | ||
| Finding Needles In Big Data Haystacks.... | Rashmi Shamprasad, Ajit Koti | 34m |
This talk video is a node in devreal.ai, the developer community graph.
It connects to:
Talk: Scale By The Bay 2019: Rashmi Shamprasad & Ajit Koti, Finding… | ||
| Next-Level Diagnostics.... with ZIO | Salar Rahmanian, John A De Goes | 32m |
ZIO provides line‑level execution traces that show what did and would have happened across async and parallel regions, plus a fiber dump that lists every… | ||
| Change Data Capture in Distributed Systems | Petr Zapletal | 20m |
Compares three practical CDC patterns — polling with status timestamps (simple but wasteful and brittle), triggers/store procedures (low-latency but… | ||
| Delta Lake: Open Source Reliability and Quality for Data Lakes | Michael Paul Armbrust | 31m |
Delta Lake is an open source Parquet storage layer that adds ACID transactions and scalable metadata so Spark can safely mix streaming and batch on the same… | ||
| The Renaissance for Big Data and Parallelism with GraalVM | François Farquet | 30m |
A new benchmark collection targets modern parallel JVM workloads and ships with Renaissance suite (21 benchmarks) that measure synchronized blocks, atomic ops… | ||
| 8 Keys for Successful Serverless Architectures | Ryan Knight | 28m |
Argues that serverless keeps value for security and cost-efficiency but fails because functions are stateless and durable data stores don’t autoscale, causing… | ||
| Moonshot Spark: serverless with GraalVM | Rose Toomey | 35m |
Argues that running Apache Spark serverless is feasible but constrained: GraalVM native images cut cold starts and memory use yet require workarounds for… | ||
| Reliable Machine Learning | David Andrzejewski | 32m |
Machine learning introduces new failure modes across data pipelines, models and serving code, and practical fixes are concrete: assert upstream data with… | ||
| Fast and scalable domain-specific knowledge graphs generation | Omar Alonso | 30m |
Builds three domain KGs: a social KG that ingests the Twitter firehose, extracts users links topics and time, uses pseudo‑relevance feedback and a hashtag… | ||
| Integrating React with Scala... | Niole Nelson | 34m |
Describes a concrete pattern that injects React into a Scala Play monolith by serving Play templates that include script tags pointing to a separate JS server… | ||
| Linkerd and the Service Mesh | Charles Pretzer | 32m |
Linkerd migrates from a JVM/Scala/Finagle/Netty proxy that weighs ~150 MB per sidecar to a Kubernetes-first architecture with a Go control plane (~200 MB… | ||
| Maximizing Throughput and Scalability for Akka Streams | Colin Breck | 32m |
Shows that inserting async boundaries or using mapAsync parallelism turns fused single-actor stages into real parallel work, e.g. CPU-bound spin loops (~10ms)… | ||
| Hack Weekend: ML models on mobile | Erik Reppel | 31m |
Attempts to run GPT-2 inside an iMessage extension over a weekend produce a working prototype only by falling back to server inference because Core ML refuses… | ||
| Machine Learning's Missed Opportunity in Visual Data Management | Vishakha Gupta-Cledat | 27m |
This talk video is a node in devreal.ai, the developer community graph.
It connects to:
Talk: ApertureData: Managing Visual Data for Machine Learning —… | ||
| Swift for TensorFlow: Machine Learning with No Boundaries | Paige Bailey | 39m |
Swift for TensorFlow puts differentiable programming into the language so models differentiate with a small prefix, interoperates with C/C++ and Python, and… | ||
| Apache Flink 2.0: Unified Enterprise Data Processing System and Beyond | Bowen Li | 35m |
Alibabas Singles Day runs on Flink and hits 2 billion events per second, processing ~3 TB/s and supporting 500k+ transactions/s while generating $38B sales in… | ||
| Scaling Financial Automation on TypeBus | Kaoru Kohashigawa, Tabitha Blagdon | 33m |
Tally builds a code-first event platform called TypeBus that runs Scala case classes over akka streams and Kafka, using avro4s to wrap messages in an envelope… | ||
| End-to-End ML Pipelines with KubeFlow and TensorFlow Extended (TFX) | Chris Fregly | 37m |
Live demo of end-to-end pipelines that actually run on a Kubernetes cluster using Docker images, Airflow or Kubeflow as orchestrators, TFX components for TFX… | ||
| Was He Wright All Along? Software After Moore's Law | Bryan Cantrill | 33m |
Argues that the historic gains attributed to Moore’s Law are better explained by Wright’s Law explains cost decline: cost per gate falls with cumulative… | ||
| Lessons Learnt Building Domain Specific NLP Pipelines | Rajesh Muppalla | 32m |
This talk video is a node in devreal.ai, the developer community graph.
It connects to:
Talk: Scale By The Bay 2019: Rajesh Muppalla, Lessons Learnt Building… | ||
| Build Your Own ML Data Feedback Loop | Sophia Sanchez | 27m |
Describes a production loop where model suggestions are shown to doctors whose yes/no feedback becomes ground truth for retraining, with PHI scrubbed and joins… | ||
| Large Scale On-Demand Low-Latency Near Real-Time Predictions | Gabor Melli | 24m |
Sony PlayStation builds an internal relevance-scoring platform that serves personalized recommendations under 20 milliseconds at scale — about 50,000 requests… | ||
| The Art of Asking Questions | Oli Makhasoeva | 32m |
Practical checklist for interviews and team reviews: prepare specific questions about how the company adopts new libraries (Scala 2.13, Dotty, Java 8)… | ||
| Machine Learning and Mobile | Brett Koonce | 27m |
Shows an end to end pipeline that trains a CIFAR convolutional model with Swift for TensorFlow, exports weights via the NumPy Keras bridge to an h5, freezes to… | ||
| Reliable, High Scale Tensorflow Inference.... | Briac Marcatté, Shajan Dasan | 29m |
Twitter deploys a JVM-backed Tensorflow prediction service using Finagle/Thrift and JNI, serving 50+ models at ~20–40M+ requests per second. Engineering… | ||
| Discovering Your Model's Known Unknowns and Unknown Unknowns | Rob Monro | 25m |
Active learning recipes with PyTorch examples show how to find a model’s known unknowns (least confidence, margin, ratio, entropy) and unknown unknowns… | ||
| Introduction to geospatial analysis for uninitiated SQL Data Engineer | Michael Entin | 32m |
Claim: SQL data engineers can use geospatial types and functions with existing workflows because BigQuery geospatial adds geometry/geography columns, ST_… | ||
| From datasets to tables in a multitenant data lake | Thomas Gerber | 26m |
Salesforce moves from immutable dataset blobs to a table abstraction that hides hydration and schema drift, claiming simpler, reproducible access for many… | ||
| Scale By The Bay 2019: Sharanya Santhanam & Shiv Toolsidass | Sharanya Santhanam, Shiv Toolsidass | 20m |
Lyft builds a real-time pipeline using Apache Flink to transform events, Kafka (protobuf) to transport them, and Apache Druid to serve queries, achieving… | ||
| Weld: An Optimizing Runtime for High Performance Data | Shoumik Palkar | 31m |
Weld builds a JIT runtime that stitches IR snippets from different libraries, JIT-compiles with LLVM (and partial PTX GPU support), and optimizes across calls… | ||
| Vectorized Query Processing for CPUs using Apache Arrow | Jacques Nadeau | 32m |
Explains a concrete execution path using Apache Arrow as the in-memory columnar format, JIT-compiling expression trees with Gandiva, and moving data with Arrow… | ||
| Interview | Oli Makhasoeva | 06m |
Implements recursion schemes from scratch by copying signatures from the Scala standard library and iterating to a working open-source library called scheme… | ||
| Interview | Rashmi Shamprasad | 04m |
Senior data engineer describes building a concrete solution to the persistent problem of joining disparate datasets for acquisition and experimentation, used… | ||
| Interview | Richard Whaling | 03m |
Short interview explains why Scala Native is used only as an open source side project rather than at M1 Finance, and how it serves as a teaching tool for… | ||
| Interview | Russell Alexander Spitzer | 05m |
Explains tradeoffs of adding Spark when Cassandra already exists, claims Spark gives analytics at the cost of ops complexity and sometimes simpler solutions… | ||
| Interview | Tabitha Blagdon | 04m |
TypeBus is an event-driven reactive microservices framework built on Kafka with aqua streams and aqua cluster to enable real-time processing for the new… | ||
| Interview | Thomas Gerber | 05m |
Describes replacing ad hoc dataset blobs with a table abstraction inside a multitenant data lake so ML pipelines can get consistent, point‑in‑time features… | ||
| interview | Umayah Abdennabi | 04m |
Overview of an internal data analytics platform that already ingests user event data and generates marketing reports using Spark, with a concrete claim that… | ||
| Interview | Yifan Xing | 04m |
A community organizer describes a nonprofit that teaches underrepresented people Scala for free through one-day workshops and a 12-week program, and asks for… | ||
| Interview | Paul Cleary | 04m |
Describes moving from decades of OO work to functional programming in Scala, documenting concrete changes to architecture and developer habits while adapting… | ||
| Interview | Adelbert Chang | 06m |
Claims that core Internet-era system designs from 30–40 years ago still apply to modern infra, and describes practical work building deployment infrastructure… | ||
| Interview | Ajit Koti | 03m |
This talk video is a node in devreal.ai, the developer community graph.
It connects to:
Talk: Scale By The Bay 2019: Rashmi Shamprasad & Ajit Koti, Finding… | ||
| Interview | Alexander Ioffe | 10m |
Works at Nasdaq managing data flow and big SQL pipelines and describes building an integration layer between two frameworks in a single collaborative session… | ||
| Interview | Chris Fregly | 05m |
Joined AWS as Technical Evangelist focused on machine learning and AI and describes creating scalable, open-source content from a developer-relations role that… | ||
| Interview | Erik Repple | 05m |
Machine learning platform work at Coinbase focuses on shipping models to phones for on-device fraud detection and reducing hosted compute costs. The talk… | ||
| Interview | Heather Miller | 04m |
Research on distributed runtimes and programming languages aims to make microservice development feel like one language even when teams use 20 different… | ||
| Interview | Jacques Nadeau | 06m |
Describes how Apache Arrow couples transport and in-memory processing to cut serialization/deserialization overhead, and how Dremio uses Arrow as its linchpin… | ||
| Interview | James Ward | 04m |
Argues that a serverless platform can act as the runtime environment for effectful Scala code, letting an environmental effects model (notably by functor IO /… | ||
| Interview | Jason Swartz | 06m |
Argues that running vanilla Scala compiled to GraalVM native Linux executables on AWS Lambda yields near-native latency, with cold boots around 200 ms and… | ||
| Interview | Justin Kaeser | 04m |
Works on the IntelliJ Scala tooling and describes concrete efforts to make Scala tooling integrate with other tools by improving Build Server Protocol support… | ||
| Interview | Kaoru Kohashigawa | 06m |
At Tally the platform team builds a framework that uses TypeBus to spin up services quickly and is migrating monorepo from SBT to Basil to cut build thrash and… | ||
| Interview | Justin Heyes-Jones | 02m |
Consulting firm YoppWorks offers reactive, pure functional solutions and this interview claims comonads can be taught simply and used to scale program… | ||
| Interview | Nick Pentreath | 05m |
A live demo assembles streaming components, scales them on Kubernetes, and plugs an open‑source ML model into an end‑to‑end pipeline that actually runs… | ||
| Interview | Josh Suereth | 03m |
Staff software engineer at Google working on Assistant argues that services must run in many environments, not just one production deployment, and draws… | ||
| Interview | Oscar Boykin | 05m |
A Scala library that scores ONNX models on the JVM and integrates with existing Java/Scala services, built and deployed at Stripe, lets teams train in Python… | ||
| Interview | Joe Beda | 05m |
Argues that Kubernetes as a platform platform is an actual engineering claim, not marketing: Kubernetes becomes a control plane for distributed systems, not… | ||
| Interview | Ville Tuulos | 03m |
Describes building machine learning infrastructure that prioritizes human-centric usability for people operating recommendation and content workflows, not just… | ||
| Interview | Michael Paul Armbrust | 04m |
Claims that the metadata problem for data lakes is converted into a data problem and handled with Spark so systems can process metadata in parallel, enabling… | ||
| Interview | Tikhon Jelvis | 04m |
Work at Target's supply chain AI group uses Haskell (and past projects in Racket and OCaml) to build event-driven simulations and optimization services, and an… | ||
| Interview | Briac Marcatté | 02m |
Describes building reliable TensorFlow inference pipelines with concrete tooling: automated performance tests, memory-profiling and root-cause techniques that… | ||
| Interview | Shajan Dasan | 03m |
A practitioner with a distributed systems background describes applying distributed systems principles to machine learning infrastructure at Twitter and claims… | ||
| Next-generation frameworks for Large-scale Machine Learning | Anima Anandkuma | 28m |
Argues that keeping multidimensional structure yields practical gains: tensor layers (tensor-train and convolutional tensor-train) compress fully connected and… | ||
| Human-Centric ML Infrastructure at Netflix | Savin Goyal, Ville Tuulos | 30m |
Metaflow packages each run’s Python code, Conda environment and immutable data snapshots so executions are reproducible across local laptop or cloud; it… | ||
| Reasoning about types and code | Sergei Winitzki | 23m |
A practical roadmap for reasoning about types and code: derive when a function is implementable (example: two signatures where F impossible, G implementable)… | ||
| What did category theory do for us? | Sergei Winitzki | 42m |
Argues that category theory explains why common typeclass methods are "liftings" and why their behavior reduces to two laws, identity and composition laws, so… | ||
| Intro and Overview to Rust | Sara Moore | 25m |
Covers Rust basics with concrete examples: mutability vs default immutability, explicit integer types like usize when indexing Vec, formatting with {:?} debug… | ||
| What Not to do When Pitching a Book | Jess Haberman | 13m |
Stop before filing a proposal: confirm the right acquisitions editor and prove demand with data — Google Trends, conference presentation counts, YouTube view… | ||
| Highlights | 03m | |
Multi-day conference (Nov 13–15) organized around applied ML, data and cloud engineering with an LLM workshop and distinct tracks for data science, software… | ||
| nothing matches | ||