How to Combine Data from Different DBMSs and Do It Efficiently
summary
Shows how to join data stored in different database systems and file stores (for example, relational databases and CSV files on S3) efficiently. Covers practical approaches and tools for performing distributed data joins and analysis across heterogeneous storage backends.
AI-written summary · about
chapters (24)
- 0:00Introduction to federated queries
- 2:13Imperative vs declarative data processing
- 3:38Federated systems overview
- 5:46Key scenarios for federated queries
- 7:53Data model and interface diversity
- 10:45Schema strictness and flexibility
- 13:18Type system challenges in federated systems
- 15:01Federated query processing basics
- 16:50Connector layer architecture
- 19:24Built-in vs external connectors
- 21:33Connector implementation in popular systems
- 26:18Push-down of operations to external sources
- 31:39Open challenges in federated systems
- 32:00Choosing a federated system
- 33:03Available federated solutions
- 33:48Q&A: Transactions and consistency
- 36:20Q&A: Fault tolerance in federated systems
- 37:23Q&A: Federation at Graco level
- 38:29Q&A: Neural networks and federated queries
- 39:56Q&A: Data inconsistency profiling
- 41:28Q&A: Query optimization with statistics
- 42:32Q&A: Handling large intermediate results
- 44:40Q&A: Security in federated systems
- 46:50Q&A: On-demand vs pre-replicated data
related talks
34mJoining Heterogeneous Databases is a reality, not a Myth
39mHigh-volume Data Streaming and Storage
47mFun with BIG Data with various databases
58mUnderstanding Big Data for Software Engineers
50mSQL 2022 and S3 – Dell’s overview of consuming object data via Microsoft SQL
45mArchitecture and Tools for Selected Data Integration Cases (Part 1)
25mHybrid Applications with MongoDB and RDBMS
39mPostgreSQL: Joining Heterogeneous Databases is a reality, not a Myth