Federated Queries with HAWQ - SQL on Hadoop and Beyond
summary
Covers integration patterns between Pivotal HAWQ (a scalable OLAP SQL-on-Hadoop) and GemFire (an in-memory OLTP-like data grid), showing concrete approaches for exchanging data between them. Describes using Spring XD to ingest GemFire data into HDFS and using Spring Boot to build a RESTful proxy for HAWQ Web Table integration, with implementation details and practical experiences.
AI-written summary · about
chapters (25)
- 0:00Intro
- 0:17SPRINGONE2GX WASHINGTON, DC
- 1:14Contents
- 9:17Data System Principles
- 13:11Data System - Key Properties
- 14:19Data System Architectural Patterns How to build Data Systems?
- 15:31Data Lake (From ETL to ELT) Single Store For All Data
- 17:05Lambda Architecture
- 22:00Tachyon
- 26:03Apache Geode & GemFire (OLTP) • Cache. Performance / Consistency / Resiliency
- 26:46Geode use cases...
- 27:23HAWQ (SQL-ON-HADOOP)
- 31:33Test Environment - Ambari Management
- 34:26Passive Synchronization (Case 1) - Demo Zeppelin
- 34:43Passive Synchronization (Case 2) - gpfdist
- 35:43Passive Synchronization (Case 2): Demo
- 47:40HAWQ Web Table - Architecture
- 51:16HAWQ PXF/Geode External Table
- 51:44Pivotal extension Framework (PXF) • Java-Based
- 52:49PXF Architecture
- 56:11PXF Processors
- 56:48PXF Runtime Model
- 57:04PXF External Tables Definition
- 57:45Available PXF Plugins
- 59:57PXF Geode Profile
related talks
44mHAWQ Architecture
44mHAWQ Architecture
37mWhen Distributed Database Meets Cloud - Lessons Learned
01h 25mGetting started with Spring Data and Distributed Database Grids
59mHadoop Essentials for the SQL Server Professional
01h 13mHow We Built SQL in Hazelcast
31mBuilding large scale, transactional data lakes with Apache Hudi
26mMulti-Cloud Native Data with Spring and VMware Data Solutions