MetaConfig driven FeatureStore

on YouTube ↗51m

" summary

Describes building a feature store and metadata-driven standardization (MetaConfigs and a FeatureCatalog) to eliminate dataset fragmentation and reduce data-tech debt for MakeMyTrip’s personalization ML. Explains their implementation for both online (Spark Streaming + Kafka, RocksDB, Redis) and historical/batch (Spark SQL, Hive, Presto, S3/EMR/Glue) feature generation and serving, plus Akka HTTP data APIs and tools used (Scala, Python, Cassandra, Druid, AWS).

AI-written summary · about

" related talks

• 20mNavigating the Skies of Data: A Flight Through Feature StoresDave Ruijter · SQLBits 2024
• 30mHow Feature Stores Enable Operational MLMike Del Balso · TWIMLcon: AI Platforms 2021
• 08mFeature Store Usage Patterns - From a Single Data Scientist to an EnterpriseSimba Khadder · Feature Store Summit 2022
• 35mPanel 03- How to make the Feature Store become a hidden part of the productionalization of ML modelsAchint Thomas · Feature Store Summit 2022
• 15mFeature Stores: Powering Machine Learning Infrastructure with Apache FlinkSimba Khadder · Flink Forward Global Virtual 2020
• 32mA feature Store for Machine LearningJim Dowling · PyData Fest Amsterdam 2020
• 33mFeathr, an Open Source, Battletested, Scalable and Enterprise-Grade Feature StoreHangfei Lin · Kubernetes AI Day North America 2022
• 30mAmazon SageMaker Feature Store: Store, discover, & share features for ML appsVidhi Kastuar · AWS re:Invent 2020