Architecting a Data Privacy Platform to Scale to Billions of Identities

on YouTube ↗13m

" summary

Manetu Consumer Privacy Management (CPM) is a cloud service built as microservices on Kubernetes with CQRS personalities (gateway, command, query, event processors), Kafka-partitioned command topics and Yugabyte & Kafka transactions to get total order and exactly-once command execution. Persistence mixes Yugabyte for indexed reads, S3 for objects, and per-relationship encrypted triple-store vaults keyed separately, exposed via RDF/SPARQL. The system autos-scales command/query pods and storage to claim support for billions of identities, tens of billions of vaults and hundreds of billions of triples.

AI-written summary · about

" chapters (7)

1. 0:00Introduction
2. 1:00DNA
3. 3:00Microservice Architecture
4. 4:48CQRS
5. 8:37Event Processing
6. 10:00Attribute Service
7. 11:44Summary

" related talks

• 24mScaling to Billions of Internet Observations per DayHudson Clark, Ethan Uberseder · Distributed SQL Summit 2020
• 25mProtecting Your Data in Yugabyte Cloud with HashiCorpGrant Orchard · Distributed SQL Summit Asia 2022
• 35mImplementing A User-Centric DatastorePaul Malone · OWASP AppSec Research (AppSecEU) 2015
• 47mData Mesh in KubernetesAndrii Syniuk · Software Architecture fwdays'23
• 18mData Modernization Using Confluent and YugabyteAnanda Bose · Distributed SQL Summit Asia 2022
• 51mImmutable data stores and CORS for safety, flexibility and profitSidney Shek · Scala Days Berlin 2016
• 17mBuilding a Scalable and Performant SaaS Platform with YugabyteDBMANOJ MATHEW · Distributed SQL Summit Asia 2023
• 30mFrom monolith to petabyte scale: Built a cloud-native, AI-ready data layerSanjeev Gautam · Distributed SQL Summit Mumbai 2026