Splunk as a Big Data Platform for Developers

on YouTube ↗01h 33m

" summary

Splunk claims to index heterogeneous, unstructured machine data and give realtime operational intelligence and the talk shows concrete tooling and results: examples use the Splunk Java SDK to stream events over HTTP/TCP, run real-time search windows, and avoid writing MapReduce, a Node.js + JavaScript SDK + D3 demo builds a live Twitter dashboard that works in real time, Spring shows Spring Integration adapters inbound/outbound, free dev license is 500 MB/day and clusters scale to hundreds of terabytes per day.

AI-written summary · about

" chapters (37)

1. 0:00Intro
2. 0:37About me
3. 2:59So What is Splunk, Exactly?
4. 4:25Indexes any Machine Data
5. 6:04Centralizes Data Across the Environment
6. 7:10Provides Strong Machine Data Governance
7. 8:02Splunk and Apache Hadoop MR/HDFS
8. 12:03Getting Data into Splunk Agent and Agent-less Approach for Flexibility.
9. 12:59Universal Data Forwarder Forward data without negatively impacting production performance
10. 15:05Send Data to Other Systems Route raw data in real time or send alerts based on searches
11. 15:28High Availability / DR Combine auto load balancing and data replication
12. 17:02Integrate External Data Extend search with lookups to extemal data sources
13. 18:31Integrate Users and Roles Integrate authentication with LDAP and Active Directory
14. 18:52Centralized Licensing Management Groups, Stacks, and Pools for Enterprise Deployments.
15. 19:36Real-time Search
16. 20:07Real-time Alerting
17. 20:34New Approach to Heterogeneous Data
18. 22:56Inside Universal Indexing
19. 23:37Inside Search-time Knowledge Extraction
20. 25:43Splunk & Developers
21. 27:48Splunk in the Developer Community
22. 28:57How does Splunk Accelerate Dev/Test?
23. 31:21Integration into existing IT tools
24. 32:06Splunk REST API
25. 32:58Developer Platform SDKS
26. 37:18Realtime Twitter Visualization Demo
27. 42:55Get the Java SDK
28. 44:15Java SDK Class Model
29. 45:27Key Java SDK Use cases
30. 46:21Connect and Authenticate
31. 47:59Input Events
32. 54:09Blocking Searches
33. 55:26Non Blocking Search
34. 57:20Realtime Search
35. 58:07Alternate JVM Languages
36. 58:59Groovy
37. 59:26Scala

" related talks

• 38mKeynote: The Evolution of Data Infrastructure at SplunkEric Sammer · Flink Forward Virtual 2020
• 27mRealise the value of data with SplunkTrav Kane · AWS Summit Sydney 2025
• 01h 20mSupercharging Operations and Analytics: Using Spring XD to Support Analytics and CEPJoseph Paulchell · SpringOne2GX 2015
• 01h 35mDeep Dive into the Big Data LandscapeDean Wampler, Eva Andreasson · GOTO Copenhagen 2014
• 20mData lake deep dive: Using accelerated indexing for threat detectionColin Gibbens, Javier Santoyo · AWS re:Inforce 2024
• 29mRealtime Analytics Unleashed: Unlocking the Potential of Data at ScaleBarkha Herman · Shift Miami 2023
• 01h 37mGaining Application Lifecycle IntelligenceDamien Dallimore · SpringOne2GX 2014
• 21mSplunk Data Stream ProcessorJamie Grier · Flink Forward Global Virtual 2020