Straightforward stream processing with CSP

on YouTube ↗39m

" summary

CSP is an open‑source stream processing library with a C++ engine and Python API that aims to keep research and deployment identical: the same graph runs in real time or as replayed historical data, enabling replay years of data without changing logic. It uses mandatory static typing, offers Parquet and Kafka adapters (fewer than Flink), includes a profiler with cycle times and memory snapshots, supports Perspective for live visualization (earthquake and MTA demos), handles medium‑frequency market feeds at Point72, and limits parallelism to disjoint graphs rather than per‑node threads.

AI-written summary · about

" chapters (14)

1. 0:00CSP and stream processing intro
2. 3:27Developer experience defined
3. 5:56Static vs dynamic data
4. 10:30CSP in data toolbox
5. 13:41CSP development cycle
6. 16:31Building CSP graph
7. 21:47Composable subgraphs
8. 24:54Enhancing developer experience with CSP
9. 25:36Research and backtesting
10. 27:42Debugging pipelines
11. 29:50Performance optimization
12. 31:16Jupyter interactive execution
13. 32:18CSP applications summary
14. 32:43Q&A on CSP

" related talks

• 32mEnabling real-time insights through stream processing in PythonAdam Glustein · PyData London 2024
• 51mSimplifying Event-Driven Services with FastStream: A How-To GuideTvrtko Sternak · DSC Europe 23
• 27mReal-time Event Streaming with PythonDave Klein · PyCon Italia 2023
• 20mStreaming DataFrames: A New Way to Process Streaming DataTomas Neubauer · Berlin Buzzwords 2024
• 29mUnified batch and stream processing in pythonArthur Andres · PyData Global 2023
• 36mKafka Streaming, the Pythonic WayArthur Andres · PyData London 2026
• 41mTalk Python To Me: Stream Processing in your favorite language with Beam on FlinkAljoscha Krettek · Flink Forward Berlin 2017
• 22mPython for bridging between researchers and service operators: from CFFI to JupyterhubKevin Plastow · PyCon AU 2016 Science and Data Miniconf