Advanced Pickling with Stackless Python and sPickle
summary
Demonstrates using Stackless Python plus sPickle to serialize running tasklets, implement checkpoint/restart and send live computation over SSH; includes a working checkpoint demo that restarts mid-loop and remote-call examples returning results or exceptions. Implementation uses backtracking to reconstruct module/dict relationships, special-case handling for native types and trace functions, and reports restart of long-lived tasks in under a second. Proof‑of‑concept only: requires Stackless 2.7, incomplete coverage and not production‑ready.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 1:11What is pickling
- 2:36Security risks of unpickling
- 4:26Stackless Python advantages
- 8:47sPickle introduction
- 11:19Advanced pickling applications
- 15:17Checkpointing with sPickle
- 18:22Remote computation with sPickle
- 26:19sPickle implementation
- 33:08Module collisions in unpickling
- 36:06Class objects challenges
- 43:09Development plans
- 45:29Community contributions
- 50:14Performance and Q&A
related talks
30mPython Serialization and the Hallows of Parallel Computing
30mParallel computing in Python: Current state and recent advances
51mMarshalling Pickles
36mDeserializing Python objects in Go with GoPickle
25mPython as a Hackable Language for Interactive Data Science
24mEngage the Hyper-Python
20mCustom Python Applications in Neuroscience
40mPyData Stack: Pure Python open source data platforms