Assurance of Learning Through Testing
summary
Claim: automated marking can detect shortcuts and ensure learning by running students' programs, not just diffing outputs. Uses p expect to spawn and feed random inputs repeatedly, unitest.mock monkeypatches to assert use of super, and AST checks to find literal dicts; also inspects imports with modulefinder. Deploys as an arbitrary marking script on Ed/edstem, tolerates Monte Carlo variance with regex matching, and flags hard‑coding by sampling many cases.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 1:05Assurance of learning
- 2:30Diff-based testing limits
- 4:56Student vs teacher paradigm
- 8:25Pexpect for dynamic testing
- 13:46Complex interactive programs
- 14:07OOP and literals techniques
- 18:05Illegal imports detection
- 19:30Recommendations and trade-offs
- 23:22Q&A
related talks
42mDo Your Homework! Writing tests for Data Science and Stochastic Code
01h 05mMicrotesting
30mApplying AI to testing
59m.NET Testing Techniques You Didn’t Know You Needed
18mPreview: Verification (Correctness & Debugging)
31mCan You Trust Your Tests?
51mAutomating to augment testing
41mMutation testing in practice