Evaluation Strategies for HCI Toolkit Research
summary
Paper analyzes 68 papers (50 from ACM) and claims HCI toolkits need clearer evaluation, categorizing methods into four evaluation types: demonstration, usage, performance, and heuristics. Examples include WorldKit, Prefab, Pinocchio, RetroFab, Paper Maché and D3. Demonstrations appear in almost every paper while usage and performance appear in roughly half or a third and often serve as small add‑ons. The study shows adoption (D3) is the hardest-to-measure success signal and recommends choosing the evaluation that directly tests the paper’s concrete claim.
AI-written summary · about