Prompt Tweaking Engineering - A Case Study in Test Design
summary
Shows that raw LLM prompting produces about 10–11 pairwise tests while a deterministic Python library returns 22 test cases, proving LLMs struggle with the math but excel at parsing freeform input. Demonstrates a working pipeline: extract variables via GPT, call an allpair/AllPairs library through a custom GPT + tool wrapper, emit CSV/Excel exports and human-in-the-loop review. The hybrid approach works; pure prompting alone fails.
AI-written summary · about
chapters (8)
- 0:00Introduction and context
- 1:56LLMs and test design challenges
- 3:44Pairwise testing fundamentals
- 6:13Initial LLM attempt failure
- 10:06Prompt tweaking and limitations
- 12:39Custom GPT integration solution
- 14:25Advanced automation with APIs
- 16:53Community and future directions
related talks
13mPrompt Engineering Simplified
46mBeyond Basic Prompting: Supercharging Open Source LLMs with LMQL's Structured Generation
16mPrompt Engineering for Testers: Types and Applications
49mPrompt Darwinism: Evolutionary Engineering in the Age of LLMs
59mTutorial: How to Win Friends & Influence LLMs (with Prompt Engineering)
19mTesting LLM-Powered Applications
38mPrompt Engineering ⚙️ - Addressing the sensitivity of Large language models
01h 39mMastering LLMs: From Prompt Engineering to Agentic AI