Make Sure Your Tests Fail
summary
Postman tests often pass when they should fail because of concrete mistakes: closing the pm.test call before supplying the assertion function or misusing the shorthand pm.expect leads to silent green results, missing environment variables plus the wrong property produces undefined equals undefined, and running collections with newman can still return exit code zero even when tests log failures. Remedies include using code snippets, switching to test first workflows, mocking endpoints with Postman mock servers, validating schemas with Ajv, and calling pm.fail to force failures.
AI-written summary · about
chapters (17)
- 0:00Introduction to API testing
- 1:22The importance of failing
- 4:25When tests don't provide value
- 5:39Identifying false positives
- 7:43Shandhand syntax pitfalls
- 8:52Unexpected test results
- 10:20Environment variable issues
- 13:05Risks of automated schemas
- 16:11CI/CD pipeline blind spots
- 17:32Exit code issues in Newman
- 18:42Mitigation strategies
- 20:10Validating test integrity
- 22:03Implementing TDD
- 23:11Using mock servers
- 24:46Testing the pipeline
- 26:47Feature requests for Postman
- 27:53Q&A session
related talks
18mBreaking the Mold Innovative Approaches to Testing GraphQL APIs
40mAutomated API Tests with Postman
21mTests That Help you Find Defects Faster
17mAPI Testing Unlocked
01h 22mWorkshop: How to Create a Test Data-Driven Framework with Postman
01mEntrevista a Celia Martinez
21mDon't just test, my friend, test better
14mIntegrate Postman with Generative AI for Effective API Testing