Evidence · 04
What's your testing philosophy?
~24 automated pytest suites on ConfirmHai, kept green before any commit lands — organized around what breaks silently, not just what's easy to test.
| Suite category | What it verifies | Why it exists |
|---|---|---|
| auth | Registration, OTP issuance/verification, JWT issuance and expiry | Prevents silent regressions in login flows as AI-directed changes touch shared auth code |
| authz / IDOR | Deliberate cross-account access attempts against every resource route | Authorization bugs rarely show up in happy-path testing — they show up when you try to access someone else's data on purpose |
| confidence engine | Score computation across freshness/vacancy signal combinations, WhatsApp reply override | The scoring engine is the product's moat — a silent regression here is a trust regression for real users |
| rate limiting | OTP attempt and TTL enforcement under repeated requests | Confirms brute-force protection actually triggers at runtime, not just that a config value exists |
| security headers | Required response headers present on every route | Security headers regress silently when middleware order or route registration changes |
Philosophy
Test for what an AI-directed change is most likely to break silently
Happy-path tests catch obvious regressions. They don't catch an authorization check that quietly stopped firing, or a rate limit that silently no-oped after a refactor. Directing AI coding tools across a shared codebase makes that failure mode more likely, not less — so the suite is weighted toward the tests that fail loudly when a security or business invariant breaks, with a hard gate: nothing merges while red.
Full context in the ConfirmHai case study → · The operating model behind this →