tldr: A test case is a single verification: one input, one expected outcome. A test suite is a collection of test cases, often grouped by feature, area, or test type. The right hierarchy keeps a 1,000-test suite navigable.
Test suite vs test case: definitions
Test case. The smallest meaningful test unit. Has a setup, a single execution, an assertion (or small set of assertions verifying one behavior), and a teardown.
Test suite. A grouping of test cases. Suites usually correspond to a feature, a module, a test type, or a release.
In code, a test suite is often a file or a directory. A test case is a function or method within it.
How to organize the hierarchy
Two levels of grouping work for most teams.
Suite by feature or module
tests/checkout/, tests/auth/, tests/dashboard/. Each suite covers one product area.
This works well for product-focused teams. Tests are easy to find when working on a feature.
Suite by test type
tests/unit/, tests/integration/, tests/e2e/. Within each, sub-suites by feature.
This works well for engineering-focused teams. The CI pipeline maps cleanly: unit tests run on every commit, E2E on every merge.
Most teams combine both. tests/e2e/checkout/ is a typical structure.
What makes a good test case
Five properties.
Single responsibility. One behavior tested. Multiple assertions are fine if they verify the same outcome.
Independent. Does not rely on other test cases running first.
Deterministic. Same input produces same result. No random data unless seeded.
Fast. As fast as possible while still being meaningful.
Clear name. "test_login_with_valid_credentials_redirects_to_dashboard" beats "test_1."
A test failing any of these will eventually cause maintenance pain.
What makes a good test suite
Three properties.
Cohesive. All tests in the suite cover related functionality.
Reasonably sized. 5 to 50 test cases per suite. Above that, split.
Stable. The pass rate of the suite is consistent over time. A suite that swings between 100% and 80% green is not actually testing anything.
When to split a suite
A test suite should split when:
- It exceeds 50 to 100 test cases.
- It mixes test types (unit and integration in one suite).
- It has multiple distinct setup patterns.
- Different parts have different stability or speed characteristics.
Splitting late is harder than splitting early. Be willing to split as soon as one of these triggers.
How AI testing fits
AI testing platforms collapse the test case definition. Where a traditional test case has setup, steps, and assertions, an AI test case has a goal in plain language.
Bug0 tests are organized as goals grouped into suites by feature or flow. The grouping logic is similar; the per-test cost is much lower.
FAQs
How long should a test case be?
A few lines for unit tests. 10 to 30 lines for integration. 30 to 100 lines for E2E. Anything longer probably should be split.
How many test cases per feature?
Depends on complexity. Simple features: 5 to 15. Complex features: 30 to 100.
Can a single test case have multiple assertions?
Yes, when they verify the same outcome from different angles. "After login, the URL changes AND the dashboard renders AND the welcome message contains the user's name" is one outcome with three assertions.
How does Bug0 organize tests?
Bug0 groups tests into projects and suites by feature or flow. Each test is a goal in plain language. The hierarchy mirrors how QA teams already think about coverage.