Skip to content

Commit 0dc0a55

Browse files
author
Jintao Ling
committed
init version2.0
2 parents 602bb30 + a4acf56 commit 0dc0a55

82 files changed

Lines changed: 17757 additions & 180 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/ISSUE_TEMPLATE.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
## What
2+
3+
## Why
4+
5+
## Proposal
6+

‎.github/PULL_REQUEST_TEMPLATE.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
## Summary
2+
3+
## Coverage Impact
4+
5+
## Notes
6+

‎.github/workflows/link-check.yml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
name: Link Check
2+
on:
3+
push:
4+
pull_request:
5+
jobs:
6+
link-check:
7+
runs-on: ubuntu-latest
8+
steps:
9+
- uses: actions/checkout@v4
10+
- uses: lycheeverse/lychee-action@v1
11+
with:
12+
args: --verbose --no-progress --exclude-mail './**/*.md'
13+

‎.github/workflows/markdown-lint.yml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
name: Markdown Lint
2+
on:
3+
push:
4+
pull_request:
5+
jobs:
6+
markdownlint:
7+
runs-on: ubuntu-latest
8+
steps:
9+
- uses: actions/checkout@v4
10+
- uses: DavidAnson/markdownlint-cli2-action@v16
11+
with:
12+
globs: |
13+
**/*.md
14+

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
**/.*.md
2+
!/.*.md

‎00-start-here/glossary.md‎

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
# Glossary (Quick, Practical)
2+
3+
> Short, practitioner-friendly definitions you can use during design reviews and when writing cases.
4+
> Tip: skim this once; link back when terms pop up in playbooks or checklists.
5+
6+
---
7+
8+
## Notation & Abbreviations
9+
- **MAE** — Main / Alternative / Exception (scenario families).
10+
- **MECE** — Mutually Exclusive, Collectively Exhaustive (no overlap, no gaps).
11+
- **SUT** — System Under Test.
12+
- **I/O** — Input / Output.
13+
- **RACI** — Responsible, Accountable, Consulted, Informed (ownership).
14+
15+
---
16+
17+
## Core Test-Design Concepts
18+
- **Coverage thinking** — Structuring the space of behaviors so tests are *representative* and *sufficient* (not every permutation).
19+
- **Equivalence classes** — Group inputs that should behave the same; test one per group + boundaries.
20+
- **Boundary value analysis** — Hit “just in / just out” values around each limit (e.g., −1, 0, 1).
21+
- **Decision table** — Tabular rules: conditions × actions. Ensures no conflicting or missing rules.
22+
- **State model** — Valid states + transitions + guards; tests verify allowed/blocked moves and side-effects.
23+
- **Pairwise testing** — Cover all pairs across factors; reduces explosion while catching many interaction bugs.
24+
- **CRUD grid** — For each resource: Create/Read/Update/Delete × roles × constraints.
25+
- **Oracle (test oracle)** — Mechanism to decide pass/fail (calc, invariant, golden file, service contract).
26+
- **Traceability** — Links across Requirement ↔ Scenario ↔ Case ↔ Evidence; often a matrix (RTM).
27+
- **Risk-based prioritization** — Test higher business/technical risk first (impact × likelihood).
28+
29+
---
30+
31+
## Scenario & Workflow
32+
- **Main/Alt/Exception flows (MAE)** — Happy path, common variations, and error/edge paths.
33+
- **Invariants** — Conditions that must always hold (e.g., sum of line items = order total).
34+
- **Idempotent action** — Safe to repeat: same key/request ⇒ same outcome (e.g., retry POST with idempotency key).
35+
- **Race condition** — Outcome depends on timing/order of concurrent events.
36+
- **Idempotent consumer** — Consumer deduplicates replays, guaranteeing single-effect processing.
37+
38+
---
39+
40+
## API & Data Contracts
41+
- **Contract test** — Verify provider/consumer agree on schema, types, required fields, and error shapes.
42+
- **Schema validation** — Enforce structure (e.g., JSON Schema/OpenAPI) before deeper checks.
43+
- **Pagination (offset vs cursor)** — Offset is page index; cursor is opaque pointer for stable traversal.
44+
- **Consistency guarantees** — Strong / Read-after-write / Eventual; dictates assertion timing.
45+
- **Idempotency key** — Client-generated token to dedupe retried operations.
46+
- **Error taxonomy** — Canonical set of errors (auth, validation, rate-limit, transient) mapped to UX messages.
47+
- **HTTP semantics** — Safe (GET), idempotent (PUT/DELETE), unsafe (POST) per RFC intent.
48+
- **ETag / If-Match / If-None-Match** — Conditional requests for concurrency control & caching.
49+
- **Pagination invariants** — Sorting stability or cursor monotonicity to avoid skips/dupes across pages.
50+
51+
---
52+
53+
## Non-Functional (Perf, Resilience, Compatibility, A11y)
54+
- **p95 / p99** — 95th/99th percentile latency; budgets set expectations beyond average.
55+
- **TTFB / TTI** — Time to First Byte / Time to Interactive (web UX timing signals).
56+
- **SLO / SLI / SLA** — Objective / Indicator / Agreement for reliability commitments.
57+
- **Budget** — Agreed threshold (e.g., p95 ≤ 500 ms) used as an acceptance criterion.
58+
- **Retry w/ backoff** — Reattempt with increasing delay; add jitter to avoid thundering herds.
59+
- **Circuit breaker** — Stop calling a failing dependency until it recovers to protect the system.
60+
- **Graceful degradation** — Reduced functionality with clear UX when dependencies fail.
61+
- **Compatibility matrix** — Supported OS/Browser/Device/App versions and features.
62+
- **Internationalization (i18n) / Localization (l10n)** — Build for multiple locales / adapt strings, formats, rules.
63+
64+
---
65+
66+
## Observability & Evidence
67+
- **Structured logs** — Key-value logs parsable by machines (include correlation/trace IDs).
68+
- **Metrics** — Numeric time-series (counters, gauges, histograms) for SLOs and budgets.
69+
- **Tracing** — End-to-end spans for a request; proves where time/errors occur.
70+
- **Correlation ID** — ID that ties logs/metrics/traces for a single workflow.
71+
- **Evidence artifact** — Saved proof: log snippet, screenshot, export, API transcript validating the expected result.
72+
73+
---
74+
75+
## Security & Privacy
76+
- **Authentication vs Authorization** — Who you are vs what you may do.
77+
- **Least privilege** — Grant minimal permissions needed for a task.
78+
- **Input validation & encoding** — Validate on trust boundary; encode at sink (HTML/SQL/OS).
79+
- **Secret handling** — No secrets in logs; use vaults/kms; rotate regularly.
80+
- **PII / Data minimization** — Store the least personal data necessary; mask in lower environments.
81+
- **Audit trail** — Tamper-evident record of who did what, when, and why.
82+
83+
---
84+
85+
## Accessibility (WCAG)
86+
- **POUR** — Perceivable, Operable, Understandable, Robust (WCAG principles).
87+
- **Keyboard accessibility** — Full operation without a mouse (focus order, visible focus).
88+
- **ARIA semantics** — Roles/states to aid assistive tech (use sparingly and correctly).
89+
- **Color contrast** — Sufficient luminance difference for text/UI elements.
90+
- **Alt text & labels** — Programmatic names for images, inputs, controls.
91+
92+
---
93+
94+
## AI-Augmented Test Design (Optional)
95+
- **Grounding (explicit-only)** — Use *only* provided requirements/artifacts as sources; no outside guessing.
96+
- **Schema discipline** — Outputs adhere to a specified JSON/CSV/Markdown schema.
97+
- **Deduplication** — Avoid semantic or structural duplicates in generated scenarios/cases.
98+
- **Human-in-the-loop** — Reviewer notes → regenerate loop to refine outputs.
99+
- **Hallucination** — Fabricated content not present in sources; must be prevented by guardrails.
100+
101+
---
102+
103+
## Review Gates & Quality Signals
104+
- **Gate** — A checklist or automated criterion that must pass before merging/releasing.
105+
- **Defect leakage** — Escaped defects per release; track ↓ as design quality ↑.
106+
- **Case effectiveness** — Share of defects caught by designed tests vs ad-hoc discovery.
107+
- **Coverage diff** — Change in scenario/case coverage between versions (what’s new or removed).
108+
109+
---
110+
111+
## Tiny Examples (copy-pasteable)
112+
113+
- **Boundary set (inclusive 0–100):** `-1, 0, 1, 99, 100, 101`
114+
- **Idempotency test:** Send POST `/payments` with `Idempotency-Key: K1` twice → **same** result; with `K2` → new payment.
115+
- **Retry/backoff:** `100ms, 200ms, 400ms (+ jitter)`; abort after N attempts or on non-retryable errors.
116+
- **Error taxonomy → UX:** `VALIDATION_ERROR.amount.invalid` ⇒ “Enter a whole number between 0 and 10,000.”
117+
118+
---
119+
120+
## See Also
121+
- `10-fundamentals/*` for deeper theory
122+
- `20-techniques/*` for playbooks
123+
- `60-checklists/*` for gates
124+
- `65-review-gates-metrics-traceability/*` for metrics & traceability

‎00-start-here/quickstart.md‎

Lines changed: 152 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,152 @@
1+
# Quickstart
2+
3+
Welcome! This page gets you productive **today**—with time‑boxed tracks, role shortcuts, and a minimal repo setup. Pick one track, follow the steps, and ship a small improvement you can share.
4+
5+
---
6+
7+
## Time‑boxed tracks
8+
9+
### ⏱️ 10 minutes — “One technique, one feature”
10+
**Goal:** apply a single technique on a real feature and produce 6–12 checkable cases.
11+
12+
1. Open **[`20-techniques/`](../20-techniques/)** and pick **one** (e.g., _Boundary & Equivalence_).
13+
2. Choose a small, real target (e.g., “discount code” field).
14+
3. Follow the playbook steps and write **explicit expected results**.
15+
4. Run the **mini checklist** at the end of the playbook.
16+
5. Save as `playground/<feature>/<technique>.md` (or add to your team notes).
17+
18+
**Deliverable:** a short Markdown file with the chosen technique, value sets, and expected results.
19+
20+
---
21+
22+
### ⌛ 60 minutes — “Scenario → Cases”
23+
**Goal:** model flows, add edges, and produce a first set of cases with quality gates.
24+
25+
1. Map **MAE flows** (Main / Alternative / Exception) using [`30-scenario-patterns/main-alt-exception.md`](../30-scenario-patterns/main-alt-exception.md).
26+
2. Add **edge conditions** from one technique (boundary/equivalence or decision tables).
27+
3. If an API is involved, check **idempotency & retries** in [`40-api-and-data-contracts/idempotency-and-retries.md`](../40-api-and-data-contracts/idempotency-and-retries.md).
28+
4. Turn scenarios into **test cases with explicit expected results**.
29+
5. Run **coverage gates**:
30+
- Functional: [`60-checklists/functional-coverage.md`](../60-checklists/functional-coverage.md)
31+
- API (if applicable): [`60-checklists/api-coverage.md`](../60-checklists/api-coverage.md)
32+
33+
**Deliverable:** `scenarios.md` + `cases.md` under `playground/<feature>/`.
34+
35+
---
36+
37+
### 📆 2 hours/week — “4‑week improvement loop”
38+
**Goal:** build repeatable muscle memory and measurable coverage gains.
39+
40+
- **Week 1 — Fundamentals:**
41+
Read [`10-fundamentals/coverage-thinking.md`](../10-fundamentals/coverage-thinking.md) and apply to a feature.
42+
- **Week 2 — Non‑functional angles:**
43+
Add perf/resiliency/security using [`50-non-functional/`](../50-non-functional/).
44+
- **Week 3 — API contracts:**
45+
Validate schemas, errors, and idempotency in [`40-api-and-data-contracts/`](../40-api-and-data-contracts/).
46+
- **Week 4 — Review & metrics:**
47+
Use [`65-review-gates-metrics-traceability/`](../65-review-gates-metrics-traceability/) to compare coverage before/after.
48+
49+
**Deliverable:** a short “before/after” note under `05-field-notes/before-after-coverage.md` (or a new file using `_templates/field-note.md`).
50+
51+
---
52+
53+
## Role shortcuts
54+
55+
Pick the lane that matches your work today:
56+
57+
### 👤 Solo QA / SDET
58+
- Start: `20-techniques/` → `30-scenario-patterns/`
59+
- Gate with: `60-checklists/functional-coverage.md`
60+
- Share a **field note** using `_templates/field-note.md`.
61+
62+
### 👥 Team Lead / QA Manager
63+
- Define a **taxonomy** + naming rules in `65-review-gates-metrics-traceability/traceability.md`.
64+
- Add **review gates** in `65-review-gates-metrics-traceability/review-gates.md`.
65+
- Seed a **mini‑project** under `70-mini-projects/` and ask the team to contribute.
66+
67+
### 🧰 Vendor / Tooling partner
68+
- Read `80-tools-and-integrations/mapping-to-test-managers.md`.
69+
- Propose an export mapping or integration idea.
70+
- See `90-community/partnership.md` for co‑marketing + credits.
71+
72+
### 📋 PM / Product
73+
- Use `57-cross-discipline-bridges/for-pms.md` to write **testable acceptance criteria**.
74+
- Link PRD items → scenarios → cases (traceability starter in `65-*/traceability.md`).
75+
76+
### 👨‍💻 Developer
77+
- Use `57-cross-discipline-bridges/for-developers.md` to define **observability contracts** (logs/metrics) that make results verifiable.
78+
- Add **error taxonomy → UX messages** with QA via `40-*/error-taxonomy.md`.
79+
80+
### 🛠️ SRE
81+
- Translate SLOs into **synthetic checks** seeded by scenarios (`57-*/for-sres.md`).
82+
- Capture **failure modes** and retries/backoff in `40-*/idempotency-and-retries.md`.
83+
84+
### 🔐 Security / Compliance
85+
- Map **threats → tests** with `57-*/for-security-compliance.md`.
86+
- Capture **evidence** for controls in `65-*/traceability.md`.
87+
88+
### 🎨 Design / UX
89+
- Design **error & empty states** up front (`57-*/for-design-ux.md`).
90+
- Ensure **a11y acceptance** via `50-non-functional/accessibility-a11y.md`.
91+
92+
---
93+
94+
## Minimal repo setup
95+
96+
> You only need Git + a Markdown editor. The items below are optional helpers.
97+
98+
### 1) Clone
99+
```bash
100+
git clone https://github.com/Treeify-ai/Awesome-Test-Case-Design.git
101+
cd Awesome-Test-Case-Design
102+
```
103+
104+
### 2) Optional: local lint & link checks
105+
- **Markdown lint (Node)**
106+
```bash
107+
npm install -g markdownlint-cli2
108+
markdownlint-cli2 "**/*.md"
109+
```
110+
- **Link check (Docker)**
111+
```bash
112+
docker run --rm -v "$PWD":/work ghcr.io/lycheeverse/lychee lychee --no-progress --verbose "./**/*.md"
113+
```
114+
115+
> CI runs these automatically via `.github/workflows/markdown-lint.yml` and `link-check.yml`.
116+
117+
### 3) Use templates
118+
Copy an appropriate template from `_templates/` and fill it:
119+
- Technique → `_templates/technique.md`
120+
- Scenario pattern → `_templates/scenario-pattern.md`
121+
- Mini‑project → `_templates/mini-project.md`
122+
- Checklist → `_templates/checklist.md`
123+
- Field note / Case study / Postmortem → corresponding templates
124+
125+
Place new content in the matching folder (see the repo root README’s ToC).
126+
127+
### 4) Commit & PR
128+
```bash
129+
git checkout -b feat/<short-topic>
130+
git add .
131+
git commit -m "feat: add boundary examples for checkout discount"
132+
git push origin feat/<short-topic>
133+
```
134+
Open a PR and complete the **PR template**. Label suggestions: `technique`, `scenario`, `checklist`, `mini-project`, `api`, `non-functional`, `beginner|intermediate|advanced`.
135+
136+
---
137+
138+
## First success checklist
139+
140+
- [ ] I applied **one** technique to a real feature.
141+
- [ ] My cases include **explicit expected results**.
142+
- [ ] I ran at least one **coverage checklist**.
143+
- [ ] I wrote a **short field note** with outcomes or lessons.
144+
- [ ] I shared it via PR (or a gist) to help the next person.
145+
146+
---
147+
148+
## Optional: Try Treeify (light touch)
149+
150+
Exploring AI‑augmented test design? [Treeify](https://treeifyai.com/?utm_source=github&utm_medium=awesome_repo&utm_campaign=readme_hero) can help you turn long PRDs into structured **test objects → scenarios → cases**, exportable to your tools—while you stay in control.
151+
152+
- Contact: **contact@treeifyai.com** (mention “Awesome repo” for early‑user perks).

0 commit comments

Comments
 (0)