Skip to content

Commit d5d89c9

Browse files
docs(skills): clarify env example requirements (#47)
* docs(skills): clarify env example requirements * test: cap aiohttp for aioresponses compatibility
1 parent 4be829b commit d5d89c9

5 files changed

Lines changed: 57 additions & 14 deletions

File tree

pyproject.toml

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,8 @@ test = [
3434
"pytest>=8.0",
3535
"pytest-asyncio>=0.24",
3636
"aioresponses>=0.7",
37+
# aioresponses 0.7.8 is not compatible with aiohttp 3.14's ClientResponse constructor.
38+
"aiohttp<3.14",
3739
"pytest-cov>=6.0",
3840
]
3941

@@ -43,4 +45,4 @@ asyncio_mode = "auto"
4345

4446
[project.urls]
4547
Homepage = "https://github.com/Autohive-AI/integrations-sdk"
46-
Issues = "https://github.com/Autohive-AI/integrations-sdk/issues"
48+
Issues = "https://github.com/Autohive-AI/integrations-sdk/issues"

skills/building-integration/SKILL.md

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -145,7 +145,16 @@ Only runs tests in `test_*_unit.py` files (configured in `pyproject.toml`).
145145

146146
If the integration has `test_*_integration.py` files, check whether they reference environment variables via `env_credentials(...)`, `os.environ.get(...)`, `os.getenv(...)`, `os.environ[...]`, or equivalent helpers. Every referenced variable must appear as a blank template entry in the repository root `.env.example`.
147147

148-
This is a manual review/build requirement: `.env` stays local and uncommitted, but `.env.example` is the committed contract that tells humans, reviewers, and automation what to provide.
148+
This is a manual review/build requirement: `.env` stays local and uncommitted, but `.env.example` is the committed contract that tells humans, reviewers, and automation what to provide. Missing entries should be treated as a validation failure for PRs adding or changing integration tests.
149+
150+
Use a focused cross-check:
151+
152+
```bash
153+
grep -RInE 'env_credentials\(|os\.environ|getenv\(' <name>/tests/test_*_integration.py
154+
grep -nE 'INTEGRATION_PREFIX|SERVICE_PREFIX' .env.example
155+
```
156+
157+
Document required credentials, optional test IDs, and destructive-test-only variables. If an env var read is unused, remove it rather than documenting unnecessary setup.
149158

150159
### Step 6 — Auto-fix lint and formatting issues
151160

@@ -201,4 +210,4 @@ Key settings:
201210
| Config-code sync | Ensure actions in `config.json` match handlers in source code |
202211
| Missing test files | Create `tests/test_<name>_unit.py` (see `writing-unit-tests` skill) |
203212
| Import errors | Check `requirements.txt` has all dependencies; check `__init__.py` exports |
204-
| Integration tests use env vars but `.env.example` missing entries | Add blank entries for every referenced env var to root `.env.example` |
213+
| Integration tests use env vars but `.env.example` missing entries | Add blank entries for every referenced env var to root `.env.example`; treat as blocking for PRs adding/changing integration tests |

skills/reviewing-integration-prs/SKILL.md

Lines changed: 15 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ The goal is not just a generic code review. Validate the PR against the relevant
1414
- Review the actual diff and the surrounding integration code; do not rely only on CI status.
1515
- Treat security and secret leakage as blockers.
1616
- Treat failed tooling/validation as blockers unless there is a documented, intentional exception.
17+
- Treat missing root `.env.example` entries as blockers when a PR adds or changes integration tests that read env vars.
1718
- Use the existing skills as sub-skill checklists when their context applies.
1819
- Prefer actionable review comments with file/line references and concrete fixes.
1920
- Separate blockers from should-fix items and minor suggestions.
@@ -104,6 +105,8 @@ For migrations to public, apply the `migrating-private-integration` security/saf
104105

105106
### 5. Tests and `.env.example`
106107

108+
The root `.env.example` check is mandatory for any PR that adds or changes `test_*_integration.py`. Do not rely on CI to catch this; review it manually against the test file.
109+
107110
For unit tests, apply `writing-unit-tests`:
108111

109112
- Tests are named `test_*_unit.py`.
@@ -118,8 +121,16 @@ For integration tests, apply `writing-integration-tests`:
118121
- `live_context` returns `FetchResponse` for SDK 2.x `context.fetch` paths.
119122
- Tests skip cleanly when credentials or required test IDs are missing.
120123
- Every env var read by `env_credentials(...)`, `os.environ.get(...)`, `os.getenv(...)`, `os.environ[...]`, or equivalent helpers is listed as a blank template entry in root `.env.example`.
124+
- Optional test IDs, destructive-test-only variables, and module-level env constants count; either document them in root `.env.example` or remove unused reads.
125+
126+
Use a focused search while reviewing:
127+
128+
```bash
129+
grep -RInE 'env_credentials\(|os\.environ|getenv\(' <integration>/tests/test_*_integration.py
130+
grep -nE 'INTEGRATION_PREFIX|SERVICE_PREFIX' .env.example
131+
```
121132

122-
Missing `.env.example` entries are a should-fix at minimum, and often a blocker for PRs whose main purpose is adding/changing integration tests.
133+
Missing `.env.example` entries are blockers when the PR adds or changes integration tests that read env vars. If the PR only exposes a pre-existing omission outside the changed tests, call it a should-fix unless it blocks the changed behavior.
123134

124135
### 6. SDK v2 upgrade review
125136

@@ -147,8 +158,8 @@ Check:
147158

148159
Use these categories:
149160

150-
- **🚫 Blocker** — must fix before merge. Security leaks, broken validation, schema/action mismatch, new integration input drift, missing required files, broken imports/tests, unsafe public migration.
151-
- **⚠️ Should-fix** — strongly recommended before merge. Missing `.env.example` entries, incomplete test coverage for changed behavior, unclear docs, overly broad error handling, avoidable warnings.
161+
- **🚫 Blocker** — must fix before merge. Security leaks, broken validation, schema/action mismatch, new integration input drift, missing required files, broken imports/tests, unsafe public migration, or missing root `.env.example` entries for env vars read by newly added/changed integration tests.
162+
- **⚠️ Should-fix** — strongly recommended before merge. Pre-existing `.env.example` omissions not introduced by the PR, incomplete test coverage for changed behavior, unclear docs, overly broad error handling, avoidable warnings.
152163
- **💡 Suggestion** — optional improvements. Naming clarity, small refactors, additional examples, extra edge-case tests.
153164

154165
## Review Output Template
@@ -184,5 +195,5 @@ Before completing the review, verify:
184195
- [ ] Tooling/CI status was checked or local validation was run.
185196
- [ ] Config/code/schema sync was inspected.
186197
- [ ] Unit/integration test changes were reviewed against the appropriate test skill.
187-
- [ ] Root `.env.example` was checked when integration tests reference env vars.
198+
- [ ] Root `.env.example` was checked when integration tests reference env vars; missing entries introduced by the PR were treated as blockers.
188199
- [ ] Findings are actionable and severity-labelled.

skills/upgrading-sdk-v2/SKILL.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -210,7 +210,7 @@ async def real_fetch(url, *, method="GET", json=None, headers=None, **kwargs):
210210
)
211211
```
212212

213-
If the integration tests read credentials or test resource IDs from environment variables, also verify the repository root `.env.example` lists every variable used by those tests. Add missing entries in the same PR. This applies to variables read through `env_credentials(...)`, `os.environ.get(...)`, `os.getenv(...)`, `os.environ[...]`, or equivalent helpers. The local `.env` file remains uncommitted; `.env.example` is the committed setup contract for humans and tooling.
213+
If the integration tests read credentials or test resource IDs from environment variables, also verify the repository root `.env.example` lists every variable used by those tests. Add missing entries in the same PR. This applies to variables read through `env_credentials(...)`, `os.environ.get(...)`, `os.getenv(...)`, `os.environ[...]`, or equivalent helpers. Required credentials, optional test IDs, and destructive-test-only variables all count. The local `.env` file remains uncommitted; `.env.example` is the committed setup contract for humans and tooling. Missing entries should block SDK-upgrade PRs that add or change integration tests.
214214

215215
### Step 7 — Local validation (required before pushing)
216216

@@ -265,7 +265,7 @@ Before considering an integration upgraded, verify:
265265
- [ ] Unit test error assertions use `result.type == ResultType.ACTION_ERROR` and `result.result.message`
266266
- [ ] `pytest.raises(ValidationError)` replaced with `result.type == ResultType.VALIDATION_ERROR`
267267
- [ ] Integration test `real_fetch` returns `FetchResponse(...)` (if applicable)
268-
- [ ] Integration test env vars are documented in root `.env.example` (if integration tests read env vars)
268+
- [ ] Integration test env vars are documented in root `.env.example` (if integration tests read env vars, including optional and destructive-test-only variables)
269269
- [ ] `FetchResponse` and `ResultType` are imported where needed
270270
- [ ] All unit tests pass
271271
- [ ] `ruff check` and `ruff format --config ../autohive-integrations-tooling/ruff.toml` pass
@@ -303,4 +303,4 @@ Before considering an integration upgraded, verify:
303303

304304
10. **PyPI package name collision**: If your integration folder name matches a PyPI package the source imports (e.g. an integration in `supadata/` that does `from supadata import Supadata`), an empty `<integration>/__init__.py` will shadow the real PyPI package and every test fails with `ImportError`. Drop `<integration>/__init__.py` — the validator's "missing __init__.py" warning is correct to ignore in this case, and the Lambda runtime is unaffected (the entry point is the action source file, not the package). See the `writing-unit-tests` skill for the matching test-side guidance.
305305

306-
11. **Missing `.env.example` entries after touching integration tests**: SDK upgrades often touch `live_context` and credential handling. If the integration test file references `FOO_ACCESS_TOKEN`, `FOO_TEST_ID`, or similar, root `.env.example` must contain `FOO_ACCESS_TOKEN=`, `FOO_TEST_ID=`, etc. Do not rely on future contributors or automation to grep test code for setup requirements.
306+
11. **Missing `.env.example` entries after touching integration tests**: SDK upgrades often touch `live_context` and credential handling. If the integration test file references `FOO_ACCESS_TOKEN`, `FOO_TEST_ID`, or similar, root `.env.example` must contain `FOO_ACCESS_TOKEN=`, `FOO_TEST_ID=`, etc. Optional IDs and destructive-test-only variables count too. Do not rely on future contributors or automation to grep test code for setup requirements; missing entries should block PRs that add or change those tests.

skills/writing-integration-tests/SKILL.md

Lines changed: 25 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,18 @@ description: "Writes pytest end-to-end integration tests for an Autohive integra
55

66
# Writing Integration Tests for an Integration
77

8+
## Required: root `.env.example` update
9+
10+
If a PR adds or changes integration tests that read environment variables, it must update the repository root `.env.example` in the same PR. This is part of the integration-test deliverable, not optional documentation.
11+
12+
This applies to every variable read through `env_credentials(...)`, `os.environ.get(...)`, `os.getenv(...)`, `os.environ[...]`, or equivalent helpers, including:
13+
14+
- Required credentials such as access tokens and API keys
15+
- Optional test resource IDs used to avoid list→get chaining
16+
- Variables used only by destructive tests
17+
18+
If a variable is read by the test file, either add a blank template entry to root `.env.example` or remove the unused env-var read. Missing entries should block integration-test PRs until fixed.
19+
820
## Prerequisites
921

1022
- The integration must be on **SDK 2.0.0** (`autohive-integrations-sdk~=2.0.0` in `requirements.txt`)
@@ -134,7 +146,16 @@ Why this matters:
134146
- Automated and AI tooling can discover which credentials, test IDs, and optional knobs it needs to ask for.
135147
- Reviewers have one canonical place to verify the local test contract.
136148

137-
Add every variable used by `env_credentials(...)`, `os.environ.get(...)`, `os.getenv(...)`, `os.environ[...]`, or equivalent helpers. Include both required credentials and optional test resource IDs. Keep real secrets only in local `.env`; never commit `.env`.
149+
Add every variable used by `env_credentials(...)`, `os.environ.get(...)`, `os.getenv(...)`, `os.environ[...]`, or equivalent helpers. Include required credentials, optional test resource IDs, and destructive-test-only variables. Keep real secrets only in local `.env`; never commit `.env`.
150+
151+
Before finishing, run a focused cross-check:
152+
153+
```bash
154+
grep -RInE 'env_credentials\(|os\.environ|getenv\(' <integration>/tests/test_*_integration.py
155+
grep -nE 'MYINTEGRATION|SERVICE_PREFIX' .env.example
156+
```
157+
158+
Every env var found in the integration test file must be present as a blank template entry in root `.env.example`. If the variable is no longer used by tests, remove the env-var read instead of documenting unnecessary setup.
138159

139160
Use a clearly labeled block in root `.env.example`:
140161

@@ -145,7 +166,7 @@ MYINTEGRATION_TEST_ITEM_ID=
145166
MYINTEGRATION_TEST_PROJECT_ID=
146167
```
147168

148-
Before finishing, cross-check the integration test file against `.env.example`: every env var referenced in tests must appear in `.env.example`, even if the test skips when it is missing.
169+
Before finishing, cross-check the integration test file against `.env.example`: every env var referenced in tests must appear in `.env.example`, even if the test skips when it is missing or the variable is optional.
149170

150171
## The live_context Fixture
151172

@@ -466,7 +487,7 @@ Write actions (create/update/delete) need at least:
466487
1. **Read the integration source** — understand each action and its auth mechanism
467488
2. **Check `config.json`** — determine auth type (none / API key / platform OAuth)
468489
3. **Choose the right `live_context` variant** — see the three variants above
469-
4. **Document env vars in root `.env.example` (required)** — add every env var referenced by the integration test file in the same PR
490+
4. **Document env vars in root `.env.example` (required)** — add every env var referenced by the integration test file in the same PR; missing entries should block integration-test PRs
470491
5. **Write read-only tests first** — these are safe to run repeatedly
471492
6. **Add destructive tests** with `@pytest.mark.destructive` for write actions
472493
7. **Run read-only tests**:
@@ -494,7 +515,7 @@ Write actions (create/update/delete) need at least:
494515

495516
5. **Destructive test cleanup**: Always clean up created data at the end of lifecycle tests. Use `os.getpid()` in created object names to avoid collisions when running tests in parallel.
496517

497-
6. **Missing `.env.example` entries**: If you add `env_credentials("FOO")`, `os.environ.get("FOO")`, or similar test env-var usage, add `FOO=` to root `.env.example` before you commit. Do not make reviewers or future contributors grep test files to discover setup requirements.
518+
6. **Missing `.env.example` entries**: If you add `env_credentials("FOO")`, `os.environ.get("FOO")`, `os.getenv("FOO")`, `os.environ["FOO"]`, or similar test env-var usage, add `FOO=` to root `.env.example` before you commit. Do not make reviewers or future contributors grep test files to discover setup requirements. Optional IDs and destructive-test-only variables count too.
498519

499520
7. **OAuth token expiry**: Platform OAuth tokens expire. Document the refresh process in the integration's README or root `.env.example`.
500521

0 commit comments

Comments
 (0)