You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: skills/building-integration/SKILL.md
+11-2Lines changed: 11 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -145,7 +145,16 @@ Only runs tests in `test_*_unit.py` files (configured in `pyproject.toml`).
145
145
146
146
If the integration has `test_*_integration.py` files, check whether they reference environment variables via `env_credentials(...)`, `os.environ.get(...)`, `os.getenv(...)`, `os.environ[...]`, or equivalent helpers. Every referenced variable must appear as a blank template entry in the repository root `.env.example`.
147
147
148
-
This is a manual review/build requirement: `.env` stays local and uncommitted, but `.env.example` is the committed contract that tells humans, reviewers, and automation what to provide.
148
+
This is a manual review/build requirement: `.env` stays local and uncommitted, but `.env.example` is the committed contract that tells humans, reviewers, and automation what to provide. Missing entries should be treated as a validation failure for PRs adding or changing integration tests.
Document required credentials, optional test IDs, and destructive-test-only variables. If an env var read is unused, remove it rather than documenting unnecessary setup.
149
158
150
159
### Step 6 — Auto-fix lint and formatting issues
151
160
@@ -201,4 +210,4 @@ Key settings:
201
210
| Config-code sync | Ensure actions in `config.json` match handlers in source code |
202
211
| Missing test files | Create `tests/test_<name>_unit.py` (see `writing-unit-tests` skill) |
203
212
| Import errors | Check `requirements.txt` has all dependencies; check `__init__.py` exports |
204
-
| Integration tests use env vars but `.env.example` missing entries | Add blank entries for every referenced env var to root `.env.example`|
213
+
| Integration tests use env vars but `.env.example` missing entries | Add blank entries for every referenced env var to root `.env.example`; treat as blocking for PRs adding/changing integration tests|
Copy file name to clipboardExpand all lines: skills/reviewing-integration-prs/SKILL.md
+15-4Lines changed: 15 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -14,6 +14,7 @@ The goal is not just a generic code review. Validate the PR against the relevant
14
14
- Review the actual diff and the surrounding integration code; do not rely only on CI status.
15
15
- Treat security and secret leakage as blockers.
16
16
- Treat failed tooling/validation as blockers unless there is a documented, intentional exception.
17
+
- Treat missing root `.env.example` entries as blockers when a PR adds or changes integration tests that read env vars.
17
18
- Use the existing skills as sub-skill checklists when their context applies.
18
19
- Prefer actionable review comments with file/line references and concrete fixes.
19
20
- Separate blockers from should-fix items and minor suggestions.
@@ -104,6 +105,8 @@ For migrations to public, apply the `migrating-private-integration` security/saf
104
105
105
106
### 5. Tests and `.env.example`
106
107
108
+
The root `.env.example` check is mandatory for any PR that adds or changes `test_*_integration.py`. Do not rely on CI to catch this; review it manually against the test file.
109
+
107
110
For unit tests, apply `writing-unit-tests`:
108
111
109
112
- Tests are named `test_*_unit.py`.
@@ -118,8 +121,16 @@ For integration tests, apply `writing-integration-tests`:
118
121
-`live_context` returns `FetchResponse` for SDK 2.x `context.fetch` paths.
119
122
- Tests skip cleanly when credentials or required test IDs are missing.
120
123
- Every env var read by `env_credentials(...)`, `os.environ.get(...)`, `os.getenv(...)`, `os.environ[...]`, or equivalent helpers is listed as a blank template entry in root `.env.example`.
124
+
- Optional test IDs, destructive-test-only variables, and module-level env constants count; either document them in root `.env.example` or remove unused reads.
Missing `.env.example` entries are a should-fix at minimum, and often a blocker for PRs whose main purpose is adding/changing integration tests.
133
+
Missing `.env.example` entries are blockers when the PR adds or changes integration tests that read env vars. If the PR only exposes a pre-existing omission outside the changed tests, call it a should-fix unless it blocks the changed behavior.
123
134
124
135
### 6. SDK v2 upgrade review
125
136
@@ -147,8 +158,8 @@ Check:
147
158
148
159
Use these categories:
149
160
150
-
-**🚫 Blocker** — must fix before merge. Security leaks, broken validation, schema/action mismatch, new integration input drift, missing required files, broken imports/tests, unsafe public migration.
151
-
-**⚠️ Should-fix** — strongly recommended before merge. Missing`.env.example`entries, incomplete test coverage for changed behavior, unclear docs, overly broad error handling, avoidable warnings.
161
+
-**🚫 Blocker** — must fix before merge. Security leaks, broken validation, schema/action mismatch, new integration input drift, missing required files, broken imports/tests, unsafe public migration, or missing root `.env.example` entries for env vars read by newly added/changed integration tests.
162
+
-**⚠️ Should-fix** — strongly recommended before merge. Pre-existing`.env.example`omissions not introduced by the PR, incomplete test coverage for changed behavior, unclear docs, overly broad error handling, avoidable warnings.
152
163
-**💡 Suggestion** — optional improvements. Naming clarity, small refactors, additional examples, extra edge-case tests.
153
164
154
165
## Review Output Template
@@ -184,5 +195,5 @@ Before completing the review, verify:
184
195
-[ ] Tooling/CI status was checked or local validation was run.
185
196
-[ ] Config/code/schema sync was inspected.
186
197
-[ ] Unit/integration test changes were reviewed against the appropriate test skill.
187
-
-[ ] Root `.env.example` was checked when integration tests reference env vars.
198
+
-[ ] Root `.env.example` was checked when integration tests reference env vars; missing entries introduced by the PR were treated as blockers.
188
199
-[ ] Findings are actionable and severity-labelled.
If the integration tests read credentials or test resource IDs from environment variables, also verify the repository root `.env.example` lists every variable used by those tests. Add missing entries in the same PR. This applies to variables read through `env_credentials(...)`, `os.environ.get(...)`, `os.getenv(...)`, `os.environ[...]`, or equivalent helpers. The local `.env` file remains uncommitted; `.env.example` is the committed setup contract for humans and tooling.
213
+
If the integration tests read credentials or test resource IDs from environment variables, also verify the repository root `.env.example` lists every variable used by those tests. Add missing entries in the same PR. This applies to variables read through `env_credentials(...)`, `os.environ.get(...)`, `os.getenv(...)`, `os.environ[...]`, or equivalent helpers. Required credentials, optional test IDs, and destructive-test-only variables all count. The local `.env` file remains uncommitted; `.env.example` is the committed setup contract for humans and tooling. Missing entries should block SDK-upgrade PRs that add or change integration tests.
214
214
215
215
### Step 7 — Local validation (required before pushing)
216
216
@@ -265,7 +265,7 @@ Before considering an integration upgraded, verify:
265
265
-[ ] Unit test error assertions use `result.type == ResultType.ACTION_ERROR` and `result.result.message`
266
266
-[ ]`pytest.raises(ValidationError)` replaced with `result.type == ResultType.VALIDATION_ERROR`
267
267
-[ ] Integration test `real_fetch` returns `FetchResponse(...)` (if applicable)
268
-
-[ ] Integration test env vars are documented in root `.env.example` (if integration tests read env vars)
268
+
-[ ] Integration test env vars are documented in root `.env.example` (if integration tests read env vars, including optional and destructive-test-only variables)
269
269
-[ ]`FetchResponse` and `ResultType` are imported where needed
270
270
-[ ] All unit tests pass
271
271
-[ ]`ruff check` and `ruff format --config ../autohive-integrations-tooling/ruff.toml` pass
@@ -303,4 +303,4 @@ Before considering an integration upgraded, verify:
303
303
304
304
10. **PyPI package name collision**: If your integration folder name matches a PyPI package the source imports (e.g. an integration in`supadata/` that does `from supadata import Supadata`), an empty `<integration>/__init__.py` will shadow the real PyPI package and every test fails with`ImportError`. Drop `<integration>/__init__.py` — the validator's "missing __init__.py" warning is correct to ignore in this case, and the Lambda runtime is unaffected (the entry point is the action source file, not the package). See the `writing-unit-tests` skill for the matching test-side guidance.
305
305
306
-
11. **Missing `.env.example` entries after touching integration tests**: SDK upgrades often touch `live_context`and credential handling. If the integration test file references `FOO_ACCESS_TOKEN`, `FOO_TEST_ID`, or similar, root `.env.example` must contain `FOO_ACCESS_TOKEN=`, `FOO_TEST_ID=`, etc. Do not rely on future contributors or automation to grep test code for setup requirements.
306
+
11. **Missing `.env.example` entries after touching integration tests**: SDK upgrades often touch `live_context`and credential handling. If the integration test file references `FOO_ACCESS_TOKEN`, `FOO_TEST_ID`, or similar, root `.env.example` must contain `FOO_ACCESS_TOKEN=`, `FOO_TEST_ID=`, etc. Optional IDs and destructive-test-only variables count too. Do not rely on future contributors or automation to grep test code for setup requirements; missing entries should block PRs that add or change those tests.
Copy file name to clipboardExpand all lines: skills/writing-integration-tests/SKILL.md
+25-4Lines changed: 25 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,6 +5,18 @@ description: "Writes pytest end-to-end integration tests for an Autohive integra
5
5
6
6
# Writing Integration Tests for an Integration
7
7
8
+
## Required: root `.env.example` update
9
+
10
+
If a PR adds or changes integration tests that read environment variables, it must update the repository root `.env.example` in the same PR. This is part of the integration-test deliverable, not optional documentation.
11
+
12
+
This applies to every variable read through `env_credentials(...)`, `os.environ.get(...)`, `os.getenv(...)`, `os.environ[...]`, or equivalent helpers, including:
13
+
14
+
- Required credentials such as access tokens and API keys
15
+
- Optional test resource IDs used to avoid list→get chaining
16
+
- Variables used only by destructive tests
17
+
18
+
If a variable is read by the test file, either add a blank template entry to root `.env.example` or remove the unused env-var read. Missing entries should block integration-test PRs until fixed.
19
+
8
20
## Prerequisites
9
21
10
22
- The integration must be on **SDK 2.0.0** (`autohive-integrations-sdk~=2.0.0` in `requirements.txt`)
@@ -134,7 +146,16 @@ Why this matters:
134
146
- Automated and AI tooling can discover which credentials, test IDs, and optional knobs it needs to ask for.
135
147
- Reviewers have one canonical place to verify the local test contract.
136
148
137
-
Add every variable used by `env_credentials(...)`, `os.environ.get(...)`, `os.getenv(...)`, `os.environ[...]`, or equivalent helpers. Include both required credentials and optional test resource IDs. Keep real secrets only in local `.env`; never commit `.env`.
149
+
Add every variable used by `env_credentials(...)`, `os.environ.get(...)`, `os.getenv(...)`, `os.environ[...]`, or equivalent helpers. Include required credentials, optional test resource IDs, and destructive-test-only variables. Keep real secrets only in local `.env`; never commit `.env`.
Every env var found in the integration test file must be present as a blank template entry in root `.env.example`. If the variable is no longer used by tests, remove the env-var read instead of documenting unnecessary setup.
138
159
139
160
Use a clearly labeled block in root `.env.example`:
140
161
@@ -145,7 +166,7 @@ MYINTEGRATION_TEST_ITEM_ID=
145
166
MYINTEGRATION_TEST_PROJECT_ID=
146
167
```
147
168
148
-
Before finishing, cross-check the integration test file against `.env.example`: every env var referenced in tests must appear in `.env.example`, even if the test skips when it is missing.
169
+
Before finishing, cross-check the integration test file against `.env.example`: every env var referenced in tests must appear in `.env.example`, even if the test skips when it is missing or the variable is optional.
149
170
150
171
## The live_context Fixture
151
172
@@ -466,7 +487,7 @@ Write actions (create/update/delete) need at least:
466
487
1.**Read the integration source** — understand each action and its auth mechanism
467
488
2.**Check `config.json`** — determine auth type (none / API key / platform OAuth)
468
489
3.**Choose the right `live_context` variant** — see the three variants above
469
-
4.**Document env vars in root `.env.example` (required)** — add every env var referenced by the integration test file in the same PR
490
+
4.**Document env vars in root `.env.example` (required)** — add every env var referenced by the integration test file in the same PR; missing entries should block integration-test PRs
470
491
5.**Write read-only tests first** — these are safe to run repeatedly
471
492
6.**Add destructive tests** with `@pytest.mark.destructive` for write actions
472
493
7.**Run read-only tests**:
@@ -494,7 +515,7 @@ Write actions (create/update/delete) need at least:
494
515
495
516
5.**Destructive test cleanup**: Always clean up created data at the end of lifecycle tests. Use `os.getpid()` in created object names to avoid collisions when running tests in parallel.
496
517
497
-
6.**Missing `.env.example` entries**: If you add `env_credentials("FOO")`, `os.environ.get("FOO")`, or similar test env-var usage, add `FOO=` to root `.env.example` before you commit. Do not make reviewers or future contributors grep test files to discover setup requirements.
518
+
6.**Missing `.env.example` entries**: If you add `env_credentials("FOO")`, `os.environ.get("FOO")`, `os.getenv("FOO")`, `os.environ["FOO"]`, or similar test env-var usage, add `FOO=` to root `.env.example` before you commit. Do not make reviewers or future contributors grep test files to discover setup requirements. Optional IDs and destructive-test-only variables count too.
498
519
499
520
7.**OAuth token expiry**: Platform OAuth tokens expire. Document the refresh process in the integration's README or root `.env.example`.
0 commit comments