Commit 2c61c05
authored
[test-improver] Improve tests for config package: fix flaky network error test (#6050)
# Test Improvements: validation_schema_fetch_test.go
## File Analyzed
- **Test File**: `internal/config/validation_schema_fetch_test.go`
- **Package**: `internal/config`
## Improvements Made
### 1. Stabler, More Reliable Tests
- ✅ **Fixed flaky `TestFetchAndFixSchema_NetworkError`**: replaced
external DNS hostname (`invalid-host-that-does-not-exist-12345.com`)
with a controlled `httptest.Server` that is immediately closed. The old
approach assumed the host would be unreachable at the network layer, but
in sandboxed CI environments the firewall intercepts the request and
returns HTTP 403, causing the assertion `ErrorContains(..., "failed to
fetch schema from")` to fail against the actual message `"failed to
fetch schema: HTTP 403"`.
- ✅ **Broadened error assertion**: changed `"failed to fetch schema
from"` → `"failed to fetch schema"` with an explanatory comment. This
common prefix matches both the network-failure form (`"failed to fetch
schema from <url>: ..."`) and any HTTP-error form (`"failed to fetch
schema: HTTP NNN"`), making the test environment-agnostic.
### 2. Root Cause
`fetchAndFixSchema` produces two distinct error message shapes:
- Line 280: `"failed to fetch schema from %s: %w"` — for
transport/network errors
- Lines 292, 300: `"failed to fetch schema: HTTP %d"` — for non-2xx HTTP
responses
A closed `httptest.Server` reliably produces a connection-refused
transport error (the first form), while also eliminating any dependency
on external network reachability.
## Test Execution
All Go tests pass:
```
ok github.com/github/gh-aw-mcpg/internal/config 1.792s
ok github.com/github/gh-aw-mcpg/test/integration 14.220s
```
(Rust guard tests fail only due to sandboxed firewall blocking crates.io
— pre-existing infrastructure issue unrelated to this change.)
## Why These Changes?
`TestFetchAndFixSchema_NetworkError` was actively failing in CI because
the sandboxed environment's network firewall intercepts outbound HTTP
requests and returns 403 instead of letting DNS resolution fail. This is
a common source of test instability: tests that depend on "external host
is unreachable" are fragile by definition. Using a locally-controlled
closed server is the idiomatic Go approach and is 100% reliable
regardless of the network environment.
---
*Generated by Test Improver Workflow*
*Focuses on better patterns, increased coverage, and more stable tests*
> [!WARNING]
> <details>
> <summary>Firewall blocked 2 domains</summary>
>
> The following domains were blocked by the firewall during workflow
execution:
>
> - `index.crates.io`
> - `invalidhostthatdoesnotexist12345.com`
>> To allow these domains, add them to the `network.allowed` list in
your workflow frontmatter:
>
> ```yaml
> network:
> allowed:
> - defaults
> - "index.crates.io"
> - "invalidhostthatdoesnotexist12345.com"
> ```
>
> See [Network
Configuration](https://github.github.com/gh-aw/reference/network/) for
more information.
>
> </details>
> Generated by [Test
Improver](https://github.com/github/gh-aw-mcpg/actions/runs/26132839859/agentic_workflow)
· ● 1.3M ·
[◷](https://github.com/search?q=repo%3Agithub%2Fgh-aw-mcpg+%22gh-aw-workflow-id%3A+test-improver%22&type=pullrequests)
<!-- gh-aw-agentic-workflow: Test Improver, engine: copilot, version:
1.0.40, model: claude-sonnet-4.6, id: 26132839859, workflow_id:
test-improver, run:
https://github.com/github/gh-aw-mcpg/actions/runs/26132839859 -->
<!-- gh-aw-workflow-id: test-improver -->1 file changed
Lines changed: 6 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
119 | 119 | | |
120 | 120 | | |
121 | 121 | | |
122 | | - | |
123 | | - | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
124 | 127 | | |
125 | | - | |
| 128 | + | |
126 | 129 | | |
127 | 130 | | |
128 | 131 | | |
| |||
0 commit comments