|
| 1 | +# GitHub API calls in `che-api` extension fail behind TLS-intercepting proxies (ZScaler, corporate proxies) |
| 2 | + |
| 3 | +## Problem |
| 4 | + |
| 5 | +The `che-api` extension's GitHub API calls (`getUser()`, `getTokenScopes()`) fail in enterprise environments that use TLS-intercepting proxies such as **ZScaler**, **Symantec**, or any corporate HTTPS proxy that re-signs certificates with a custom CA. |
| 6 | + |
| 7 | +Two distinct issues were identified: |
| 8 | + |
| 9 | +### 1. `axios` does not send CONNECT requests for HTTPS over HTTP proxy |
| 10 | + |
| 11 | +The original implementation used `axios` to call `https://api.github.com/user`. When an HTTP proxy is configured (e.g. `HTTPS_PROXY=http://proxy:8080`), `axios` fails to issue a `CONNECT` request to establish a tunnel for the HTTPS connection. This is a known `axios` defect ([axios#4531](https://github.com/axios/axios/issues/4531), [axios#3384](https://github.com/axios/axios/issues/3384)). |
| 12 | + |
| 13 | +**Result**: GitHub API calls hang or fail outright behind any HTTPS-intercepting proxy. |
| 14 | + |
| 15 | +### 2. `fetch` (undici) does not honor `NODE_EXTRA_CA_CERTS` in VS Code extension host |
| 16 | + |
| 17 | +After replacing `axios` with `globalThis.fetch`, a second problem emerged: `fetch` (backed by Node.js's `undici`) fails with `UNABLE_TO_VERIFY_LEAF_SIGNATURE` when HTTPS traffic is intercepted by a proxy using a custom CA certificate — even when the CA is correctly provided via the `NODE_EXTRA_CA_CERTS` environment variable. |
| 18 | + |
| 19 | +In contrast, Node.js's native `https` module (patched by VS Code's `@vscode/proxy-agent`) correctly loads and trusts the custom CA under the same conditions. |
| 20 | + |
| 21 | +**Result**: GitHub API calls fail with a TLS certificate error despite the custom CA being mounted and configured. |
| 22 | + |
| 23 | +## How to reproduce |
| 24 | + |
| 25 | +### Prerequisites |
| 26 | + |
| 27 | +- [mitmproxy](https://mitmproxy.org/) installed on the host (simulates ZScaler's TLS interception) |
| 28 | +- `podman` (or `docker`) available |
| 29 | +- A valid GitHub personal access token |
| 30 | + |
| 31 | +### Steps |
| 32 | + |
| 33 | +1. **Start mitmproxy on the host:** |
| 34 | + |
| 35 | + ```bash |
| 36 | + mitmdump --listen-port 8080 |
| 37 | + ``` |
| 38 | + |
| 39 | +2. **Run the che-code container with proxy settings and the mitmproxy CA:** |
| 40 | + |
| 41 | + ```bash |
| 42 | + podman run --rm -it -p 3100:3100 \ |
| 43 | + -e CODE_HOST=0.0.0.0 \ |
| 44 | + -e HTTPS_PROXY=http://host.containers.internal:8080 \ |
| 45 | + -e HTTP_PROXY=http://host.containers.internal:8080 \ |
| 46 | + -e NODE_EXTRA_CA_CERTS=/public-certs/mitmproxy-ca.crt \ |
| 47 | + -v ~/.mitmproxy/mitmproxy-ca-cert.pem:/public-certs/mitmproxy-ca.crt:ro \ |
| 48 | + quay.io/redhat-user-workloads/devspaces-tenant/devspaces/code-rhel9:3.28 |
| 49 | + ``` |
| 50 | + |
| 51 | +3. **Open the editor** in a browser at `http://localhost:3100`. |
| 52 | + |
| 53 | +4. **Trigger any flow that calls `che-api`'s `GithubService.getUser()`** (e.g. the Device Authentication flow after completing GitHub OAuth). |
| 54 | + |
| 55 | +### Expected result |
| 56 | + |
| 57 | +The GitHub API call succeeds, using the custom CA provided via `NODE_EXTRA_CA_CERTS` to verify the proxy's re-signed certificate. |
| 58 | + |
| 59 | +### Actual result |
| 60 | + |
| 61 | +- **With `axios` (before the fix):** the request hangs or fails because no `CONNECT` tunnel is established. |
| 62 | +- **With `fetch` (after removing axios):** the request fails with: |
| 63 | + ``` |
| 64 | + TypeError: fetch failed |
| 65 | + cause: Error: unable to verify the first certificate |
| 66 | + code: 'UNABLE_TO_VERIFY_LEAF_SIGNATURE' |
| 67 | + ``` |
| 68 | + |
| 69 | +### Workaround for testing |
| 70 | + |
| 71 | +Setting `NODE_TLS_REJECT_UNAUTHORIZED=0` makes `fetch` succeed, confirming the issue is strictly about custom CA trust — not connectivity. |
| 72 | + |
| 73 | +## Root cause analysis |
| 74 | + |
| 75 | +### axios issue |
| 76 | + |
| 77 | +`axios` uses Node.js's `http`/`https` modules internally but has a bug in its proxy handling: it does not issue a `CONNECT` request when making HTTPS requests through an HTTP proxy. This prevents establishing the required TLS tunnel. |
| 78 | + |
| 79 | +### fetch/undici issue |
| 80 | + |
| 81 | +VS Code's extension host (`proxyResolver.ts`) patches both `globalThis.fetch` and `require('https')` via `@vscode/proxy-agent` to inject proxy and certificate handling. However, the patched `fetch` (undici-based) does not properly integrate custom CAs from `NODE_EXTRA_CA_CERTS` in all environments. The patched `https` module does handle this correctly. |
| 82 | + |
| 83 | +This is the same class of issue that upstream VS Code addressed in the `github-authentication` extension by implementing a fetcher fallback chain: `electron.net.fetch` → `globalThis.fetch` → `Node http/s` (see `code/extensions/github-authentication/src/node/fetch.ts`). |
| 84 | + |
| 85 | +## Fix |
| 86 | + |
| 87 | +The fix addresses both issues by: |
| 88 | + |
| 89 | +1. **Removing the `axios` dependency** from `che-api`'s `GithubServiceImpl`. |
| 90 | +2. **Implementing a fetch → https fallback chain** (similar to upstream `github-authentication`): |
| 91 | + - First attempts the request using `globalThis.fetch` (works in most environments, supports modern APIs). |
| 92 | + - If `fetch` fails (e.g. due to custom CA not being trusted), falls back to `https.request` (Node.js native module, correctly patched by VS Code to honor `NODE_EXTRA_CA_CERTS`). |
| 93 | +3. **Adding diagnostic logging** at each step to simplify troubleshooting in customer environments. |
| 94 | + |
| 95 | +## Environment |
| 96 | + |
| 97 | +- **Affected component**: `code/extensions/che-api/src/impl/github-service-impl.ts` |
| 98 | +- **Proxy types affected**: Any TLS-intercepting proxy (ZScaler, Symantec, mitmproxy, corporate MITM proxies) |
| 99 | +- **Tested with**: `mitmproxy` 11.x simulating TLS interception, `podman` container with `che-code:next` and `code-rhel9:3.28` images |
0 commit comments