Auto-allow topology-attached container hostnames in Squid ACL (#6473)

* Initial plan

* Auto-allow topology-attached container hostnames in Squid ACL

In network-isolation mode, topology-attached containers (e.g. awmg-mcpg)
are already added to NO_PROXY so proxy-aware clients connect to them
directly. However, tools that honour HTTP(S)_PROXY but ignore NO_PROXY
route those requests through Squid, which blocks them because the
container hostname is not in the allowed-domain ACL.

Fix: in resolveAllowedDomains(), when networkIsolation is true and
topologyAttach container names are present, automatically add each name
to allowedDomains so Squid permits those requests.

* fix: align topology ACL gating with proxy env and mirror difcProxyHost

Address review feedback on auto-allowing topology peers in the Squid ACL:

- Gate the ACL addition on runtimeUsesComposeAgent() in addition to
  networkIsolation, mirroring buildProxyEnvironment() exactly. microVM
  backends (Docker sbx) run the agent off awf-net and reach peers via
  their own proxy-chaining path, so topology hostnames are skipped there
  too, keeping the ACL and NO_PROXY code paths consistent.
- Also auto-allow the DIFC/cli-proxy host (difcProxyHost) through Squid,
  mirroring the NO_PROXY entry added in #6438, so NO_PROXY-ignoring proxy
  clients can reach it. The scheme/port are stripped via parseDifcProxyHost.
- Document Squid dstdomain wildcard semantics (formatDomainForSquid
  prepends a leading dot, matching the host and its subdomains).
- Add tests: non-compose runtime skip, explicit compose runtime add, and
  difcProxyHost auto-allow.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 5f6caa3b-9079-4d7f-a752-524b0f990e0c

* fix: make Squid actually serve topology peers (DNS + non-standard ports)

Address reviewer feedback that adding topology peer hostnames to the
domain allowlist alone does not make them reachable through Squid:

1. DNS: Squid resolves proxied destinations via external `dns_nameservers`,
   which cannot resolve Docker-only peer names (e.g. `awmg-mcpg`).
   `patchComposeWithTopologyHosts` now also patches the `squid-proxy`
   service `extra_hosts` (Squid reads /etc/hosts before DNS). The phased
   startup applies this before the full `docker compose up`, which recreates
   squid-proxy to pick up the entries.

2. Ports: `http_access deny !Safe_ports` (80/443) fires before the domain
   allowlist, so `http://awmg-mcpg:8080` was denied on the port. A new
   `SquidConfig.topologyPeers` emits a per-peer `http_access allow`
   (dstdomain) *before* the Safe_ports and raw-IP deny rules, permitting any
   port to these operator-attached, trusted peers. This mirrors the existing
   apiProxyIp/apiProxyPorts precedent.

Also extract `resolveTopologyPeerHosts()` (topology-peers.ts) shared by
preflight (domain ACL) and config-writer (Squid allow), so the peer set and
compose-runtime gating stay in sync with buildProxyEnvironment().

Tests: topology-peers helper, Squid topology-peer section + rule ordering,
and squid-proxy extra_hosts patching. Full suite: 3986 passing.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 5f6caa3b-9079-4d7f-a752-524b0f990e0c

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Landon Cox <landon.cox@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Author: 3 people

src/commands/preflight.test.ts

@@ -296,6 +296,79 @@ describe('resolveAllowedDomains', () => {
     expect(mockedRules.loadAndMergeDomains).not.toHaveBeenCalled();
     expect(result.allowedDomains).toEqual([]);
   });
+
+  it('auto-adds topology-attached container names to allowedDomains in network-isolation mode', () => {
+    const result = resolveAllowedDomains({
+      networkIsolation: true,
+      topologyAttach: ['awmg-mcpg', 'awmg-cli-proxy'],
+    });
+    expect(result.allowedDomains).toContain('awmg-mcpg');
+    expect(result.allowedDomains).toContain('awmg-cli-proxy');
+    expect(mockedLogger.debug).toHaveBeenCalledWith(
+      expect.stringContaining('auto-allowing topology peer "awmg-mcpg"')
+    );
+    expect(mockedLogger.debug).toHaveBeenCalledWith(
+      expect.stringContaining('auto-allowing topology peer "awmg-cli-proxy"')
+    );
+  });
+
+  it('does not add topology-attached container names when networkIsolation is false', () => {
+    const result = resolveAllowedDomains({
+      networkIsolation: false,
+      topologyAttach: ['awmg-mcpg'],
+    });
+    expect(result.allowedDomains).not.toContain('awmg-mcpg');
+  });
+
+  it('does not duplicate topology container names already in allowedDomains', () => {
+    mockedDomainUtils.parseDomains.mockReturnValue(['awmg-mcpg']);
+    mockedOptionParsers.processLocalhostKeyword.mockReturnValue({
+      allowedDomains: ['awmg-mcpg'],
+      localhostDetected: false,
+      shouldEnableHostAccess: false,
+    });
+
+    const result = resolveAllowedDomains({
+      allowDomains: 'awmg-mcpg',
+      networkIsolation: true,
+      topologyAttach: ['awmg-mcpg'],
+    });
+    expect(result.allowedDomains.filter(d => d === 'awmg-mcpg')).toHaveLength(1);
+  });
+
+  it('does not add topology containers when topologyAttach is empty', () => {
+    const result = resolveAllowedDomains({
+      networkIsolation: true,
+      topologyAttach: [],
+    });
+    expect(result.allowedDomains).toEqual([]);
+  });
+
+  it('does not add topology container names for a non-compose runtime (e.g. sbx)', () => {
+    const result = resolveAllowedDomains({
+      networkIsolation: true,
+      containerRuntime: 'sbx',
+      topologyAttach: ['awmg-mcpg'],
+    });
+    expect(result.allowedDomains).not.toContain('awmg-mcpg');
+  });
+
+  it('adds topology container names for an explicit compose runtime', () => {
+    const result = resolveAllowedDomains({
+      networkIsolation: true,
+      containerRuntime: 'runc',
+      topologyAttach: ['awmg-mcpg'],
+    });
+    expect(result.allowedDomains).toContain('awmg-mcpg');
+  });
+
+  it('auto-allows the DIFC/cli-proxy host even when not listed in topologyAttach', () => {
+    const result = resolveAllowedDomains({
+      networkIsolation: true,
+      difcProxyHost: 'https://awmg-cli-proxy:18443',
+    });
+    expect(result.allowedDomains).toContain('awmg-cli-proxy');
+  });
 });
 
 describe('parseDomainOptions', () => {

src/commands/preflight.ts

@@ -8,6 +8,7 @@ import { parseDomains, parseDomainsFile } from '../domain-utils';
 import { processLocalhostKeyword } from '../option-parsers';
 import { resolveCopilotApiRouting } from '../copilot-api-resolver';
 import { resolveApiTargetsToAllowedDomains } from '../api-proxy-config';
+import { resolveTopologyPeerHosts } from '../topology-peers';
 
 /**
  * Resolves the Commander option-value source for a given option name.
@@ -176,6 +177,30 @@ export function resolveAllowedDomains(options: Record<string, unknown>): Allowed
     logger.debug.bind(logger)
   );
 
+  // In network-isolation (topology) mode, automatically add trusted topology
+  // peer hostnames to the Squid allowed-domain ACL. NO_PROXY is also set for
+  // these peers (in proxy-environment.ts) so that proxy-aware clients
+  // (undici/rmcp) connect directly; adding them here ensures Squid does not
+  // block requests from tools that honour HTTP(S)_PROXY but ignore NO_PROXY.
+  //
+  // This covers the standard-port (80/443) path. Non-standard MCP ports (e.g.
+  // http://awmg-mcpg:8080) and Squid's DNS resolution of these Docker-only
+  // hostnames are handled separately via SquidConfig.topologyPeers and the
+  // squid-proxy extra_hosts patch (see config-writer.ts / topology.ts).
+  //
+  // NOTE ON SQUID SEMANTICS: these names are emitted as dstdomain ACL entries
+  // via formatDomainForSquid, which prepends a leading dot (e.g. "awmg-mcpg" ->
+  // ".awmg-mcpg"). Squid therefore matches the host itself *and* any subdomain
+  // (*.awmg-mcpg). This is safe for internal Docker hostnames (a bare label
+  // like "github" matches host "github", not "github.com"), but operators
+  // should avoid topology names that collide with trusted public labels.
+  for (const peer of resolveTopologyPeerHosts(options)) {
+    if (!allowedDomains.includes(peer)) {
+      allowedDomains.push(peer);
+      logger.debug(`Network-isolation: auto-allowing topology peer "${peer}" in Squid ACL`);
+    }
+  }
+
   validateAllowedDomains(allowedDomains);
 
   return { allowedDomains, localhostResult, resolvedCopilotApiTarget, resolvedCopilotApiBasePath };

src/config-writer.ts

@@ -4,6 +4,7 @@ import * as yaml from 'js-yaml';
 import { WrapperConfig, API_PROXY_PORTS, DockerComposeConfig } from './types';
 import { logger } from './logger';
 import { generatePolicyManifest, generateSquidConfig } from './squid-config';
+import { resolveTopologyPeerHosts } from './topology-peers';
 import { generateSessionCa, initSslDb, isOpenSslAvailable } from './ssl-bump';
 import { parseUrlPatterns } from './domain-matchers';
 import { SslConfig, SQUID_PORT } from './host-env';
@@ -331,6 +332,11 @@ export async function writeConfigs(config: WrapperConfig): Promise<void> {
       apiProxyIp: networkConfig.proxyIp,
       apiProxyPorts: Object.values(API_PROXY_PORTS),
     } : {}),
+    // Allow trusted topology peers (MCP gateway, DIFC/cli-proxy) on any port in
+    // network-isolation mode, for proxy clients that ignore NO_PROXY. DNS for
+    // these Docker-only names is provided via the squid-proxy extra_hosts patch
+    // (see patchComposeWithTopologyHosts in topology.ts).
+    topologyPeers: resolveTopologyPeerHosts(config),
   });
   const squidConfigPath = path.join(config.workDir, 'squid.conf');
   fs.writeFileSync(squidConfigPath, squidConfig, { mode: 0o644 });

src/squid-config-topology-peers.test.ts

@@ -0,0 +1,67 @@
+import { generateSquidConfig } from './squid-config';
+
+describe('generateSquidConfig: topology peers (network-isolation)', () => {
+  const port = 3128;
+
+  it('emits a dstdomain allow rule for each topology peer', () => {
+    const config = generateSquidConfig({
+      domains: ['github.com'],
+      port,
+      topologyPeers: ['awmg-mcpg', 'awmg-cli-proxy'],
+    });
+    expect(config).toContain('acl topology_peer_awmg_mcpg dstdomain .awmg-mcpg');
+    expect(config).toContain('http_access allow topology_peer_awmg_mcpg');
+    expect(config).toContain('acl topology_peer_awmg_cli_proxy dstdomain .awmg-cli-proxy');
+    expect(config).toContain('http_access allow topology_peer_awmg_cli_proxy');
+  });
+
+  it('places the peer allow before the Safe_ports deny (so any port is reachable)', () => {
+    const config = generateSquidConfig({
+      domains: ['github.com'],
+      port,
+      topologyPeers: ['awmg-mcpg'],
+    });
+    const allowPos = config.indexOf('http_access allow topology_peer_awmg_mcpg');
+    const safePortsDenyPos = config.indexOf('http_access deny !Safe_ports');
+    expect(allowPos).toBeGreaterThan(-1);
+    expect(safePortsDenyPos).toBeGreaterThan(-1);
+    expect(allowPos).toBeLessThan(safePortsDenyPos);
+  });
+
+  it('places the peer allow before the raw-IP deny rules', () => {
+    const config = generateSquidConfig({
+      domains: ['github.com'],
+      port,
+      topologyPeers: ['awmg-mcpg'],
+    });
+    const allowPos = config.indexOf('http_access allow topology_peer_awmg_mcpg');
+    const rawIpDenyPos = config.indexOf('http_access deny dst_ipv4');
+    expect(allowPos).toBeLessThan(rawIpDenyPos);
+  });
+
+  it('emits no topology peer rules when the list is empty or omitted', () => {
+    const withEmpty = generateSquidConfig({ domains: ['github.com'], port, topologyPeers: [] });
+    const withUndefined = generateSquidConfig({ domains: ['github.com'], port });
+    expect(withEmpty).not.toContain('topology_peer_');
+    expect(withUndefined).not.toContain('topology_peer_');
+  });
+
+  it('sanitizes peer names into safe ACL identifiers', () => {
+    const config = generateSquidConfig({
+      domains: ['github.com'],
+      port,
+      topologyPeers: ['mcp.gateway-01'],
+    });
+    expect(config).toContain('acl topology_peer_mcp_gateway_01 dstdomain .mcp.gateway-01');
+  });
+
+  it('rejects a peer name containing squid-config-breaking characters', () => {
+    expect(() =>
+      generateSquidConfig({
+        domains: ['github.com'],
+        port,
+        topologyPeers: ['evil\nhttp_access allow all'],
+      })
+    ).toThrow(/unsafe for Squid config/i);
+  });
+});

src/squid/config-generator.ts

@@ -32,7 +32,7 @@ const { version: AWF_VERSION } = require('../../package.json') as { version: str
  * // Blocked: internal.example.com -> acl blocked_domains dstdomain .internal.example.com
  */
 export function generateSquidConfig(config: SquidConfig): string {
-  const { domains, blockedDomains, port, sslBump, caFiles, sslDbPath, urlPatterns, enableHostAccess, allowHostPorts, enableDlp, dnsServers, upstreamProxy, apiProxyIp, apiProxyPorts } = config;
+  const { domains, blockedDomains, port, sslBump, caFiles, sslDbPath, urlPatterns, enableHostAccess, allowHostPorts, enableDlp, dnsServers, upstreamProxy, apiProxyIp, apiProxyPorts, topologyPeers } = config;
 
   validateApiProxyIp(apiProxyIp);
 
@@ -61,6 +61,7 @@ export function generateSquidConfig(config: SquidConfig): string {
     apiProxySection,
     allowedIpSection,
     dnsSection,
+    topologyPeersSection,
   } = buildConfigSections({
     enableDlp,
     port,
@@ -76,6 +77,7 @@ export function generateSquidConfig(config: SquidConfig): string {
     apiProxyPorts,
     apiProxyIp,
     dnsServers,
+    topologyPeers,
   });
 
   return `# Squid configuration for egress traffic control
@@ -119,7 +121,7 @@ acl localnet src 192.168.0.0/16
 acl localnet src fc00::/7
 acl localnet src fe80::/10
 
-${portAclsAndRules}
+${topologyPeersSection}${portAclsAndRules}
 ${apiProxySection}
 ${allowedIpSection}# Deny CONNECT to raw IP addresses (IPv4 and IPv6)
 # Prevents bypassing domain-based filtering via direct IP connections

src/squid/config-sections.ts

@@ -3,6 +3,7 @@ import { generateDlpSquidConfig } from '../dlp';
 import { DEFAULT_DNS_SERVERS } from '../dns-resolver';
 import { generateSslBumpSection } from './ssl-bump';
 import { validateAndSanitizeHostAccessPort, validateApiProxyPort } from './validation';
+import { formatDomainForSquid } from './domain-acl';
 
 type DomainsByProto = ReturnType<typeof import('./domain-acl').parseDomainConfig>['domainsByProto'];
 type PatternsByProto = ReturnType<typeof import('./domain-acl').parseDomainConfig>['patternsByProto'];
@@ -141,6 +142,37 @@ http_access allow from_api_proxy
 
 const IPV4_REGEX = /^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$/;
 
+/**
+ * Generate allow rules for trusted topology-peer hostnames in network-isolation
+ * (topology) mode.
+ *
+ * Docker-internal peers such as an MCP gateway (`awmg-mcpg`) are typically
+ * served on non-standard ports (e.g. 8080). The normal domain allowlist cannot
+ * reach them because `http_access deny !Safe_ports` (80/443) fires first. These
+ * peers are operator-attached and trusted (the same trust level as the direct
+ * NO_PROXY connection), so we emit an explicit `http_access allow` per peer
+ * *before* the Safe_ports and raw-IP deny rules, permitting any port.
+ *
+ * Scoped by dstdomain to the exact peer hostnames, so this does not widen access
+ * to arbitrary internet destinations.
+ */
+function generateTopologyPeersSection(topologyPeers?: string[]): string {
+  if (!topologyPeers || topologyPeers.length === 0) return '';
+
+  const lines = [
+    '# Allow trusted topology peers (network-isolation mode) on any port before',
+    '# the Safe_ports and raw-IP deny rules. These are operator-attached',
+    '# containers (e.g. an MCP gateway) served on non-standard ports; clients',
+    '# that honour HTTP(S)_PROXY but ignore NO_PROXY reach them through Squid.',
+  ];
+  for (const peer of topologyPeers) {
+    const aclName = `topology_peer_${peer.replace(/[^a-zA-Z0-9]/g, '_')}`;
+    lines.push(`acl ${aclName} dstdomain ${formatDomainForSquid(peer)}`);
+    lines.push(`http_access allow ${aclName}`);
+  }
+  return lines.join('\n') + '\n\n';
+}
+
 /**
  * Generate allow rules for raw IP addresses that appear in the allowed domains list.
  *
@@ -185,6 +217,7 @@ function generateConfigSections(options: {
   apiProxyPorts?: number[];
   apiProxyIp?: string;
   dnsServers?: string[];
+  topologyPeers?: string[];
 }): {
   dlpAclSection: string;
   dlpAccessSection: string;
@@ -195,6 +228,7 @@ function generateConfigSections(options: {
   apiProxySection: string;
   allowedIpSection: string;
   dnsSection: string;
+  topologyPeersSection: string;
 } {
   const {
     enableDlp,
@@ -211,6 +245,7 @@ function generateConfigSections(options: {
     apiProxyPorts,
     apiProxyIp,
     dnsServers,
+    topologyPeers,
   } = options;
 
   const { aclSection: dlpAclSection, accessSection: dlpAccessSection } = generateDlpSections(enableDlp);
@@ -226,6 +261,7 @@ function generateConfigSections(options: {
   const portAclsAndRules = generatePortAclsAndRules(enableHostAccess, allowHostPorts, apiProxyPorts);
   const apiProxySection = generateApiProxySection(apiProxyIp);
   const allowedIpSection = generateAllowedIpSection(domains ?? []);
+  const topologyPeersSection = generateTopologyPeersSection(topologyPeers);
   const dnsSection = generateDnsSection(dnsServers);
 
   return {
@@ -238,6 +274,7 @@ function generateConfigSections(options: {
     apiProxySection,
     allowedIpSection,
     dnsSection,
+    topologyPeersSection,
   };
 }
 

src/topology-peers.test.ts

@@ -0,0 +1,63 @@
+import { resolveTopologyPeerHosts } from './topology-peers';
+
+describe('resolveTopologyPeerHosts', () => {
+  it('returns topology-attach names in network-isolation + compose runtime', () => {
+    expect(
+      resolveTopologyPeerHosts({
+        networkIsolation: true,
+        topologyAttach: ['awmg-mcpg', 'awmg-cli-proxy'],
+      })
+    ).toEqual(['awmg-mcpg', 'awmg-cli-proxy']);
+  });
+
+  it('returns [] when network isolation is off', () => {
+    expect(
+      resolveTopologyPeerHosts({
+        networkIsolation: false,
+        topologyAttach: ['awmg-mcpg'],
+      })
+    ).toEqual([]);
+  });
+
+  it('returns [] for a non-compose runtime (e.g. sbx)', () => {
+    expect(
+      resolveTopologyPeerHosts({
+        networkIsolation: true,
+        containerRuntime: 'sbx',
+        topologyAttach: ['awmg-mcpg'],
+      })
+    ).toEqual([]);
+  });
+
+  it('includes the DIFC/cli-proxy host with scheme and port stripped', () => {
+    expect(
+      resolveTopologyPeerHosts({
+        networkIsolation: true,
+        difcProxyHost: 'https://awmg-cli-proxy:18443',
+      })
+    ).toEqual(['awmg-cli-proxy']);
+  });
+
+  it('deduplicates when the DIFC host is also in topologyAttach', () => {
+    expect(
+      resolveTopologyPeerHosts({
+        networkIsolation: true,
+        topologyAttach: ['awmg-cli-proxy'],
+        difcProxyHost: 'awmg-cli-proxy:18443',
+      })
+    ).toEqual(['awmg-cli-proxy']);
+  });
+
+  it('ignores empty / non-string topologyAttach entries', () => {
+    expect(
+      resolveTopologyPeerHosts({
+        networkIsolation: true,
+        topologyAttach: ['awmg-mcpg', '', '  ', 42 as unknown as string],
+      })
+    ).toEqual(['awmg-mcpg']);
+  });
+
+  it('returns [] when nothing is attached', () => {
+    expect(resolveTopologyPeerHosts({ networkIsolation: true, topologyAttach: [] })).toEqual([]);
+  });
+});