Skip to content

Commit b4fcc4e

Browse files
strimo378strimo378
committed
docs: add copilot container log investigation notes
1 parent ef2fe0f commit b4fcc4e

2 files changed

Lines changed: 255 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,11 @@ This repository isolates one question:
1313
It also contains a normal Actions workflow using the same container so regular
1414
Actions logs can be compared against Copilot session logs.
1515

16+
## Investigation Notes
17+
18+
Detailed background, observed behavior, tested hypotheses, and next steps are
19+
captured in [docs/copilot-container-log-investigation.md](docs/copilot-container-log-investigation.md).
20+
1621
## Included Workflows
1722

1823
- `.github/workflows/copilot-setup-steps.yml`
@@ -28,3 +33,15 @@ Actions logs can be compared against Copilot session logs.
2833
`notes.txt`.
2934
3. Check whether the Copilot session UI shows live output.
3035
4. Compare that with the normal Actions run logs from `container-smoke.yml`.
36+
37+
## Current Minimal Test Shape
38+
39+
- official GitHub-hosted runner: `ubuntu-latest`
40+
- explicit Ubuntu 24.04 container image:
41+
`mcr.microsoft.com/devcontainers/base:ubuntu-24.04`
42+
- no extra `container.options`
43+
- no custom `HOME`
44+
- no extra user/capability/mount overrides
45+
46+
This keeps the repro intentionally close to the smallest possible
47+
container-enabled Copilot setup.

‎docs/copilot-container-log-investigation.md‎

Lines changed: 238 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,238 @@
1+
# Copilot Container Log Investigation
2+
3+
This document records the current understanding behind this reproduction
4+
repository.
5+
6+
## Goal
7+
8+
Determine whether GitHub Copilot cloud agent sessions lose visible session
9+
output when `.github/workflows/copilot-setup-steps.yml` uses a real job
10+
`container:` on an official GitHub-hosted runner.
11+
12+
## Why This Repo Exists
13+
14+
A larger production repository was made to work with Copilot plus a job
15+
container, but remote Copilot agent sessions showed no visible output in the
16+
session UI even though the session itself started and ran.
17+
18+
That larger setup had additional moving parts:
19+
20+
- a custom GHCR image
21+
- a Blacksmith runner instead of `ubuntu-latest`
22+
- custom container `options`
23+
- bind mounts under `/home/runner`
24+
- attempts to emulate GitHub runner assumptions inside the container
25+
26+
This repo strips the scenario down to the smallest practical version so the
27+
question can be tested in isolation on official GitHub-hosted infrastructure.
28+
29+
## Minimal Repro Shape
30+
31+
Current workflow choices:
32+
33+
- `runs-on: ubuntu-latest`
34+
- `container.image: mcr.microsoft.com/devcontainers/base:ubuntu-24.04`
35+
- no extra `container.options`
36+
- no custom environment overrides
37+
- one setup step that prints simple diagnostics
38+
39+
The normal comparison workflow `container-smoke.yml` uses the same runner and
40+
container so standard Actions logging can be compared against Copilot session
41+
logging.
42+
43+
## Observations From The Original Investigation
44+
45+
The original repository reached these states successfully:
46+
47+
- container initialization succeeded
48+
- Copilot preparation succeeded
49+
- firewall validation succeeded
50+
- MCP server startup succeeded
51+
- user-configured setup steps all succeeded inside the container
52+
53+
The problematic transition happened later, when the job entered:
54+
55+
- `Processing Request (Linux)`
56+
57+
At that point, the Copilot session UI showed no useful live output.
58+
59+
## Concrete Run Evidence
60+
61+
Observed in the original repository:
62+
63+
- workflow run: `24592941055`
64+
- job: `71917366754`
65+
- workflow name: `Running Copilot cloud agent`
66+
- branch: `copilot/fix-reduced-test-case`
67+
68+
Important observation:
69+
70+
- GitHub's Actions job-logs endpoint for that job returned `404 BlobNotFound`
71+
while the job was in progress.
72+
73+
That strongly suggests the problem is not only that the session UI is empty.
74+
It suggests the underlying log materialization/persistence path may already be
75+
broken or missing for the Copilot processing phase.
76+
77+
## What Was Tried In The Original Repository
78+
79+
Several hypotheses were tested in the larger repository.
80+
81+
### 1. Broad `/home/runner` bind mount
82+
83+
Reasoning:
84+
85+
- Copilot helper scripts appeared to resolve paths through
86+
`/home/runner/_work/_temp/...` even with a job container active.
87+
88+
What was tried:
89+
90+
- bind-mounting `/home/runner:/home/runner`
91+
92+
Outcome:
93+
94+
- the containerized setup ran, but Copilot session output was still missing
95+
96+
### 2. Runner-like container environment
97+
98+
Reasoning:
99+
100+
- perhaps Copilot expected specific runner-like assumptions inside the image
101+
102+
What was tried:
103+
104+
- adding a `runner` user
105+
- setting runner-like working directory behavior
106+
- adding `iptables`
107+
- setting `CONTAINER=true`
108+
- adjusting image defaults
109+
110+
Outcome:
111+
112+
- no confirmed recovery of session output
113+
114+
### 3. UID / HOME / capability tuning
115+
116+
Reasoning:
117+
118+
- maybe temp files or log artifacts were created with the wrong ownership or
119+
helper processes needed extra privileges
120+
121+
What was tried:
122+
123+
- `--user 1001:1001`
124+
- `HOME=/home/runner`
125+
- `--cap-add=NET_ADMIN`
126+
- `--cap-add=NET_RAW`
127+
128+
Outcome:
129+
130+
- no confirmed recovery of session output
131+
132+
### 4. Narrower mount instead of full `/home/runner`
133+
134+
Reasoning:
135+
136+
- a full bind mount may shadow too much of the hosted runner environment
137+
138+
What was tried:
139+
140+
- switching from `/home/runner:/home/runner` to a narrower `_work` mount
141+
142+
Outcome:
143+
144+
- still no confirmed recovery of session output
145+
146+
## Why The Dockerfile Is Probably Not The Main Issue
147+
148+
Public repositories exist whose `copilot-setup-steps.yml` uses a real
149+
`container:` without any obvious Copilot-specific Dockerfile changes.
150+
151+
Examples observed during investigation:
152+
153+
- `justin/dotfiles`
154+
- `raynigon/raylevation`
155+
- `UCLA-PHP/school.epi.abm`
156+
- `bitcoin-sv/bitcoin-sv`
157+
- `intel/torch-xpu-ops`
158+
- `talitahalboth/wip-tests`
159+
160+
Several of those workflows use simple off-the-shelf images or very light custom
161+
images. That weakens the hypothesis that a special package, entrypoint, or
162+
runner bootstrap inside the Dockerfile is required for Copilot session logs.
163+
164+
## Why This Repo Uses An Explicit Ubuntu 24.04 Image
165+
166+
This repo intentionally uses:
167+
168+
- `mcr.microsoft.com/devcontainers/base:ubuntu-24.04`
169+
170+
instead of a generic `ubuntu` tag so the test is pinned to a concrete Ubuntu
171+
24.x base.
172+
173+
This keeps the reproduction closer to current GitHub-hosted Ubuntu behavior
174+
without introducing a custom image build.
175+
176+
## Current Working Theory
177+
178+
The most likely interpretations at this point are:
179+
180+
1. Copilot cloud agent session logging has a GitHub-side defect for at least
181+
some containerized setups.
182+
2. A job `container:` may still be partially unsupported or fragile for the
183+
session-log path even when setup steps themselves run successfully.
184+
3. The issue is less likely to be caused by a missing package in the image and
185+
more likely to be caused by an incompatibility in GitHub's internal Copilot
186+
processing/logging path.
187+
188+
## Relevant External References
189+
190+
Useful references gathered during investigation:
191+
192+
- GitHub Docs, customize the agent environment:
193+
`https://docs.github.com/en/copilot/how-tos/use-copilot-agents/cloud-agent/customize-the-agent-environment`
194+
- GitHub Docs, troubleshoot the cloud agent:
195+
`https://docs.github.com/en/copilot/how-tos/use-copilot-agents/cloud-agent/troubleshoot-cloud-agent`
196+
- GitHub changelog entry dated March 19, 2026 about more visibility into
197+
Copilot coding agent sessions:
198+
`https://github.blog/changelog/2026-03-19-more-visibility-into-copilot-coding-agent-sessions/`
199+
200+
The March 19, 2026 changelog is especially relevant because it explicitly
201+
states that setup-step output should be visible in session logs.
202+
203+
## How To Use This Repo
204+
205+
Recommended test flow:
206+
207+
1. Start a Copilot coding agent session on this repository.
208+
2. Ask the agent to make a tiny visible change, ideally in `notes.txt`.
209+
3. Observe whether the session UI shows live output.
210+
4. Run `container-smoke.yml` manually and compare the normal Actions logs.
211+
212+
## Suggested Next A/B Tests
213+
214+
If this minimal repro still shows no Copilot session output:
215+
216+
1. Remove `container:` entirely and confirm whether Copilot logs appear.
217+
2. Swap the container image to another known public image while keeping the
218+
workflow otherwise identical.
219+
3. Test a plain `ubuntu:24.04` image.
220+
4. Test a devcontainer base image from a different family.
221+
222+
The key principle is to vary one axis at a time:
223+
224+
- with container vs without container
225+
- image A vs image B
226+
- no extra options vs one extra option
227+
228+
## Interpretation Guidance
229+
230+
If Copilot session logs are missing here too, despite:
231+
232+
- official runner
233+
- tiny workflow
234+
- simple public image
235+
- no extra container options
236+
237+
then the evidence against "your custom repo setup is the cause" becomes much
238+
stronger.

0 commit comments

Comments
 (0)