Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
9ec1642
feat: add openadapt-viewer dependency and adapter module (Phase 1)
abrichr Jan 17, 2026
a168d7e
feat: add workflow segmentation system with capture adapter
abrichr Jan 17, 2026
a96aa5c
Enhance vm monitor command with comprehensive VM usage visibility
abrichr Jan 17, 2026
8cc8711
Refactor segmentation pipeline to use screen.frame events
abrichr Jan 17, 2026
5a4dac8
Add VM monitoring dashboard with comprehensive usage visibility
abrichr Jan 17, 2026
e32b235
Add segmentation testing documentation and test files
abrichr Jan 17, 2026
5884cdf
Document archived OpenAdapter repository
abrichr Jan 18, 2026
c880f1e
Add search functionality to training viewer
abrichr Jan 18, 2026
b4caa01
fix: resolve ruff linting and formatting issues
abrichr Jan 19, 2026
736e3d8
fix: resolve test failures from missing dependencies
abrichr Jan 19, 2026
f78c55a
feat(dashboard): add Azure ops dashboard with live VNC embed
abrichr Jan 20, 2026
d5b2dc2
fix: auto-cleanup before Docker builds, use VERSION=11 for unattended…
abrichr Jan 20, 2026
38e9509
fix(waa): use VERSION=11e consistently to prevent product key prompt
abrichr Jan 21, 2026
4e0b2a4
docs: fix inverted VERSION documentation in CLAUDE.md
abrichr Jan 21, 2026
8fe7e6f
feat: automate vanilla WAA bootstrap
abrichr Jan 22, 2026
fd11326
docs: clarify unattended WAA bootstrap
abrichr Jan 22, 2026
914513e
fix(waa): don't replace dockurr/windows autounattend.xml
abrichr Jan 22, 2026
72c800b
refactor(cli): remove deprecated WAA handlers, add auto-cleanup
abrichr Jan 22, 2026
b64149c
docs: add CLI-first and VM workflow guidelines to CLAUDE.md
abrichr Jan 22, 2026
95e96a1
docs: fix markdown formatting in waa_vanilla_automation.md
abrichr Jan 22, 2026
6b9f744
fix(waa): unattended Windows installation now works
abrichr Jan 23, 2026
8ae35f1
fix(waa): fix unattended installation and benchmark execution
abrichr Jan 23, 2026
50d12b1
fix(dashboard): use Azure CLI for live VM IP, fix activity detection
abrichr Jan 24, 2026
3482349
Merge branch 'main' into feat/waa-vanilla-automation
abrichr Jan 24, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
601 changes: 530 additions & 71 deletions CLAUDE.md

Large diffs are not rendered by default.

39 changes: 39 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -827,6 +827,45 @@ uv run python -m openadapt_ml.benchmarks.cli vm monitor --auto-shutdown-hours 2

For complete VM management commands and Azure setup instructions, see [`CLAUDE.md`](CLAUDE.md) and [`docs/azure_waa_setup.md`](docs/azure_waa_setup.md).

### 13.5 Screenshot Capture Tool

Capture screenshots of dashboards and VMs for documentation and PR purposes:

```bash
# Capture all available targets
uv run python -m openadapt_ml.benchmarks.cli screenshot

# List available targets
uv run python -m openadapt_ml.benchmarks.cli screenshot --list

# Capture specific targets
uv run python -m openadapt_ml.benchmarks.cli screenshot --target terminal
uv run python -m openadapt_ml.benchmarks.cli screenshot --target azure-ops --target vnc

# Custom output directory
uv run python -m openadapt_ml.benchmarks.cli screenshot --output /path/to/screenshots

# Without timestamp in filename
uv run python -m openadapt_ml.benchmarks.cli screenshot --target terminal --no-timestamp
```

**Available targets:**

| Target | Description |
|--------|-------------|
| `azure-ops` | Azure ops dashboard (localhost:8765) |
| `vnc` | VNC viewer (localhost:8006) - Windows VM |
| `terminal` | VM monitor terminal output (mock mode) |
| `terminal-live` | VM monitor terminal output (live, requires running VM) |
| `training` | Training dashboard (localhost:8080) |
| `vm-screen` | Windows VM screen capture via QEMU |

**Notes:**
- Terminal screenshots use PIL to render terminal output as PNG images
- Web page screenshots work best with playwright installed (`uv add playwright && playwright install chromium`)
- On macOS, interactive capture using `screencapture` is available as a fallback
- Screenshots are saved to `docs/screenshots/` by default with timestamps

---

## 14. Limitations & Notes
Expand Down
26 changes: 26 additions & 0 deletions deprecated/Dockerfile.simple
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
FROM dockurr/windows:latest

RUN apt-get update && apt-get install -y fuse dos2unix wget curl python3 python3-pip git && rm -rf /var/lib/apt/lists/*

ENV YRES="900"
ENV XRES="1440"
ENV RAM_SIZE="8G"
ENV CPU_CORES="8"
ENV VERSION="11e"
ENV DISK_SIZE="30G"
ENV ARGUMENTS="-qmp tcp:0.0.0.0:7200,server,nowait"

COPY src/win-arena-container/client /client
COPY src/win-arena-container/vm/setup /oem
COPY src/win-arena-container/entry.sh /entry.sh
COPY src/win-arena-container/entry_setup.sh /entry_setup.sh
COPY src/win-arena-container/start_client.sh /start_client.sh
COPY src/win-arena-container/start_vm.sh /start_vm.sh

RUN pip3 install --no-cache-dir -r /client/requirements.txt 2>/dev/null || true

RUN find / -maxdepth 3 -type f -name "*.sh" -exec dos2unix {} \; 2>/dev/null; chmod +x /*.sh 2>/dev/null || true

RUN sed -i "s|20\.20\.20\.21|172.30.0.2|g" /entry_setup.sh /entry.sh /start_client.sh 2>/dev/null || true

ENTRYPOINT ["/bin/bash", "-c"]
7 changes: 7 additions & 0 deletions deprecated/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Deprecated WAA Legacy Materials

This folder contains legacy WAA automation files and documents that are no longer
part of the vanilla WAA workflow. They are retained for review only.

Use `docs/waa_vanilla_automation.md` and the scripts in `scripts/` for the
current vanilla automation flow.
87 changes: 87 additions & 0 deletions deprecated/docs/WAA_ACR_DESIGN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# WAA ACR Design (Unattended + Vanilla)

## Goals
- Make WAA image pulls reliable (no Docker Hub throttling/timeouts).
- Preserve unattended Windows install (no license prompts).
- Use existing CLI/scripts wherever possible.

## Constraints
- Windows install must be fully unattended (VERSION=11e + OEM Azure mode).
- Prefer vanilla WAA components; no dev-mode UNC paths.
- Avoid custom tooling if existing commands cover the flow.

## Proposed ACR Naming
ACR names must be globally unique and <= 50 chars. Use a deterministic pattern tied to the subscription and region:

```
openadapt-evals-<region>-<suffix>
```

Suggested suffix: last 6 chars of the Azure subscription ID.

Example (eastus + sub id ...1234ab):

```
openadapt-evals-eastus-1234ab
```

If name is taken, append `-01`, `-02`, etc.

## Implementation Plan

### 1) Create ACR + import WinArena image
Use the existing helper script in `openadapt-evals`:

```bash
cd /Users/abrichr/oa/src/openadapt-evals
./scripts/setup_acr.sh \
--acr-name openadapt-evals-eastus-1234ab \
--resource-group openadapt-agents \
--workspace openadapt-ml \
--location eastus
```

This script:
- Creates the registry (Basic tier).
- Imports `docker.io/windowsarena/winarena:latest`.
- Grants `AcrPull` to the Azure ML workspace identity.

### 2) Use ACR image for Azure ML runs
No new code needed; use the existing config/env support:

```bash
export AZURE_DOCKER_IMAGE="openadapt-evals-eastus-1234ab.azurecr.io/winarena:latest"
```

Then run Azure evals as usual:

```bash
uv run python -m openadapt_evals.benchmarks.cli azure --workers 1 --task-ids notepad_1 --waa-path /path/to/WAA
```

### 3) Use ACR image for dedicated VM builds
The VM flow already supports ACR via existing CLI commands:

```bash
uv run python -m openadapt_ml.benchmarks.cli vm pull-image --acr openadapt-evals-eastus-1234ab
```

When building the custom `waa-auto` image on the VM, set:

```bash
export WAA_SOURCE_IMAGE="openadapt-evals-eastus-1234ab.azurecr.io/winarena:latest"
uv run python -m openadapt_ml.benchmarks.cli vm prepare-windows
```

This uses the simplified Dockerfile (OEM Azure mode) and keeps installs unattended.

## Verification Checklist
- ACR import succeeded (`az acr repository show --name <acr> --repository winarena`).
- Azure ML run logs show pulls from the ACR login server.
- VM `prepare-windows` completes without product key prompts.
- WAA `/probe` endpoint responds on port 5000 after boot.

## Notes
- The simplified Dockerfile copies OEM assets from the source image and uses `VERSION=11e` for unattended installs.
- If Windows prompts for a product key, treat it as a regression and follow `docs/RECURRING_ISSUES.md`.
- Keep Azure ML and ACR in the same region to avoid throttling and reduce pull time.
Loading
Loading