Skip to content

Commit 1d6b151

Browse files
authored
windows support (#19)
* update SSH testing: improve CI environment compatibility with mock fallback, enhance macOS SSH configuration, and add robust connection retries * update SSH testing: improve CI environment compatibility (#18) * update SSH testing: improve CI environment compatibility with mock fallback, enhance macOS SSH configuration, and add robust connection retries * update tests and workflows: make SSH username dynamic in tests and add concurrency control to GitHub Actions * update workflows: simplify device serial description in bench workflow inputs * update README: revise supported platforms table with expanded host-device compatibility details * update README: revise supported platforms table with expanded host-device compatibility details * refactor tests and core utilities: use `datetime.now(timezone.utc)` for consistent timestamping, replace `subprocess.Popen` with `subprocess.run` for simpler command execution, enhance filesystem utility robustness, and improve cross-platform path handling * refactor tests and core utilities: remove unnecessary whitespace, simplify `Mock` return values in subprocess tests, standardize string quotation style, and update README with additional platform compatibility * update SSH setup and testing: add Windows support with PowerShell script, refactor setup script generation for platform detection, and improve CI workflow integration * refactor tests: remove unnecessary whitespace in SSH-related test files for improved readability and standardization * refactor SSH setup: simplify script for CI environments by removing OpenSSH Server installation and configuration logic, streamline SSH connection testing, and improve logging for compatibility checks * enhance CLI and CI encoding handling: add UTF-8 defaults for Windows, improve interactive and CI pipeline behavior, and enforce strict error handling in SSH setup and device tests * remove unnecessary whitespace: clean up extra blank lines and trailing spaces in `cli.py` for improved readability and code style consistency * handle exceptions explicitly: replace bare `except` with `except Exception` in `cli.py` for clearer and safer error handling * refactor SSH tests and setup: simplify platform checks, enhance logging for failed SSH connections, streamline macOS SSH daemon startup logic, and add detailed diagnostics for CI environments * refactor SSH tests and setup: simplify platform checks, enhance logging for failed SSH connections, streamline macOS SSH daemon startup logic, and add detailed diagnostics for CI environments * remove redundant platform checks: simplify `test_ssh_in_ci` by removing unused platform-specific logic * update SSH testing: remove unused variables in test scripts for cleanup and better maintainability * add shell specification for SSH commands in CI workflow for consistency and compatibility * install and configure OpenSSH Server: enhance CI SSH setup with server installation, configuration updates, improved key handling, and robust connection testing * clean up `test_ssh_in_ci`: adjust formatting for consistent style and readability * update SSH configurations and tests: replace `localhost` with `127.0.0.1` for consistency, remove unused experiment files, and enhance `test_ssh_in_ci` with Paramiko test server integration * validate SSH device config: ensure `host` is specified in device configuration and raise `ConfigError` if missing; minor imports adjustment * format SSH code and tests: adjust indentation, replace single quotes with double quotes for consistency, and apply minor cleanup for improved readability * add platform-specific logic to SSH tests: update `test_main_setup` and `test_main_all` to handle Windows and Unix separately, ensuring platform compatibility in script generation * refactor SSH tests: adjust formatting in `test_generate_ssh_config` for consistent style and readability * remove SSH localhost experiments and enhance mock mode: delete unused `ssh_localhost_ci.yaml`, refactor SSH device logic with `mock_mode` for dry-runs, and simplify SSH CI testing with Paramiko test server integration * remove generate_ssh_config.py and associated files: delete unused script, configuration, and test files related to SSH config generation for CI
1 parent 22232e5 commit 1d6b151

28 files changed

Lines changed: 648 additions & 943 deletions

.github/codecov.yml

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,6 @@ ignore:
44
- "**/__pycache__"
55
- "**/*.pyc"
66
- "setup.py"
7-
- "scripts/generate_ssh_config.py"
8-
- "scripts/test_ssh_device_ci.py"
97

108
coverage:
119
status:

.github/workflows/bench.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ jobs:
2020
ci-matrix:
2121
strategy:
2222
matrix:
23-
os: [ubuntu-latest, macos-latest] # , windows-latest]
23+
os: [ubuntu-latest, macos-latest, windows-latest]
2424
uses: ./.github/workflows/ci-orchestrator.yml
2525
with:
2626
os: ${{ matrix.os }}

.github/workflows/stage-device-tests.yml

Lines changed: 10 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -28,24 +28,20 @@ jobs:
2828
pip install -r requirements.txt
2929
pip install -e .
3030
31-
- name: Set up SSH server
31+
- name: Test SSH with Paramiko test server
32+
shell: bash
3233
run: |
33-
python scripts/generate_ssh_config.py --type setup
34-
bash scripts/setup_ssh_ci.sh || echo "SSH setup had warnings, continuing..."
35-
36-
- name: List SSH devices
37-
run: |
38-
ovmobilebench list-ssh-devices || echo "Command not yet implemented"
39-
40-
- name: Test SSH deployment
41-
run: |
42-
python scripts/generate_ssh_config.py --type test
43-
python scripts/test_ssh_device_ci.py
34+
set -e
35+
python tests/test_ssh_device_ci.py
4436
4537
- name: Run benchmark dry-run via SSH
38+
shell: bash
39+
env:
40+
PYTHONIOENCODING: utf-8
41+
PYTHONUTF8: 1
4642
run: |
47-
python scripts/generate_ssh_config.py --type config
48-
ovmobilebench all -c experiments/ssh_localhost_ci.yaml --dry-run || true
43+
set -e
44+
ovmobilebench all -c experiments/ssh_test_ci.yaml --dry-run
4945
5046
- name: Upload SSH test results
5147
if: always()

.gitignore

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -131,7 +131,5 @@ dmypy.json
131131
.claude
132132
CLAUDE.md
133133

134-
# Generated CI configs
135-
experiments/ssh_localhost_ci.yaml
134+
# Test results
136135
experiments/results/
137-
scripts/setup_ssh_ci.sh

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,7 @@ cat experiments/results/*.csv
5656
| Windows | x86_64 | Android | ARM64 | ADB | adbutils | ✅ Stable |
5757
| Linux | x86_64 | Linux | ARM64/ARM32 | SSH | paramiko | ✅ Stable |
5858
| macOS | x86_64/ARM64 | Linux | ARM64/ARM32 | SSH | paramiko | ✅ Stable |
59+
| Windows | x86_64 | Linux | ARM64/ARM32 | SSH | paramiko | ✅ Stable |
5960
| Any | Any | iOS | ARM64 | USB | - | 🚧 Planned |
6061

6162
## 📋 Requirements

docs/ci-cd.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -613,7 +613,7 @@ def collect_metrics(results_path, metadata):
613613
def send_to_influxdb(metrics):
614614
from influxdb import InfluxDBClient
615615
616-
client = InfluxDBClient('localhost', 8086, database='ovmobilebench')
616+
client = InfluxDBClient('127.0.0.1', 8086, database='ovmobilebench')
617617
618618
points = []
619619
for result in metrics['results']:

experiments/ssh_localhost.yaml

Lines changed: 0 additions & 44 deletions
This file was deleted.

experiments/ssh_test_ci.yaml

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# SSH test configuration for CI
2+
project:
3+
name: ssh-test-ci
4+
run_id: ci-test
5+
6+
# SSH device configuration for mock testing
7+
device:
8+
type: linux_ssh
9+
host: 127.0.0.1
10+
username: testuser
11+
push_dir: /tmp/ovmobilebench
12+
13+
# Build configuration (disabled for CI)
14+
build:
15+
enabled: false
16+
openvino_repo: /tmp/openvino
17+
18+
# Dummy models for testing
19+
models:
20+
- name: dummy_model
21+
path: /tmp/dummy_model.xml
22+
precision: FP32
23+
24+
# Run configuration
25+
run:
26+
repeats: 1
27+
warmup: 0
28+
cooldown_sec: 0
29+
matrix:
30+
niter: [10]
31+
nstreams: ["1"]
32+
device: ["CPU"]
33+
34+
# Reporting
35+
report:
36+
sinks:
37+
- type: csv
38+
path: experiments/results/ssh_test.csv
39+
- type: json
40+
path: experiments/results/ssh_test.json
41+
tags:
42+
test_type: ssh_test_ci
43+
ci: true

ovmobilebench/cli.py

Lines changed: 61 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,8 @@
33
# Apply typer compatibility patch
44
from ovmobilebench import typer_patch # noqa: F401
55

6+
import os
7+
import sys
68
import typer
79
from pathlib import Path
810
from typing import Optional
@@ -12,14 +14,27 @@
1214
from ovmobilebench.config.loader import load_experiment
1315
from ovmobilebench.pipeline import Pipeline
1416

17+
# Set UTF-8 encoding for Windows
18+
if sys.platform == "win32":
19+
os.environ["PYTHONIOENCODING"] = "utf-8"
20+
# Also set console code page to UTF-8 if possible
21+
try:
22+
import subprocess
23+
24+
subprocess.run("chcp 65001", shell=True, capture_output=True)
25+
except Exception:
26+
pass
27+
1528
app = typer.Typer(
1629
name="ovmobilebench",
1730
help="End-to-end benchmarking pipeline for OpenVINO on mobile devices",
1831
add_completion=False,
1932
pretty_exceptions_enable=False, # Disable pretty exceptions
2033
rich_markup_mode=None, # Disable Rich formatting
2134
)
22-
console = Console()
35+
36+
# Configure console with safe encoding for Windows
37+
console = Console(legacy_windows=True if sys.platform == "win32" else None)
2338

2439

2540
@app.command()
@@ -33,7 +48,7 @@ def build(
3348
cfg = load_experiment(config)
3449
pipeline = Pipeline(cfg, verbose=verbose, dry_run=dry_run)
3550
pipeline.build()
36-
console.print("[bold green] Build completed[/bold green]")
51+
console.print("[bold green][OK] Build completed[/bold green]")
3752

3853

3954
@app.command()
@@ -47,7 +62,7 @@ def package(
4762
cfg = load_experiment(config)
4863
pipeline = Pipeline(cfg, verbose=verbose, dry_run=dry_run)
4964
pipeline.package()
50-
console.print("[bold green] Package created[/bold green]")
65+
console.print("[bold green][OK] Package created[/bold green]")
5166

5267

5368
@app.command()
@@ -61,7 +76,7 @@ def deploy(
6176
cfg = load_experiment(config)
6277
pipeline = Pipeline(cfg, verbose=verbose, dry_run=dry_run)
6378
pipeline.deploy()
64-
console.print("[bold green] Deployment completed[/bold green]")
79+
console.print("[bold green][OK] Deployment completed[/bold green]")
6580

6681

6782
@app.command()
@@ -77,7 +92,7 @@ def run(
7792
cfg = load_experiment(config)
7893
pipeline = Pipeline(cfg, verbose=verbose, dry_run=dry_run)
7994
pipeline.run(timeout=timeout, cooldown=cooldown)
80-
console.print("[bold green] Benchmarks completed[/bold green]")
95+
console.print("[bold green][OK] Benchmarks completed[/bold green]")
8196

8297

8398
@app.command()
@@ -90,7 +105,7 @@ def report(
90105
cfg = load_experiment(config)
91106
pipeline = Pipeline(cfg, verbose=verbose)
92107
pipeline.report()
93-
console.print("[bold green] Reports generated[/bold green]")
108+
console.print("[bold green][OK] Reports generated[/bold green]")
94109

95110

96111
@app.command()
@@ -102,11 +117,10 @@ def all(
102117
cooldown: Optional[int] = typer.Option(None, "--cooldown", help="Cooldown between runs"),
103118
):
104119
"""Execute complete pipeline: build, package, deploy, run, and report."""
105-
with Progress(
106-
SpinnerColumn(),
107-
TextColumn("[progress.description]{task.description}"),
108-
console=console,
109-
) as progress:
120+
# Check if we're in CI environment
121+
is_ci = os.environ.get("CI", "").lower() == "true"
122+
123+
try:
110124
cfg = load_experiment(config)
111125
pipeline = Pipeline(cfg, verbose=verbose, dry_run=dry_run)
112126

@@ -118,16 +132,42 @@ def all(
118132
("Generating reports...", pipeline.report),
119133
]
120134

121-
for description, stage_func in stages:
122-
task = progress.add_task(description, total=None)
123-
try:
124-
stage_func()
125-
progress.update(task, completed=True)
126-
except Exception as e:
127-
console.print(f"[bold red]✗ {description} failed: {e}[/bold red]")
128-
raise
129-
130-
console.print("[bold green]✓ Pipeline completed successfully[/bold green]")
135+
if is_ci or verbose:
136+
# Simple output for CI or verbose mode
137+
for description, stage_func in stages:
138+
print(f"[*] {description}")
139+
try:
140+
stage_func()
141+
print(f"[OK] {description} completed")
142+
except Exception as e:
143+
print(f"[FAIL] {description} failed: {e}")
144+
raise
145+
print("[OK] Pipeline completed successfully")
146+
else:
147+
# Rich progress bar for interactive use
148+
spinner = SpinnerColumn(spinner_name="dots" if sys.platform == "win32" else "aesthetic")
149+
150+
with Progress(
151+
spinner,
152+
TextColumn("[progress.description]{task.description}"),
153+
console=console,
154+
transient=True, # Clear progress when done
155+
) as progress:
156+
for description, stage_func in stages:
157+
task = progress.add_task(description, total=None)
158+
try:
159+
stage_func()
160+
progress.update(task, completed=True)
161+
except Exception as e:
162+
console.print(f"[bold red][FAIL] {description} failed: {e}[/bold red]")
163+
raise
164+
165+
console.print("[bold green][OK] Pipeline completed successfully[/bold green]")
166+
except UnicodeEncodeError as e:
167+
# Fallback for encoding errors
168+
print(f"Encoding error: {e}")
169+
print("Pipeline failed due to encoding issues. Try setting PYTHONIOENCODING=utf-8")
170+
sys.exit(1)
131171

132172

133173
@app.command()

ovmobilebench/config/loader.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
def load_yaml(path: Path) -> Dict[str, Any]:
1010
"""Load YAML configuration file."""
1111
if not path.exists():
12-
raise FileNotFoundError(f"Configuration file not found: {path}")
12+
raise FileNotFoundError(f"Configuration file not found: {path.as_posix()}")
1313

1414
with open(path, "r") as f:
1515
data: Dict[str, Any] = yaml.safe_load(f)

0 commit comments

Comments
 (0)