Skip to content

Commit 1ca2b4a

Browse files
Copilotmattleibowakoeplinger
authored
Add copilot instructions for XHarness repository (#1428)
This PR adds comprehensive GitHub Copilot instructions to help AI assistants understand and work effectively with the XHarness codebase. The instructions are located at `.github/copilot-instructions.md` as specified in the GitHub documentation. ## What's Included The copilot instructions provide detailed guidance on: ### Project Overview & Architecture - XHarness as a .NET command-line tool for cross-platform testing - Two-layer architecture (tooling and application layers) - Platform support: Android, Apple (iOS/tvOS/watchOS/xrOS/Mac Catalyst), WASM, WASI ### Technical Knowledge - Complete command structure: `xharness [platform] [command] [options]` - All supported platform commands (Android, Apple, WASM, WASI) - Standardized exit codes and error handling patterns - System requirements and build procedures - Key dependencies (ADB, mlaunch, Helix SDK) ### Development Guidelines - Platform-specific development knowledge - Testing strategy (unit, integration, E2E tests) - Common implementation patterns for commands - Environmental variables and configuration options ### Operational Knowledge - Repository structure and file organization - Troubleshooting guidelines for common issues - Development workflow for bug fixes and new features - Code quality standards and best practices ### Self-Improvement Mechanism - Instructions for autonomous updates when discovering new knowledge - Emphasis on continuous improvement for effective operation - Guidelines for maintaining accuracy and documentation structure ## Purpose These instructions enable AI assistants to: - Understand the XHarness architecture and purpose - Navigate the codebase effectively - Follow established patterns and conventions - Implement fixes and features autonomously - Troubleshoot common issues - Maintain and improve the instructions over time The goal is to create an autonomous agent capable of solving issues, fixing problems, and adding new features while maintaining code quality and project standards. Fixes #1427. Co-authored-by: mattleibow <1096616+mattleibow@users.noreply.github.com> Co-authored-by: akoeplinger <1376924+akoeplinger@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
1 parent 4e9a2ae commit 1ca2b4a

2 files changed

Lines changed: 259 additions & 0 deletions

File tree

‎.github/copilot-instructions.md‎

Lines changed: 225 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,225 @@
1+
# XHarness Repository Copilot Instructions
2+
3+
## Project Overview
4+
5+
XHarness is a .NET command-line tool that enables running xUnit/NUnit tests on mobile platforms (Android, Apple iOS/tvOS/watchOS/xrOS/Mac Catalyst), WASI, and desktop browsers (WASM). It is part of the .NET ecosystem and is essential for cross-platform testing in the .NET Foundation projects.
6+
7+
**Key Capabilities:**
8+
- Device/emulator management and discovery
9+
- Application lifecycle management (install, run, uninstall)
10+
- Test execution and result collection in multiple formats (text, xUnit/NUnit XML)
11+
- Crash dump collection and symbolication
12+
- TCP and USB connection modes
13+
- Apple Simulator runtime installation
14+
- Integration with Helix cloud testing infrastructure
15+
16+
## Architecture Overview
17+
18+
XHarness is organized into two main layers:
19+
20+
### Tooling Layer (src/)
21+
- **Microsoft.DotNet.XHarness.CLI** - Main CLI entry point with command definitions
22+
- **Microsoft.DotNet.XHarness.Android** - Android-specific operations using ADB
23+
- **Microsoft.DotNet.XHarness.Apple** - Apple platform operations using mlaunch
24+
- **Microsoft.DotNet.XHarness.iOS.Shared** - Apple mobile platforms shared functionality
25+
- **Microsoft.DotNet.XHarness.Common** - Core building blocks (logging, execution, utilities, diagnostics)
26+
27+
### Application Layer (src/)
28+
- **Microsoft.DotNet.XHarness.TestRunners.Common** - Test discovery, execution, and results aggregation
29+
- **Microsoft.DotNet.XHarness.TestRunners.Xunit** - XUnit framework integration
30+
- **Microsoft.DotNet.XHarness.TestRunners.NUnit** - NUnit framework integration
31+
- **Microsoft.DotNet.XHarness.DefaultAndroidEntryPoint.Xunit** - Default Android entry point
32+
33+
## Command Structure
34+
35+
XHarness follows a platform → command pattern:
36+
```bash
37+
xharness [platform] [command] [options]
38+
```
39+
40+
### Supported Platforms and Commands:
41+
42+
**Android Commands:**
43+
- `AndroidTest`, `AndroidDevice`, `AndroidInstall`, `AndroidRun`, `AndroidUninstall`, `AndroidAdb`, `AndroidState`
44+
45+
**Apple Commands:**
46+
- `AppleTest`, `AppleRun`, `AppleInstall`, `AppleUninstall`, `AppleJustTest`, `AppleJustRun`, `AppleDevice`, `AppleMlaunch`, `AppleState`
47+
48+
**Apple Simulator Commands:**
49+
- `List`, `Find`, `Install`, `ResetSimulator`
50+
51+
**WASM Commands:**
52+
- `WasmTest`, `WasmTestBrowser`, `WebServer`
53+
54+
**WASI Commands:**
55+
- `WasiTest`
56+
57+
## Development Guidelines
58+
59+
### System Requirements
60+
- .NET 6+ for development and runtime
61+
- macOS with full Xcode installation for Apple scenarios
62+
- Linux/macOS/Windows for Android scenarios
63+
- Linux for browser scenarios
64+
65+
### Build System
66+
- Use `./build.sh` (Linux/macOS) or `Build.cmd` (Windows) for proper SDK setup
67+
- Alternative: `dotnet build XHarness.sln` (requires correct .NET version)
68+
- Integration with Arcade SDK for .NET Foundation build standards
69+
- Azure DevOps pipelines for CI/CD
70+
71+
### Key Dependencies
72+
- **ADB (Android Debug Bridge)** - Required for Android operations
73+
- **mlaunch** - Required for Apple platform operations
74+
- **Helix SDK** - For cloud testing integration
75+
- Downloaded automatically during CLI build process
76+
77+
### Exit Codes
78+
XHarness uses standardized exit codes (see `src/Microsoft.DotNet.XHarness.Common/CLI/ExitCode.cs`):
79+
- `0` - SUCCESS
80+
- `1` - TESTS_FAILED
81+
- `70` - TIMED_OUT
82+
- `78` - PACKAGE_INSTALLATION_FAILURE
83+
- `80` - APP_CRASH
84+
- `81` - DEVICE_NOT_FOUND
85+
- And many more specific failure scenarios
86+
87+
## Platform-Specific Knowledge
88+
89+
### Android Development
90+
- Uses ADB for device communication
91+
- Supports APK installation and logcat collection
92+
- Package name-based application identification
93+
- Emulator and physical device support
94+
95+
### Apple Development
96+
- Uses mlaunch for device/simulator communication
97+
- Supports .app bundle and .ipa installations
98+
- Requires proper code signing and provisioning profiles
99+
- Complex simulator runtime management
100+
- TCP connection workarounds for test result streaming
101+
102+
### WASM/WASI Development
103+
- Browser-based test execution
104+
- WebAssembly runtime requirements
105+
- Custom web server for test hosting
106+
107+
## Testing Strategy
108+
109+
### Unit Tests
110+
- Located in `tests/` directory with platform-specific test projects
111+
- Follow naming convention: `Microsoft.DotNet.XHarness.[Component].Tests`
112+
- Use xUnit framework consistently
113+
114+
### Integration Tests
115+
- Located in `tests/integration-tests/`
116+
- E2E tests that use Helix cloud infrastructure
117+
- Test real device/simulator scenarios
118+
- Use `./tools/run-e2e-test.sh` for execution
119+
120+
### Test Runners
121+
- Applications must include TestRunner library for `apple test` command
122+
- TestRunner handles environmental variables and TCP connections
123+
- Alternative: Use `apple run` for apps without TestRunner
124+
125+
## Common Patterns and Conventions
126+
127+
### Command Implementation
128+
- Extend `XHarnessCommand<T>` abstract base class
129+
- Implement required properties: `CommandUsage`, `CommandDescription`, `Arguments`
130+
- Use dependency injection for logging and services
131+
- Return appropriate `ExitCode` enum values
132+
133+
### Logging
134+
- Multiple logger types: `ConsoleLogger`, `FileLogger`, `MemoryLogger`, `AggregatedLogs`, `CallbackLogger`
135+
- Console logger is default for commands
136+
- File logger used for mlaunch and adb commands
137+
- Memory logger used by platform-specific command runners
138+
139+
### Error Handling
140+
- Use specific exception types in `Microsoft.DotNet.XHarness.Common.CLI`
141+
- `NoDeviceFoundException` for device discovery failures
142+
- Proper exit code mapping for different failure scenarios
143+
144+
### Environmental Variables
145+
- `XHARNESS_DISABLE_COLORED_OUTPUT` - Disable colored logging
146+
- `XHARNESS_LOG_WITH_TIMESTAMPS` - Enable timestamps
147+
- `XHARNESS_LOG_TEST_START` - Log test start messages
148+
- `XHARNESS_MLAUNCH_PATH` - Custom mlaunch path for development
149+
150+
## File and Directory Structure
151+
152+
```
153+
/
154+
├── src/ # Source code
155+
├── tests/ # Unit and integration tests
156+
├── docs/ # Documentation
157+
├── eng/ # Build and engineering files
158+
├── tools/ # Development tools and scripts
159+
├── azure-pipelines*.yml # CI/CD pipeline definitions
160+
├── XHarness.sln # Main solution file
161+
├── build.sh / Build.cmd # Build scripts
162+
└── README.md # Main documentation
163+
```
164+
165+
## Troubleshooting Guidelines
166+
167+
### Common Issues
168+
1. **Apple unit tests not running**: Ensure TestRunner is included in app bundle
169+
2. **iOS/tvOS device timeouts**: Use `--signal-app-end` flag and ensure app logs the `RUN_END_TAG`
170+
3. **Build failures**: Check .NET SDK version and use provided build scripts
171+
4. **Device not found**: Verify device connection and platform-specific tooling (ADB/mlaunch)
172+
173+
### Debugging Tips
174+
- Use appropriate verbosity levels for logging
175+
- Check device/simulator state before test execution
176+
- Verify app signing and provisioning for Apple platforms
177+
- Monitor TCP connections for test result streaming
178+
179+
## Development Workflow
180+
181+
### For Bug Fixes
182+
1. Identify affected platform and component
183+
2. Create unit tests to reproduce the issue
184+
3. Implement minimal fix in appropriate layer
185+
4. Ensure no regression in existing functionality
186+
5. Update integration tests if needed
187+
188+
### For New Features
189+
1. Understand platform-specific requirements
190+
2. Design feature following existing command patterns
191+
3. Implement with proper error handling and exit codes
192+
4. Add comprehensive tests (unit and integration)
193+
5. Update documentation and help text
194+
195+
### Code Quality
196+
- Follow existing naming conventions and code patterns
197+
- Use dependency injection for testability
198+
- Implement proper logging throughout
199+
- Handle platform-specific edge cases
200+
- Maintain backwards compatibility when possible
201+
202+
## Self-Improvement Instructions
203+
204+
**IMPORTANT**: If you discover any issues, gaps, or outdated information in these instructions while working on XHarness issues, you must update this document with your new knowledge and learnings. This includes:
205+
206+
1. **New platform-specific quirks or workarounds discovered**
207+
2. **Additional environmental variables or configuration options**
208+
3. **Updated build procedures or dependency requirements**
209+
4. **New testing patterns or debugging techniques**
210+
5. **Command structure changes or new platform support**
211+
6. **Performance optimization patterns**
212+
7. **Security considerations or best practices**
213+
214+
When updating these instructions:
215+
- Add specific examples and code snippets where helpful
216+
- Include version information for any platform-specific requirements
217+
- Document the context and scenario where the knowledge applies
218+
- Maintain the existing structure and organization
219+
- Test your changes to ensure accuracy
220+
221+
Your goal is to continuously improve these instructions to become the most effective autonomous agent for XHarness development, capable of solving issues, fixing bugs, and implementing new features efficiently.
222+
223+
---
224+
225+
*These instructions are designed to help you understand and work effectively with the XHarness codebase. Keep them updated as you learn more about the project.*

‎.github/workflows/copilot-setup-steps.yml‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# Automatically run the setup steps when they are changed to allow for easy validation, and
2+
# allow manual testing through the repository's "Actions" tab
3+
on:
4+
workflow_dispatch:
5+
push:
6+
paths:
7+
- .github/workflows/copilot-setup-steps.yml
8+
pull_request:
9+
paths:
10+
- .github/workflows/copilot-setup-steps.yml
11+
12+
permissions:
13+
contents: read
14+
15+
env:
16+
# Allow Copilot to access Visual Studio assets URLs needed for NuGet restore
17+
COPILOT_AGENT_FIREWALL_ALLOW_LIST_ADDITIONS: "vsblob.vsassets.io"
18+
19+
jobs:
20+
21+
copilot-setup-steps:
22+
runs-on: ubuntu-latest
23+
steps:
24+
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
25+
- name: Do an initial build to ensure all dependencies are restored
26+
run: |
27+
./build.sh
28+
- name: Put repo-local dotnet install on PATH
29+
run: |
30+
echo "PATH=$PWD/.dotnet:$PATH" >> $GITHUB_ENV
31+
32+
- name: Check dotnet version
33+
run: |
34+
dotnet --version

0 commit comments

Comments
 (0)