[xtask] refactor into unified logic. (#39)

* Refactor Axvisor project structure and remove unused files

- Deleted `menuconfig.rs`, `tbuld.rs`, and `vmconfig.rs` as they were no longer needed.
- Updated `cargo.rs` to support RISC-V architecture in QEMU commands.
- Enhanced `ctx.rs` to manage repository root paths more effectively.
- Modified `devspace.rs` to ensure submodules and patches are handled relative to the repository root.
- Adjusted image management in `image/mod.rs` to accept repository root paths.
- Refactored `mod.rs` to centralize path resolution for various configurations.
- Improved test cases to ensure they operate correctly with the new structure.

* 更新 `windows-sys` 依赖版本至 0.61.2，添加 `tracing` 相关依赖并在多个模块中实现日志记录功能

* Refactor QEMU integration and enhance image download functionality

- Removed redundant methods in QemuRunner and streamlined the run process.
- Updated Context struct to use Tool instead of AppContext for better abstraction.
- Improved error handling and logging in image download functions.
- Added progress indicators for downloading and decompressing root filesystem images.
- Refactored build arguments handling in axvisor and starry modules to simplify configuration.
- Introduced a new qemu_override module to manage QEMU runtime configurations.
- Enhanced the build process to ensure required artifacts are downloaded with progress feedback.
- Removed obsolete ArceosConfigOverride and related code to clean up the configuration management.

* Refactor xtask: Remove unused modules and streamline main functionality

- Deleted unused modules: `qemu_override.rs`, `build.rs`, `config.rs`, `mod.rs`, and `run.rs` from the `starry` and `xtask` directories.
- Simplified the `main.rs` file to directly call `axbuild::run()` for improved clarity and reduced complexity.
- Removed redundant code related to target handling and test execution for `StarryOS` and `ArceOS`.
- Consolidated error handling and logging for better maintainability.

* 更新 BuildConfig 结构体，修改日志级别字段为非可选类型，并在默认实现中初始化环境变量

* 更新依赖项，重构构建配置，增强日志记录功能并添加进程扩展

* 修复错误处理：将 `anyhow::anyhow!` 替换为 `anyhow!` 以简化错误消息生成

* 增强 ArceOS 构建功能：添加 U-Boot 支持，重构相关逻辑并清理依赖项

* 重构内存分配器：重命名 hv 特性为 buddy-slab

* 增强 BuildConfig 结构体：添加新构造函数以支持动态配置，修改 ArgsBuild 结构体以使用 no_dyn 选项

* 增强 ArceOS 构建功能：添加 BuildConfigLookupKey 结构体，重构相关逻辑以支持新的构建配置路径解析；更新 .gitignore 文件以排除 .arceos_* 文件

* Refactor Arceos build configuration and command handling

- Updated the `BuildConfig` structure to `ArceosBuildInfo` with new fields and methods.
- Modified the `build.rs` file to accommodate the new structure and improve feature resolution.
- Changed the command-line argument handling in `mod.rs` to use `BuildCliArgs`.
- Enhanced the `AppContext` to prepare requests with improved snapshot handling.
- Added tests for new functionality, including loading and storing build snapshots.
- Updated `.gitignore` to refine ignored patterns for build artifacts.

* Refactor ArceOS and StarryOS CLI argument handling and build process

- Renamed `no_dyn` argument to `plat_dyn` in ArceOS and updated related structures and tests.
- Introduced new `StarryCliArgs` structure for StarryOS command handling.
- Added new StarryOS build module with functions to resolve build info paths and load cargo configurations.
- Implemented command handling for StarryOS in the main application, including build, QEMU, and U-Boot commands.
- Enhanced tests for StarryOS to ensure correct argument parsing and build info handling.
- Updated serialization and deserialization logic for new structures in both ArceOS and StarryOS.

* 增强 ArceOS 和 StarryOS 的 QEMU 运行支持：更新参数以接受额外的命令行参数和正则表达式，重构相关逻辑以适应新的构建需求

* 增强 StarryOS 的 rootfs 支持：添加 rootfs 模块，下载并解压 rootfs 镜像，更新相关命令和参数以支持新的构建流程

* 增强 ArceOS 支持：为 x86_64 目标添加默认 to_bin 配置，并更新相关逻辑以支持新的构建需求

* 增强 ArceOS 支持：修改默认日志级别为 Warn，并添加测试命令以支持标准测试

* 增强 QEMU 测试支持：添加 ArceOS 和 StarryOS 的 QEMU 测试逻辑，更新测试配置文件以支持多种架构

* 为所有 StarryOS QEMU 配置文件添加超时设置

* 增强 Axvisor 支持：添加图像管理功能，更新配置和存储逻辑以支持新功能

* 更新 QEMU 配置文件：将磁盘映像路径修改为相对路径以支持工作区结构

* 增强测试支持：添加条件编译标记以优化测试模块，清理导入并修复代码风格

* 更新 ostool 版本至 0.10.0，重构 ArceosBuildInfo 和 StarryBuildInfo 中的配置方法以提高代码一致性

* 增强 ArceosBuildInfo 结构，添加 max_cpu_num 字段并实现相关验证逻辑；优化代码风格，清理导入；更新多个文件以支持新功能

* 增强 ArceOS 测试支持：为 task/affinity、irq、parallel、priority、sleep、wait_queue、yield 添加多架构构建配置

* 更新 AGENTS.md，添加关于使用 cargo xtask 的建议；更新 qemu-loongarch64.toml，修复 nographic 参数位置

* 统一测试成功输出消息为 'All tests passed!' 并更新相关配置文件

* 更新 arm-gic-driver 版本至 0.16.5，并修正相关校验和

* Refactor image handling in Axvisor

- Updated the setup_qemu.sh script to replace `cargo xtask image download` with `cargo axvisor image pull` for consistency in image downloading.
- Introduced a fallback registry URL in the image configuration to enhance registry resolution.
- Added a new `pull` command to the Axvisor CLI for downloading images, including options for output directory and extraction control.
- Implemented image specification parsing to handle image names and versions more effectively.
- Enhanced the image registry to support fetching from a fallback source if the default registry is unavailable.
- Refactored the storage module to include image pulling functionality, ensuring proper checksum verification and extraction of downloaded images.
- Created a dedicated download module to handle HTTP requests and progress tracking for downloads.
- Updated tests to cover new functionality, including pulling images and handling registry fallbacks.

* 增强 Axvisor 支持：添加构建和运行命令，更新文档，修复配置路径

* 更新 Axvisor 配置文件，使用新的构建配置文件名，修正相关文档和代码中的路径

* 增强 Axvisor 支持：添加默认板级配置生成逻辑，更新文档，修复相关测试

* 增强 Axvisor 支持：添加 QEMU 测试逻辑，更新文档，修复相关路径

* 增强 Axvisor 支持：更新 QEMU 配置逻辑，添加默认参数处理，修复相关测试

* 增强 Axvisor 支持：添加平台特性归一化逻辑，更新相关测试

* 增强 Axvisor 支持：更新依赖项版本，修复配置路径，添加 Nimbos x86_64 支持

* 增强 Axvisor 支持：移除无效的 x86_64 目标配置，添加 axvisor x86_64 测试逻辑

* 增强 Axvisor 支持：替换 CargoQemuAppendArgs 为 CargoQemuOverrideArgs，更新相关测试逻辑

Author: ZR233

Cargo.toml

@@ -28,9 +28,9 @@ include = [
 [profile.release]
 lto = true
 
-[[bin]]
-name = "xtask"
-path = "xtask/src/main.rs"
+# [[bin]]
+# name = "xtask"
+# path = "xtask/src/main.rs"
 
 [features]
 default = []
@@ -120,25 +120,10 @@ axplat-riscv64-qemu-virt = "0.3.1-pre.6"
 
 # xtask dependencies (only used on host platforms)
 [target.'cfg(any(windows, unix))'.dependencies]
-anyhow = { version = "1.0" }
-cargo_metadata = { version = "0.23" }
-chrono = { version = "0.4", features = ["serde"] }
-clap = {version = "4.4", features = ["derive"] }
-colored = { version = "3" }
-flate2 = { version = "1.0" }
-jkconfig = { version = "0.1" }
-ostool = { version = "0.8.4" }
-regex = { version = "1.12" }
-reqwest = { version = "0.13" }
-schemars = {version = "1", features = ["derive"] }
-serde = {version = "1.0", features = ["derive"] }
-serde_json = { version = "1" }
-sha2 = { version = "0.10" }
-tar = { version = "0.4" }
-tokio = {version = "1", features = ["full"] }
-toml = { version = "0.9" }
-
-axvmconfig = { version = "0.2.2", features = ["std"] }
+anyhow.workspace = true
+axbuild.workspace = true
+clap = { version = "4.6", features = ["derive"] }
+tokio = { version = "1", features = ["full"] }
 
 [build-dependencies]
 axconfig = "=0.3.0-preview.3"

configs/board/orangepi-5-plus.toml

@@ -1,9 +1,5 @@
-cargo_args = [
-    "--config",
-    'target.aarch64-unknown-none-softfloat.rustflags = ["-Crelocation-model=pic", "-Clink-args=-pie", "-Clink-args=-znostart-stop-gc", "-Clink-args=-Taxplat.x"]',
-]
+cargo_args = []
 features = [
-  # "ept-level-4",
   "dyn-plat",
   "axstd/bus-mmio",
   "driver/sdmmc",

configs/board/phytiumpi.toml

@@ -1,9 +1,5 @@
-cargo_args = [
-    "--config",
-    'target.aarch64-unknown-none-softfloat.rustflags = ["-Crelocation-model=pic", "-Clink-args=-pie", "-Clink-args=-znostart-stop-gc", "-Clink-args=-Taxplat.x"]',
-]
+cargo_args = []
 features = [
-    # "ept-level-4",
     "dyn-plat",
     "axstd/bus-mmio",
     "fs",

configs/board/qemu-aarch64.toml

@@ -1,7 +1,4 @@
-cargo_args = [
-#     "--config",
-#     'target.aarch64-unknown-none-softfloat.rustflags = ["-Crelocation-model=pic", "-Clink-args=-pie", "-Clink-args=-znostart-stop-gc", "-Clink-args=-Taxplat.x"]',
-]
+cargo_args = []
 features = [
     "ept-level-4",
     "axstd/bus-mmio",

configs/board/qemu-riscv64.toml

@@ -9,4 +9,4 @@ log = "Info"
 target = "riscv64gc-unknown-none-elf"
 to_bin = true
 vm_configs = []
-smp = 4
+max_cpu_num = 4

configs/board/roc-rk3568-pc.toml

@@ -1,9 +1,5 @@
-cargo_args = [
-    "--config",
-    'target.aarch64-unknown-none-softfloat.rustflags = ["-Crelocation-model=pic", "-Clink-args=-pie", "-Clink-args=-znostart-stop-gc", "-Clink-args=-Taxplat.x"]',
-]
+cargo_args = []
 features = [
-    # "ept-level-4",
     "dyn-plat",
     "axstd/bus-mmio",
     "fs",

configs/vms/linux-riscv64-qemu-smp1.toml

@@ -23,11 +23,11 @@ entry_point = 0x9000_0000
 image_location = "memory"
 # The file path of the kernel image.
 # kernel_path = "linux-6.6.62.bin"
-kernel_path = "/path/tmp/images/qemu_riscv64_linux/qemu-riscv64"
+kernel_path = "/tmp/.axvisor-images/qemu_riscv64_linux/qemu-riscv64"
 # The load address of the kernel image.
 kernel_load_addr = 0x9000_0000
 # The file path of the device tree blob (DTB).
-dtb_path = "/path/tmp/configs/linux-riscv64-qemu-smp1.dtb"
+# dtb_path = "/path/tmp/configs/linux-riscv64-qemu-smp1.dtb"
 # The load address of the device tree blob (DTB).
 dtb_load_addr = 0x9300_0000
 

doc/qemu-quickstart.md

@@ -114,7 +114,7 @@ After booting, you will enter the Rust user shell (`>>` prompt). Type `usertests
 
 The script automates three steps, eliminating manual work:
 
-1. **Download images**: calls `cargo xtask image download` to fetch guest images to `/tmp/.axvisor-images/`
+1. **Download images**: calls `cargo axvisor image pull` to fetch and extract guest images to `/tmp/.axvisor-images/`
 2. **Generate temp configs**: copies VM config templates to `tmp/vmconfigs/*.generated.toml`, then uses `sed` to update `kernel_path` (and `bios_path` for NimbOS) to actual image paths without modifying tracked files in `configs/vms/*.toml`
 3. **Prepare rootfs**: copies `rootfs.img` to the project's `tmp/` directory for QEMU to use
 
@@ -136,7 +136,7 @@ QEMU is not installed. Run the `apt install` command from Step 1.
 
 ### `Auto syncing from registry ... timed out`
 
-This usually indicates unstable access to GitHub Raw endpoints. `scripts/setup_qemu.sh` includes one built-in recovery attempt: when the first image download fails, it bootstraps a local registry and retries once automatically. The script also has a default fallback registry (currently pointing to `v0.0.22.toml`).
+This usually indicates unstable access to GitHub Raw endpoints. `cargo axvisor image pull` now handles registry bootstrap internally: it prefers the default registry, follows the included registry when present, and falls back to the built-in fallback registry (`v0.0.22.toml`) when the default endpoint is unavailable.
 
 If your network is unstable for specific registry URLs, you can override the fallback registry:
 
@@ -148,4 +148,3 @@ export AXVISOR_REGISTRY_FALLBACK_URL="https://raw.githubusercontent.com/arceos-h
 ### First build is very slow
 
 This is expected. AxVisor has many dependencies, and the first compilation needs to download and build all crates. Subsequent incremental builds will be much faster.
-

doc/qemu-quickstart_cn.md

@@ -114,7 +114,7 @@ cargo xtask qemu \
 
 该脚本自动完成以下三步，省去手动操作：
 
-1. **下载镜像**：调用 `cargo xtask image download` 将 Guest 镜像下载到 `/tmp/.axvisor-images/`
+1. **下载镜像**：调用 `cargo axvisor image pull` 将 Guest 镜像下载并解压到 `/tmp/.axvisor-images/`
 2. **生成临时配置**：复制模板 VM 配置到 `tmp/vmconfigs/*.generated.toml`，并用 `sed` 更新 `kernel_path`（以及 NimbOS 的 `bios_path`）到实际镜像路径，不修改仓库内 `configs/vms/*.toml`
 3. **准备 rootfs**：将 `rootfs.img` 复制到项目的 `tmp/` 目录下供 QEMU 使用
 
@@ -136,7 +136,7 @@ VM 配置中的 `kernel_path` 指向了不存在的文件。运行 `./scripts/se
 
 ### `Auto syncing from registry ... timed out`
 
-这通常是访问 GitHub Raw 不稳定导致的。`scripts/setup_qemu.sh` 已内置一次自动恢复逻辑：首次下载失败后，会尝试自动引导本地 registry 并重试镜像下载。脚本内也提供了默认 fallback registry（当前指向 `v0.0.22.toml`）。
+这通常是访问 GitHub Raw 不稳定导致的。`cargo axvisor image pull` 现在会在命令内部处理 registry 引导逻辑：优先使用默认 registry，若其中声明了 include 就继续跟随 include；若默认入口不可用，则自动回退到内建 fallback registry（当前指向 `v0.0.22.toml`）。
 
 如果你所在网络环境对部分 URL 不稳定，可显式覆盖 fallback registry：
 
@@ -148,4 +148,3 @@ export AXVISOR_REGISTRY_FALLBACK_URL="https://raw.githubusercontent.com/arceos-h
 ### 首次构建非常慢
 
 正常现象。AxVisor 依赖较多，首次编译需要下载并编译所有 crate。后续增量编译会快很多。
-

scripts/quick-start.sh

@@ -140,10 +140,10 @@ setup_qemu_aarch64() {
     run_cmd mkdir -p tmp/{configs,images}
 
     info "Downloading ArceOS image..."
-    run_cmd cargo xtask image download qemu_aarch64_arceos --output-dir tmp/images
+    run_cmd cargo axvisor image pull qemu_aarch64_arceos --output-dir tmp/images
 
     info "Downloading Linux image..."
-    run_cmd cargo xtask image download qemu_aarch64_linux --output-dir tmp/images
+    run_cmd cargo axvisor image pull qemu_aarch64_linux --output-dir tmp/images
 
     info "Preparing board config file..."
     run_cmd cp configs/board/qemu-aarch64.toml tmp/configs/
@@ -203,7 +203,7 @@ setup_qemu_x86_64() {
     run_cmd mkdir -p tmp/{configs,images}
 
     info "Downloading NimbOS image..."
-    run_cmd cargo xtask image download qemu_x86_64_nimbos --output-dir tmp/images
+    run_cmd cargo axvisor image pull qemu_x86_64_nimbos --output-dir tmp/images
 
     info "Preparing board config file..."
     run_cmd cp configs/board/qemu-x86_64.toml tmp/configs/
@@ -267,10 +267,10 @@ setup_phytiumpi() {
     fi
 
     info "Downloading ArceOS image..."
-    run_cmd cargo xtask image download phytiumpi_arceos --output-dir tmp/images
+    run_cmd cargo axvisor image pull phytiumpi_arceos --output-dir tmp/images
 
     info "Downloading Linux image (including device tree)..."
-    run_cmd cargo xtask image download phytiumpi_linux --output-dir tmp/images
+    run_cmd cargo axvisor image pull phytiumpi_linux --output-dir tmp/images
 
     info "Preparing board config file..."
     run_cmd cp configs/board/phytiumpi.toml tmp/configs/
@@ -386,10 +386,10 @@ setup_roc_rk3568_pc() {
     fi
 
     info "Downloading ArceOS image..."
-    run_cmd cargo xtask image download roc-rk3568-pc_arceos --output-dir tmp/images
+    run_cmd cargo axvisor image pull roc-rk3568-pc_arceos --output-dir tmp/images
 
     info "Downloading Linux image (including device tree)..."
-    run_cmd cargo xtask image download roc-rk3568-pc_linux --output-dir tmp/images
+    run_cmd cargo axvisor image pull roc-rk3568-pc_linux --output-dir tmp/images
 
     info "Preparing board config file..."
     run_cmd cp configs/board/roc-rk3568-pc.toml tmp/configs/