Add offline package compression and update Docker/Kubernetes instructions (#3306)

* Add offline package compression and pull skipping

* ✨ Update installation and deployment instructions for Docker and Kubernetes

---------

Co-authored-by: hhhhsc <name>

Author: hhhhsc701

.github/workflows/build-offline-package.yml

@@ -100,17 +100,16 @@ jobs:
             --include-source "${{ inputs.include_source }}" \
             --image-source "${{ inputs.image_source }}" \
             --components "${{ inputs.components }}" \
-            --target "${{ inputs.target }}"
+            --target "${{ inputs.target }}" \
+            --compress true
 
 
 
-      - name: Create zip package
+      - name: Show zip package
         run: |
           PACKAGE_NAME="${{ steps.set-vars.outputs.package-name }}"
 
-          (cd offline-output && zip -r "../${PACKAGE_NAME}.zip" .)
-
-          echo "Package created: ${PACKAGE_NAME}.zip"
+          echo "Package created by build script: ${PACKAGE_NAME}.zip"
 
           ls -lh "${PACKAGE_NAME}.zip"
 

README.md

@@ -50,12 +50,14 @@ cd nexent
 bash deploy.sh docker
 ```
 
-The root `deploy.sh` only forwards to the target deploy script; the native Docker implementation is `bash deploy/docker/deploy.sh`. The Docker and Kubernetes deploy scripts share the same deployment configuration model. Interactive runs show Bash TUI menus for component selection, port policy, and image source. `infrastructure` is required; `application` is selected by default but can be disabled. Use `b`/Backspace to return to the previous TUI step and `q` to quit. Non-interactive runs can pass the same choices with `--version`, `--components`, `--port-policy development|production`, and `--image-source general|mainland|local-latest`. Successful deployments save non-sensitive choices to each deploy directory's `deploy.options` for reuse on the next run.
+The root `deploy.sh` only forwards to the target deploy script; the native Docker implementation is `bash deploy/docker/deploy.sh`. The Docker and Kubernetes deploy scripts share the same deployment configuration model. Interactive runs show Bash TUI menus for component selection, port policy, and image source. `infrastructure` is required; `application`, `data-process`, and `supabase` are selected by default and can be disabled when you want a smaller deployment. Use `b`/Backspace to return to the previous TUI step and `q` to quit. Non-interactive runs can pass the same choices with `--version`, `--components`, `--port-policy development|production`, and `--image-source general|mainland|local-latest`. Successful deployments save non-sensitive choices to each deploy directory's `deploy.options` for reuse on the next run.
 
-Docker and Kubernetes both use the project root `.env` as the runtime configuration file. If it does not exist, the deploy scripts create it from `.env.example` or migrate an existing `docker/.env` once.
+Docker and Kubernetes both use the project root `.env` as the runtime configuration file. Existing `.env` is kept as-is. If it does not exist, the deploy scripts first reuse an existing `docker/.env`, then fall back to `.env.example` or `docker/.env.example`.
 
 Docker uninstall is handled by `bash uninstall.sh docker`. It can preserve or delete data volumes: run it interactively, pass `--delete-volumes true|false`, or use `bash uninstall.sh docker delete-all` to remove containers and persistent data.
 
+Offline image packages can be built with `bash deploy/offline/build_offline_package.sh --target docker --compress true`. The package includes image tar files, `load-images.sh`, root deploy/uninstall entrypoints, deployment scripts, SQL files, `manifest.yaml`, and `checksums.txt`; deploy it with `bash deploy.sh --load-images docker ...` on the target host.
+
 For detailed deployment instructions, see [Docker Installation](https://modelengine-group.github.io/nexent/en/quick-start/installation.html).
 
 ### Kubernetes Deployment (For Enterprise Production)
@@ -68,10 +70,12 @@ cd nexent
 bash deploy.sh k8s
 ```
 
-The native Kubernetes implementation is `bash deploy/k8s/deploy.sh`. It reads the same project root `.env` as Docker and renders explicit values into Helm ConfigMap and Secret overrides. Use `--persistence-mode local|dynamic|existing`, `--storage-class`, `--local-path`, `--local-node-name`, and `--existing-claim-prefix` to control PVC behavior.
+The native Kubernetes implementation is `bash deploy/k8s/deploy.sh`. It reads the same project root `.env` as Docker and renders explicit values into Helm ConfigMap and Secret overrides. Use `--persistence-mode local|dynamic|existing`, `--storage-class`/`--sc`, `--local-path`, `--local-node-name`, and `--existing-claim-prefix` to control PVC behavior. Local mode renders `hostPath` PVs and does not require node affinity.
 
 Kubernetes uninstall is handled by `bash uninstall.sh k8s`. It removes the Helm release first, then can optionally delete the namespace and local PV data. Use `--delete-namespace true|false`, `--delete-local-data true|false`, or `bash uninstall.sh k8s delete-all`; pass `--keep-local-data` with `delete-all` to preserve local volume contents.
 
+Kubernetes offline packages use the same builder with `--target k8s` or `--target all`. Run `load-images.sh` on every cluster node that needs the images, or push the loaded images to an internal registry before deploying with the same version and image-source options used during packaging.
+
 For detailed deployment instructions, see [Kubernetes Installation](https://modelengine-group.github.io/nexent/en/quick-start/kubernetes-installation.html).
 
 # ✨ Core Features

README_CN.md

@@ -50,9 +50,13 @@ cd nexent
 bash deploy.sh docker
 ```
 
-根目录 `deploy.sh` 只负责转发到目标部署脚本；Docker 真实实现为 `bash deploy/docker/deploy.sh`。非交互部署可传入 `--version`、`--components`、`--port-policy development|production`、`--image-source general|mainland|local-latest`。
+根目录 `deploy.sh` 只负责转发到目标部署脚本；Docker 真实实现为 `bash deploy/docker/deploy.sh`。Docker 和 Kubernetes 使用同一套部署配置模型；交互式运行会通过 Bash TUI 选择组件、端口策略和镜像源。`infrastructure` 必选，`application`、`data-process`、`supabase` 默认选中，也可以取消以部署更小的组合。非交互部署可传入 `--version`、`--components`、`--port-policy development|production`、`--image-source general|mainland|local-latest`。
 
-Docker 与 Kubernetes 统一使用项目根目录 `.env` 作为运行配置文件；如果不存在，部署脚本会从 `.env.example` 创建，或首次自动迁移已有的 `docker/.env`。
+Docker 与 Kubernetes 统一使用项目根目录 `.env` 作为运行配置文件；已有 `.env` 会原样保留。如果根目录 `.env` 不存在，部署脚本会优先复用已有的 `docker/.env`，再回退到 `.env.example` 或 `docker/.env.example`。
+
+Docker 卸载入口为 `bash uninstall.sh docker`，默认交互确认是否删除持久化数据；也可以通过 `--delete-volumes true|false` 控制，或使用 `bash uninstall.sh docker delete-all` 同时删除容器和持久化数据。
+
+离线镜像包可通过 `bash deploy/offline/build_offline_package.sh --target docker --compress true` 构建。包内包含镜像 tar、`load-images.sh`、根目录部署/卸载入口、部署脚本、SQL 文件、`manifest.yaml` 和 `checksums.txt`；在目标机器上使用 `bash deploy.sh --load-images docker ...` 加载镜像并部署。
 
 详细部署指南请参考 [Docker 安装部署](https://modelengine-group.github.io/nexent/zh/quick-start/installation.html)。
 
@@ -66,10 +70,12 @@ cd nexent
 bash deploy.sh k8s
 ```
 
-Kubernetes 真实实现为 `bash deploy/k8s/deploy.sh`。它会读取同一个根目录 `.env`，并显式渲染为 Helm ConfigMap 和 Secret 覆盖值。PVC 可通过 `--persistence-mode local|dynamic|existing`、`--storage-class`、`--local-path`、`--local-node-name`、`--existing-claim-prefix` 控制。
+Kubernetes 真实实现为 `bash deploy/k8s/deploy.sh`。它会读取同一个根目录 `.env`，并显式渲染为 Helm ConfigMap 和 Secret 覆盖值。PVC 可通过 `--persistence-mode local|dynamic|existing`、`--storage-class`/`--sc`、`--local-path`、`--local-node-name`、`--existing-claim-prefix` 控制。local 模式会渲染 `hostPath` PV，不再需要 nodeAffinity。
 
 根目录卸载入口为 `bash uninstall.sh docker ...` 或 `bash uninstall.sh k8s ...`，具体实现仍分别在 `deploy/docker/uninstall.sh` 和 `deploy/k8s/uninstall.sh`。
 
+Kubernetes 离线包使用同一个构建脚本，传入 `--target k8s` 或 `--target all`。部署前需要在每个需要运行 Pod 的节点上执行 `load-images.sh`，或将镜像推送到集群可访问的内部镜像仓库，再使用与打包时一致的版本和镜像源参数部署。
+
 详细部署指南请参考 [Kubernetes 安装部署](https://modelengine-group.github.io/nexent/zh/quick-start/kubernetes-installation.html)。
 
 # ✨ 核心特性

deploy.sh

@@ -7,11 +7,15 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 usage() {
   cat <<'USAGE'
 Usage:
-  bash deploy.sh docker [docker deploy options]
-  bash deploy.sh k8s [k8s deploy options]
+  bash deploy.sh [--load-images] docker [docker deploy options]
+  bash deploy.sh [--load-images] k8s [k8s deploy options]
 
 This root entrypoint only forwards to the target-specific deploy script.
 Implementation: deploy/deploy.sh
+
+Options:
+  --load-images    Load Docker image tar files from ./images before deploying.
+                   Defaults to off.
 USAGE
 }
 
@@ -20,4 +24,34 @@ if [ "${1:-}" = "--help" ] || [ "${1:-}" = "-h" ] || [ $# -eq 0 ]; then
   exit 0
 fi
 
-exec bash "$SCRIPT_DIR/deploy/deploy.sh" "$@"
+LOAD_IMAGES="false"
+FORWARD_ARGS=()
+
+while [ $# -gt 0 ]; do
+  case "$1" in
+    --load-images)
+      LOAD_IMAGES="true"
+      shift
+      ;;
+    *)
+      FORWARD_ARGS+=("$1")
+      shift
+      ;;
+  esac
+done
+
+if [ "${#FORWARD_ARGS[@]}" -eq 0 ]; then
+  usage
+  exit 0
+fi
+
+if [ "$LOAD_IMAGES" = "true" ]; then
+  LOAD_SCRIPT="$SCRIPT_DIR/load-images.sh"
+  if [ ! -f "$LOAD_SCRIPT" ]; then
+    echo "Error: --load-images requires $LOAD_SCRIPT" >&2
+    exit 1
+  fi
+  bash "$LOAD_SCRIPT"
+fi
+
+exec bash "$SCRIPT_DIR/deploy/deploy.sh" "${FORWARD_ARGS[@]}"

deploy/docker/generate_env.sh

@@ -8,9 +8,12 @@ DEPLOY_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
 PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
 ENV_FILE="${DEPLOYMENT_ROOT_ENV:-$PROJECT_ROOT/.env}"
 ENV_EXAMPLE="$PROJECT_ROOT/.env.example"
+LEGACY_ENV="$PROJECT_ROOT/docker/.env"
 LEGACY_ENV_EXAMPLE="$PROJECT_ROOT/docker/.env.example"
 
-echo "   📁 Target .env location: $ENV_FILE"
+if [ "${NEXENT_GENERATE_ENV_SKIP_MAIN:-false}" != "true" ]; then
+  echo "   📁 Target .env location: $ENV_FILE"
+fi
 
 update_env_var() {
   local key="$1"
@@ -41,6 +44,10 @@ prepare_env_file() {
 
   if [ -f "$ENV_FILE" ]; then
     echo "   ✅ Using existing root .env"
+  elif [ -f "$LEGACY_ENV" ]; then
+    echo "   root .env not found, copying docker/.env..."
+    cp "$LEGACY_ENV" "$ENV_FILE"
+    echo "   Created root .env from docker/.env"
   elif [ -f "$ENV_EXAMPLE" ]; then
     echo "   📋 root .env not found, copying .env.example..."
     cp "$ENV_EXAMPLE" "$ENV_FILE"
@@ -50,7 +57,7 @@ prepare_env_file() {
     cp "$LEGACY_ENV_EXAMPLE" "$ENV_FILE"
     echo "   ✅ Created root .env from docker/.env.example"
   else
-    echo "   ❌ ERROR Neither root .env nor .env.example exists"
+    echo "   ERROR Neither root .env nor docker/.env nor .env.example exists"
     ERROR_OCCURRED=1
     return 1
   fi
@@ -167,4 +174,6 @@ main() {
 }
 
 # Run main function
-main "$@"
+if [ "${NEXENT_GENERATE_ENV_SKIP_MAIN:-false}" != "true" ]; then
+  main "$@"
+fi

deploy/docker/uninstall.sh

@@ -165,6 +165,30 @@ resolve_delete_volumes() {
   [[ "$answer" =~ ^[Yy]$ ]]
 }
 
+remove_docker_named_volumes() {
+  command -v docker >/dev/null 2>&1 || return 0
+
+  local volume_names
+  volume_names="$(docker volume ls --format '{{.Name}}' 2>/dev/null || true)"
+  [ -n "$volume_names" ] || return 0
+
+  local volumes_to_remove=()
+  local volume
+  while IFS= read -r volume; do
+    [ -n "$volume" ] || continue
+    case "$volume" in
+      nexent_*|nexent-*|monitor_*)
+        volumes_to_remove+=("$volume")
+        ;;
+    esac
+  done <<< "$volume_names"
+
+  if [ "${#volumes_to_remove[@]}" -gt 0 ]; then
+    echo "🧹 Removing Docker volumes: ${volumes_to_remove[*]}"
+    docker volume rm -f "${volumes_to_remove[@]}" >/dev/null 2>&1 || true
+  fi
+}
+
 docker_compose_down_file() {
   local compose_file="$1"
   local use_project_name="$2"
@@ -190,6 +214,7 @@ docker_compose_down_file() {
 
 remove_nexent_data_dirs() {
   local root_dir="${ROOT_DIR:-$HOME/nexent-data}"
+  local work_dir="$HOME/nexent"
   root_dir="${root_dir%/}"
 
   if [ -z "$root_dir" ] || [ "$root_dir" = "/" ]; then
@@ -205,6 +230,8 @@ remove_nexent_data_dirs() {
     "$root_dir/volumes"
     "$root_dir/openssh-server"
     "$root_dir/scripts"
+    "$root_dir/skills"
+    "$work_dir"
   )
 
   local dir
@@ -238,6 +265,7 @@ main() {
   docker_compose_down_file "$COMPOSE_DIR/docker-compose.yml" true "$remove_volumes"
 
   if [ "$remove_volumes" = "true" ]; then
+    remove_docker_named_volumes
     remove_nexent_data_dirs
   fi