Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -425,6 +425,7 @@ jobs:
rpm -qlp "${RPM_FILE}" | grep -E "/usr/share/ablestack/v2k/runtime-assets/assets/VMware-vix-disklib-.*[.]tar[.]gz$" >/dev/null
rpm -qlp "${RPM_FILE}" | grep -E "/usr/share/ablestack/v2k/compat/esxi55/profile.json$" >/dev/null
rpm -qlp "${RPM_FILE}" | grep -E "/usr/share/ablestack/v2k/runtime-assets/assets/compat/esxi55/VMware-vix-disklib-6[.]0[.]2-3566099[.]x86_64[.]tar[.]gz$" >/dev/null
rpm -qlp "${RPM_FILE}" | grep -E "/usr/share/ablestack/v2k/runtime-assets/assets/compat/esxi55/nbdkit-vddk-legacy-1[.]14[.]2-rocky9-x86_64[.]tar[.]gz$" >/dev/null
rpm -qlp "${RPM_FILE}" | grep -E "/usr/share/ablestack/v2k/runtime-assets/assets/compat/esxi55/wheels/pyvmomi-5[.]5[.]0[.]2014[.]1[.]1[.]tar[.]gz$" >/dev/null
if find winpe -maxdepth 1 -type f -iname '*.iso' | grep -q .; then
rpm -qlp "${RPM_FILE}" | grep -E "/usr/share/ablestack/v2k/winpe/.*[.]iso$" >/dev/null
Expand Down
2 changes: 2 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -305,6 +305,8 @@ v2k-rpm:
(echo "[ERR] esxi55 compat profile missing in RPM: $$RPM_FILE" >&2; exit 2); \
rpm -qlp "$$RPM_FILE" | grep -qE "/usr/share/ablestack/v2k/runtime-assets/assets/compat/esxi55/VMware-vix-disklib-6[.]0[.]2-3566099[.]x86_64[.]tar[.]gz$$" || \
(echo "[ERR] esxi55 VDDK runtime asset missing in RPM: $$RPM_FILE" >&2; exit 2); \
rpm -qlp "$$RPM_FILE" | grep -qE "/usr/share/ablestack/v2k/runtime-assets/assets/compat/esxi55/nbdkit-vddk-legacy-1[.]14[.]2-rocky9-x86_64[.]tar[.]gz$$" || \
(echo "[ERR] esxi55 legacy nbdkit runtime asset missing in RPM: $$RPM_FILE" >&2; exit 2); \
rpm -qlp "$$RPM_FILE" | grep -qE "/usr/share/ablestack/v2k/runtime-assets/assets/compat/esxi55/wheels/pyvmomi-5[.]5[.]0[.]2014[.]1[.]1[.]tar[.]gz$$" || \
(echo "[ERR] esxi55 pyVmomi runtime asset missing in RPM: $$RPM_FILE" >&2; exit 2); \
if find winpe -maxdepth 1 -type f -name '*.iso' 2>/dev/null | grep -q .; then \
Expand Down
13 changes: 10 additions & 3 deletions assets/compat/esxi55/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ The role split for this profile is deliberate:
assets/compat/esxi55/
govc_Linux_x86_64.tar.gz
VMware-vix-disklib-6.0.2-3566099.x86_64.tar.gz
nbdkit-vddk-legacy-1.14.2-rocky9-x86_64.tar.gz
wheels/
pyvmomi-5.5.0.2014.1.1.tar.gz
six-*.whl
Expand All @@ -24,9 +25,15 @@ assets/compat/esxi55/
charset_normalizer-*.whl
```

The installer intentionally does not fall back to top-level `govc`, VDDK, or
wheel assets for this profile. Newer VDDK releases can fail against ESXi 5.5 NFC
sessions, so missing profile-local assets are treated as an installation error.
The installer intentionally does not fall back to top-level `govc`, VDDK,
legacy nbdkit, or wheel assets for this profile. Newer VDDK releases can fail
against ESXi 5.5 NFC sessions, while the system nbdkit VDDK plugin can reject
VDDK 6.0.2 at load time, so missing profile-local assets are treated as an
installation error.

The VDDK archive is operator-provided. The currently staged and validated
candidate is `VMware-vix-disklib-6.0.2-3566099.x86_64.tar.gz`.

The legacy nbdkit archive is public redistributable runtime support built from
nbdkit 1.14.2. It is used only by the `esxi55` profile and is invoked with the
profile-local `nbdkit-vddk-plugin.so` absolute path.
1 change: 1 addition & 0 deletions assets/compat/esxi55/SOURCES.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ the repository's binary asset policy.
| Asset | Source | Selected version | Role |
| --- | --- | --- | --- |
| `govc_Linux_x86_64.tar.gz` | `https://github.com/vmware/govmomi/releases/download/v0.46.3/govc_Linux_x86_64.tar.gz` | `govc 0.46.3` | vSphere inventory, snapshot, power, and host queries |
| `nbdkit-vddk-legacy-1.14.2-rocky9-x86_64.tar.gz` | `https://download.libguestfs.org/nbdkit/1.14-stable/nbdkit-1.14.2.tar.gz` | `nbdkit 1.14.2` | Profile-local VDDK plugin runtime that can load VDDK 6.0.2 |
| `wheels/pyvmomi-5.5.0.2014.1.1.tar.gz` | PyPI package `pyvmomi==5.5.0.2014.1.1` | `5.5.0.2014.1.1` | Changed block tracking API calls |
| `wheels/*.whl` | PyPI dependency resolver output for pyVmomi 5.5 | current compatible releases | Offline installation dependencies |

Expand Down
Binary file not shown.
4 changes: 4 additions & 0 deletions bin/ablestack_v2k.sh
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,8 @@ if [[ "${V2K_DEBUG_ENV:-0}" -eq 1 ]]; then
echo "[DEBUG] V2K_GOVC_BIN='${V2K_GOVC_BIN-}'" >&2
echo "[DEBUG] V2K_PYTHON_BIN='${V2K_PYTHON_BIN-}'" >&2
echo "[DEBUG] VDDK_LIBDIR='${VDDK_LIBDIR-}'" >&2
echo "[DEBUG] V2K_NBDKIT_BIN='${V2K_NBDKIT_BIN-}'" >&2
echo "[DEBUG] V2K_NBDKIT_VDDK_PLUGIN='${V2K_NBDKIT_VDDK_PLUGIN-}'" >&2
fi

usage() {
Expand Down Expand Up @@ -118,6 +120,8 @@ Environment:
V2K_COMPAT_SELECTED_PROFILE
V2K_GOVC_BIN
V2K_PYTHON_BIN
V2K_NBDKIT_BIN
V2K_NBDKIT_VDDK_PLUGIN
EOF
}

Expand Down
135 changes: 129 additions & 6 deletions bin/v2k_test_install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -57,10 +57,13 @@ Asset resolution order for each profile:
2. ./assets/compat/<profile>/wheels
3. ./assets/v2k/wheels

1. ./assets/compat/<profile>/nbdkit-vddk-legacy-*.tar.gz

Profiles can opt out of top-level fallback. The ESXi 5.5 profile is strict:
public govc/pyVmomi assets live under ./assets/compat/esxi55, and the operator
must add the licensed VMware VDDK archive there before --install-assets can
install it. The current ESXi 5.5 candidate is VDDK 6.0.2.
install it. The ESXi 5.5 profile also carries a legacy nbdkit VDDK runtime so
VDDK 6.0.2 can be loaded even when the system nbdkit plugin requires VDDK 6.5+.
EOF
}

Expand Down Expand Up @@ -314,7 +317,7 @@ list_profiles() {
echo "[INFO] Sample compat root: ${compat_repo_root}"
echo "[INFO] Install compat root: ${compat_install_root}"

local profile profile_json installed govc_asset vddk_asset wheel_dir
local profile profile_json installed govc_asset vddk_asset wheel_dir nbdkit_asset
while IFS= read -r profile; do
[[ -n "${profile}" ]] || continue
profile_json="${compat_repo_root}/${profile}/profile.json"
Expand All @@ -323,13 +326,15 @@ list_profiles() {
govc_asset="$(resolve_profile_govc_asset "${root}" "${profile}" || true)"
vddk_asset="$(resolve_profile_vddk_asset "${root}" "${profile}" || true)"
wheel_dir="$(resolve_profile_wheel_dir "${root}" "${profile}" || true)"
nbdkit_asset="$(resolve_profile_nbdkit_asset "${root}" "${profile}" || true)"
printf '%s\tinstalled=%s\tsupported=%s\tlabel=%s\n' \
"${profile}" \
"${installed}" \
"$(profile_supported_range "${profile_json}")" \
"$(profile_label "${profile_json}")"
printf ' govc_asset=%s\n' "$(asset_status "${govc_asset}")"
printf ' vddk_asset=%s\n' "$(asset_status "${vddk_asset}")"
printf ' nbdkit_asset=%s\n' "$(asset_status "${nbdkit_asset}")"
printf ' wheel_dir=%s\n' "$(asset_status "${wheel_dir}")"
done < <(compat_profile_ids "${compat_repo_root}")
}
Expand Down Expand Up @@ -445,6 +450,27 @@ resolve_profile_wheel_dir() {
fi
}

resolve_profile_nbdkit_asset() {
local root="$1" profile="$2"
local profile_root tgz
profile_root="$(profile_asset_root "${root}" "${profile}")"
tgz="$(ls -1 "${profile_root}"/nbdkit-vddk-legacy-*.tar.gz 2>/dev/null | sort | tail -n1 || true)"
[[ -n "${tgz}" ]] && printf '%s' "${tgz}"
}

profile_toolchain_value() {
local root="$1" profile="$2" key="$3"
local profile_json
profile_json="$(profile_json_for_asset_root "${root}" "${profile}" || true)"
[[ -n "${profile_json}" ]] || return 1
jq -r --arg key "${key}" '.toolchain[$key] // empty' "${profile_json}" 2>/dev/null
}

profile_has_local_nbdkit_toolchain() {
local root="$1" profile="$2"
[[ -n "$(profile_toolchain_value "${root}" "${profile}" "nbdkit" || true)" ]]
}

install_profile_template() {
local compat_repo_root="$1" compat_install_root="$2" profile="$3"
local src="${compat_repo_root}/${profile}"
Expand All @@ -462,6 +488,7 @@ install_profile_template() {
mkdir -p "${dst}"
[[ -f "${dst}/bin/govc" ]] && chmod 0755 "${dst}/bin/govc" 2>/dev/null || true
[[ -f "${dst}/venv/bin/python3" ]] && chmod 0755 "${dst}/venv/bin/python3" 2>/dev/null || true
[[ -f "${dst}/nbdkit/bin/nbdkit" ]] && chmod 0755 "${dst}/nbdkit/bin/nbdkit" 2>/dev/null || true
return 0
fi

Expand All @@ -470,6 +497,7 @@ install_profile_template() {
cp -a "${src}" "${dst}"
[[ -f "${dst}/bin/govc" ]] && chmod 0755 "${dst}/bin/govc" 2>/dev/null || true
[[ -f "${dst}/venv/bin/python3" ]] && chmod 0755 "${dst}/venv/bin/python3" 2>/dev/null || true
[[ -f "${dst}/nbdkit/bin/nbdkit" ]] && chmod 0755 "${dst}/nbdkit/bin/nbdkit" 2>/dev/null || true
}

mark_sample_profile_install() {
Expand Down Expand Up @@ -540,6 +568,43 @@ install_vddk_into_profile() {
echo "[OK] VDDK installed: ${dst}"
}

install_nbdkit_into_profile() {
local root="$1" compat_install_root="$2" profile="$3"
local tgz tmp runtime dst
tgz="$(resolve_profile_nbdkit_asset "${root}" "${profile}")"
if [[ -z "${tgz}" ]]; then
if profile_has_local_nbdkit_toolchain "${root}" "${profile}"; then
echo "[WARN] legacy nbdkit asset not found for profile=${profile}" >&2
return 1
fi
return 0
fi

echo "[INFO] Installing legacy nbdkit for profile=${profile} from ${tgz}"
tmp="$(mktemp -d)"
tar -xzf "${tgz}" -C "${tmp}"

runtime="$(find "${tmp}" -maxdepth 4 -type f -path '*/bin/nbdkit' -printf '%h\n' | sed 's#/bin$##' | head -n1 || true)"
[[ -n "${runtime}" && -x "${runtime}/bin/nbdkit" ]] || {
echo "[ERR] nbdkit binary not found inside ${tgz}" >&2
rm -rf "${tmp}"
return 2
}
[[ -f "${runtime}/lib/nbdkit/plugins/nbdkit-vddk-plugin.so" ]] || {
echo "[ERR] nbdkit-vddk-plugin.so not found inside ${tgz}" >&2
rm -rf "${tmp}"
return 2
}

dst="${compat_install_root}/${profile}/nbdkit"
rm -rf "${dst}"
mkdir -p "$(dirname "${dst}")"
cp -a "${runtime}" "${dst}"
chmod 0755 "${dst}/bin/nbdkit" "${dst}/lib/nbdkit/plugins/nbdkit-vddk-plugin.so" 2>/dev/null || true
rm -rf "${tmp}"
echo "[OK] legacy nbdkit installed: ${dst}"
}

install_pyvmomi_into_profile() {
local root="$1" compat_install_root="$2" profile="$3"
local wheel_dir profile_root python_bin pip_bin
Expand Down Expand Up @@ -658,6 +723,49 @@ check_profile_vddk() {
echo "[OK] profile=${profile} vddk=${vddk}"
}

vddk_ld_library_path() {
local vddk="$1"
local out=""
[[ -d "${vddk}/lib64" ]] && out="${vddk}/lib64"
[[ -d "${vddk}" ]] && out="${out:+${out}:}${vddk}"
[[ -n "${LD_LIBRARY_PATH:-}" ]] && out="${out:+${out}:}${LD_LIBRARY_PATH}"
printf '%s' "${out}"
}

check_profile_nbdkit() {
local compat_install_root="$1" profile="$2"
local profile_json="${compat_install_root}/${profile}/profile.json"
local nbdkit_rel plugin_rel nbdkit_bin plugin vddk
nbdkit_rel="$(jq -r '.toolchain.nbdkit // empty' "${profile_json}" 2>/dev/null || true)"
plugin_rel="$(jq -r '.toolchain.nbdkit_vddk_plugin // empty' "${profile_json}" 2>/dev/null || true)"
[[ -n "${nbdkit_rel}" ]] || return 0

nbdkit_bin="${compat_install_root}/${profile}/${nbdkit_rel}"
[[ -x "${nbdkit_bin}" ]] || {
echo "[ERR] profile=${profile} nbdkit not found: ${nbdkit_bin}" >&2
return 1
}

if [[ -n "${plugin_rel}" ]]; then
plugin="${compat_install_root}/${profile}/${plugin_rel}"
[[ -f "${plugin}" ]] || {
echo "[ERR] profile=${profile} nbdkit VDDK plugin not found: ${plugin}" >&2
return 1
}

vddk="${compat_install_root}/${profile}/vddk"
LD_LIBRARY_PATH="$(vddk_ld_library_path "${vddk}")" \
"${nbdkit_bin}" "${plugin}" --dump-plugin libdir="${vddk}" >/dev/null 2>&1 || {
echo "[ERR] profile=${profile} nbdkit cannot load VDDK from ${vddk}" >&2
echo " Check legacy nbdkit/VDDK ABI compatibility before running migration." >&2
return 1
}
echo "[OK] profile=${profile} nbdkit=${nbdkit_bin} vddk-plugin-load=ok"
else
echo "[OK] profile=${profile} nbdkit=${nbdkit_bin}"
fi
}

check_profile_json() {
local compat_install_root="$1" profile="$2"
local profile_json="${compat_install_root}/${profile}/profile.json"
Expand Down Expand Up @@ -692,6 +800,7 @@ validate_installed_profile() {
else
check_profile_python "${compat_install_root}" "${profile}"
check_profile_vddk "${compat_install_root}" "${profile}"
check_profile_nbdkit "${compat_install_root}" "${profile}"
fi
}

Expand All @@ -709,6 +818,9 @@ install_profile_assets() {
install_vddk_into_profile "${root}" "${compat_install_root}" "${profile}" || {
[[ "${require_assets}" -eq 0 ]] || return $?
}
install_nbdkit_into_profile "${root}" "${compat_install_root}" "${profile}" || {
[[ "${require_assets}" -eq 0 ]] || return $?
}
install_pyvmomi_into_profile "${root}" "${compat_install_root}" "${profile}"
}

Expand Down Expand Up @@ -750,16 +862,27 @@ main() {
exit 2
fi

if [[ "${INSTALL_SAMPLE_PROFILES}" -eq 0 || "${INSTALL_ASSETS}" -eq 1 ]]; then
check_nbdkit_vddk_plugin
fi

local profiles_to_process=()
while IFS= read -r profile; do
[[ -n "${profile}" ]] || continue
profiles_to_process+=("${profile}")
done < <(resolve_profiles "${compat_repo_root}" "${INSTALL_PROFILE:-${VALIDATE_PROFILE:-all}}")

if [[ "${INSTALL_SAMPLE_PROFILES}" -eq 0 || "${INSTALL_ASSETS}" -eq 1 ]]; then
local need_system_nbdkit=0
local profile
for profile in "${profiles_to_process[@]}"; do
if ! profile_has_local_nbdkit_toolchain "${asset_root}" "${profile}"; then
need_system_nbdkit=1
fi
done
if [[ "${need_system_nbdkit}" -eq 1 ]]; then
check_nbdkit_vddk_plugin
else
echo "[OK] selected profiles use profile-local nbdkit runtime"
fi
fi

if [[ "${INSTALL_SAMPLE_PROFILES}" -eq 1 ]]; then
local profile
for profile in "${profiles_to_process[@]}"; do
Expand Down
89 changes: 89 additions & 0 deletions docs/v2k/v2k_esxi55_vddk602_legacy_nbdkit_design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# v2k ESXi 5.5 VDDK 6.0.2 Legacy nbdkit Design

## Background

The `esxi55` compatibility profile intentionally uses VMware VDDK 6.0.2
because newer VDDK 6.5.x packages are no longer practically obtainable through
the current VMware/Broadcom download path.

The host system nbdkit VDDK plugin cannot be used with this VDDK. Runtime logs
from ESXi 5.5 validation showed that the system plugin fails before connecting
to vCenter or NFC:

```text
required VDDK symbol "VixDiskLib_Wait" is missing. VDDK version must be >= 6.5.
```

This means profile selection is working, but the selected VDDK is too old for
the installed system `nbdkit-vddk-plugin`.

## Decision

The `esxi55` profile must carry a profile-local legacy nbdkit runtime that can
load VDDK 6.0.2. The system nbdkit remains the default for other profiles.

The selected runtime layout is:

```text
/usr/share/ablestack/v2k/compat/esxi55/
bin/govc
venv/bin/python3
vddk/
nbdkit/
bin/nbdkit
lib/nbdkit/plugins/nbdkit-vddk-plugin.so
```

The legacy runtime asset is:

```text
assets/compat/esxi55/nbdkit-vddk-legacy-1.14.2-rocky9-x86_64.tar.gz
```

It contains nbdkit 1.14.2 server binary, `nbdkit-vddk-plugin.so`, and the
nbdkit license. It was built on Rocky Linux 9.7 x86_64 and verified with:

```bash
LD_LIBRARY_PATH=/path/to/vddk/lib64 \
/path/to/nbdkit/bin/nbdkit \
/path/to/nbdkit/lib/nbdkit/plugins/nbdkit-vddk-plugin.so \
--dump-plugin libdir=/path/to/vddk
```

## Runtime Rules

When `esxi55` is active:

- `VDDK_LIBDIR` points to the profile-local VDDK 6.0.2 directory.
- `V2K_NBDKIT_BIN` points to the profile-local legacy nbdkit binary.
- `V2K_NBDKIT_VDDK_PLUGIN` points to the profile-local VDDK plugin `.so`.
- transfer code invokes the plugin by absolute path, not by the short name
`vddk`, so the compiled default plugin directory in the legacy binary is not
relevant.
- `LD_LIBRARY_PATH` includes both `VDDK_LIBDIR/lib64` and `VDDK_LIBDIR`.

Other profiles continue to use the system `nbdkit` and the short plugin name
`vddk`.

## Installer Validation

The installer must not stop at checking whether `libvixDiskLib.so` exists.
For real profile assets it must also prove that the selected nbdkit runtime can
load the selected VDDK:

```bash
LD_LIBRARY_PATH="${VDDK_LIBDIR}/lib64:${VDDK_LIBDIR}" \
"${V2K_NBDKIT_BIN}" "${V2K_NBDKIT_VDDK_PLUGIN}" \
--dump-plugin libdir="${VDDK_LIBDIR}"
```

This catches ABI mismatches during installation instead of failing later in
`sync.base`.

## Remaining Risk

Passing `--dump-plugin` proves local ABI compatibility only. It does not prove
that ESXi 5.5 NFC transfer succeeds in the target environment. After this
change, ESXi 5.5 validation must be rerun through at least base sync to confirm
that the original NFC error is resolved or to expose the next compatibility
issue.
Loading
Loading