Skip to content

Commit 9f5e205

Browse files
committed
feat(migration): cut appliance over to my collect
Completes the migration started by the dual-send PR. After this commit, type=enterprise units talk directly to the my collect API with native my credentials; neither the legacy my.nethesis.it /isa/ and /api/ endpoints nor the /proxy/* translation routes remain on the hot path. type=community units are left on the legacy my.nethserver.com / backupd infrastructure — that is explicitly out of scope for the my migration. Credential rotation (existing enterprise units): - New /usr/sbin/migrate-to-my: idempotent one-shot, gated on type=enterprise. Calls the translation proxy's /proxy/credentials with the legacy Basic-Auth pair, reads back the mapped my system_key/system_secret and atomically rotates ns-plug.config. Preserves the legacy pair under legacy_system_id / legacy_secret for audit and manual rollback, (re)asserts collect_url because /etc/config/ns-plug is a conffile, and sets the migrated='1' marker that stops the helper from running again. - send-heartbeat / send-inventory / send-backup / remote-backup invoke migrate-to-my up front on the enterprise branch so the first successful cron tick flips a pre-migration unit over. Native my registration (fresh enterprise subscriptions): - /usr/sbin/register enterprise branch now POSTs my.nethesis.it/backend/api/systems/register with {system_secret: <pasted>} and reads back system_key. The unit lands on the new my with its native credentials, collect_url is written alongside the legacy URLs, and migrated='1' is set so migrate-to-my is a no-op. Community register is untouched; it keeps calling my.nethserver.com /api/machine/info. - /usr/sbin/subscription-info enterprise branch reads from collect /info with the rotated credentials and emits a legacy-shaped envelope so ns.subscription info and the existing Vue UI keep parsing the same keys. The new my data model no longer tracks a subscription plan at the system level, so plan_name falls back to the organization name and valid_until is null (the UI treats that as "no expiration"). Single-path send scripts: - send-heartbeat enterprise: POST $collect_url/heartbeat with rotated Basic-Auth. The old my-old /isa/ primary + proxy shadow dual-send is gone; community continues on my.nethserver.com/api/machine/heartbeats/store. - send-inventory enterprise: POST $collect_url/inventory with a phonehome payload. The my-old /isa/ primary, the /api/systems/info registration-date refresh and the proxy shadow are gone; community continues on my.nethserver.com. - send-backup / remote-backup enterprise: single upload to $collect_url/backups with native creds, via remote-backup's upload/download/list/delete (which includes the backup UI mapping). Community remote-backup keeps the legacy $backup_url/$TYPE/api/v2/backup/ layout untouched. Config & packaging: - ns-plug.config.collect_url default added so fresh enterprise images carry the new endpoint; backup_url default restored so fresh community installs keep the legacy backupd URL. - DEPENDS: +jq (used by remote-backup's list parser, migrate-to-my and subscription-info). Failure mode: - A /proxy/credentials outage during an enterprise upgrade window leaves the unit on legacy credentials against collect, which returns 401. migrate-to-my is re-invoked every 10 minutes via send-heartbeat's cron entry, so the unit recovers automatically once the proxy is back up. Accepted trade-off: no dual-mode in the scripts; the simpler single-send path is preferred.
1 parent 03728c8 commit 9f5e205

10 files changed

Lines changed: 335 additions & 86 deletions

File tree

packages/ns-api/files/ns.backup

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -170,8 +170,14 @@ elif cmd == 'call':
170170
print(json.dumps(utils.generic_error(f'remote list failed')))
171171

172172
elif action == 'registered-backup':
173-
if os.path.exists(PASSPHRASE_PATH):
174-
print(utils.validation_error('passphrase', 'missing'))
173+
if not os.path.exists(PASSPHRASE_PATH):
174+
# Refuse the call before running sysupgrade/uploading, and emit
175+
# valid JSON so the HTTP API wraps it as a 422 ValidationError
176+
# the UI can render (the previous form printed a Python dict
177+
# repr, which was silently dropped upstream and caused the run
178+
# modal to stay open after a successful upload).
179+
print(json.dumps(utils.validation_error('passphrase', 'missing')))
180+
sys.exit(0)
175181
try:
176182
# create backup
177183
file_name = create_backup()
@@ -225,10 +231,12 @@ elif cmd == 'call':
225231
elif action == 'registered-delete-backup':
226232
try:
227233
data = json.load(sys.stdin)
228-
p = subprocess.run(['/usr/sbin/remote-backup', 'delete', data['id']],
234+
subprocess.run(['/usr/sbin/remote-backup', 'delete', data['id']],
229235
check=True, capture_output=True, text=True)
230-
# return content
231-
print(p.stdout)
236+
# The remote side returns a structured JSON response; the UI
237+
# only needs a success flag, matching the pattern of the
238+
# other registered-* handlers (backup, restore).
239+
print(json.dumps({'message': 'success'}))
232240
except subprocess.CalledProcessError as error:
233241
print(json.dumps(utils.generic_error('remote backup delete failed')))
234242
except KeyError as error:

packages/ns-plug/Makefile

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ define Package/ns-plug
2121
CATEGORY:=NethSecurity
2222
TITLE:=NethSecurity controller client
2323
URL:=https://github.com/NethServer/nethsecurity-controller/
24-
DEPENDS:=+openvpn +lscpu +python3-nethsec +python3-yaml
24+
DEPENDS:=+openvpn +lscpu +python3-nethsec +python3-yaml +jq
2525
PKGARCH:=all
2626
endef
2727

@@ -75,6 +75,7 @@ define Package/ns-plug/install
7575
$(INSTALL_BIN) ./files/ns-plug.init $(1)/etc/init.d/ns-plug
7676
$(INSTALL_BIN) ./files/ns-plug $(1)/usr/sbin/ns-plug
7777
$(INSTALL_BIN) ./files/distfeed-setup $(1)/usr/sbin/distfeed-setup
78+
$(INSTALL_BIN) ./files/migrate-to-my $(1)/usr/sbin
7879
$(INSTALL_BIN) ./files/remote-backup $(1)/usr/sbin
7980
$(INSTALL_BIN) ./files/send-backup $(1)/usr/sbin
8081
$(INSTALL_BIN) ./files/send-heartbeat $(1)/usr/sbin

packages/ns-plug/files/config

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ config main 'config'
55
option unit_name ''
66
option tls_verify '1'
77
option backup_url 'https://backupd.nethesis.it'
8+
option collect_url 'https://my.nethesis.it/collect/api/systems'
89
option repository_url 'https://updates.nethsecurity.nethserver.org'
910
option channel ''
1011
option tun_mtu ''
Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
#!/bin/sh
2+
3+
#
4+
# Copyright (C) 2026 Nethesis S.r.l.
5+
# SPDX-License-Identifier: GPL-2.0-only
6+
#
7+
8+
#
9+
# Idempotent one-shot migration from legacy my.nethesis.it / backupd
10+
# credentials to the my collect native credentials, so the unit can
11+
# authenticate directly against the my collect endpoints.
12+
#
13+
# Only enterprise units are migrated. Community (my.nethserver.com)
14+
# has its own infrastructure and keeps using the legacy send-*
15+
# endpoints — no credential rotation is applicable there.
16+
#
17+
# ns-plug.config.migrated='1' is the persistent marker. It is written
18+
# by this script after a successful rotation, and also by
19+
# /usr/sbin/register when it registers a fresh unit directly against
20+
# the my collect endpoint (so a brand new install never triggers the
21+
# rotation path and never hits /proxy/credentials with unmapped
22+
# credentials).
23+
#
24+
# On the first successful invocation the script:
25+
# 1. Calls the my translation proxy's /proxy/credentials endpoint
26+
# with the legacy Basic-Auth pair and retrieves the mapped my
27+
# system key / secret.
28+
# 2. Writes the new credentials to ns-plug.config.system_id / secret
29+
# and preserves the legacy pair under legacy_system_id /
30+
# legacy_secret (for audit and manual rollback).
31+
# 3. Re-asserts ns-plug.config.collect_url, because /etc/config/
32+
# ns-plug is a conffile: on registered units opkg keeps the
33+
# user-modified copy across upgrades, so a new default alone
34+
# would not reach them.
35+
# 4. Sets the migrated='1' marker.
36+
#
37+
# The uci commit is atomic — a partial write cannot leave the unit in
38+
# an inconsistent half-migrated state.
39+
#
40+
41+
# Marker: set only after a successful rotation or a native my register.
42+
[ "$(uci -q get ns-plug.config.migrated)" = "1" ] && exit 0
43+
44+
# Community units stay on the legacy my.nethserver.com infrastructure.
45+
TYPE=$(uci -q get ns-plug.config.type)
46+
if [ "$TYPE" != "enterprise" ]; then
47+
exit 0
48+
fi
49+
50+
SYSTEM_ID=$(uci -q get ns-plug.config.system_id)
51+
SYSTEM_SECRET=$(uci -q get ns-plug.config.secret)
52+
if [ -z "$SYSTEM_ID" ] || [ -z "$SYSTEM_SECRET" ]; then
53+
# Unregistered unit — nothing to migrate yet.
54+
exit 0
55+
fi
56+
57+
# Fetch the mapped my credentials via the translation proxy.
58+
resp=$(/usr/bin/curl --silent --location-trusted --fail-with-body \
59+
--max-time 30 --retry 2 \
60+
--user "$SYSTEM_ID:$SYSTEM_SECRET" \
61+
https://my.nethesis.it/proxy/credentials 2>/dev/null) || {
62+
logger -t migrate-to-my "credential fetch failed; will retry on next run"
63+
exit 0
64+
}
65+
66+
new_key=$(echo "$resp" | jq -r '.data.system_key // empty' 2>/dev/null)
67+
new_secret=$(echo "$resp" | jq -r '.data.system_secret // empty' 2>/dev/null)
68+
if [ -z "$new_key" ] || [ -z "$new_secret" ]; then
69+
logger -t migrate-to-my "credentials missing in response"
70+
exit 0
71+
fi
72+
73+
# Rotate atomically; legacy pair preserved for audit / rollback.
74+
uci -q batch <<EOI
75+
set ns-plug.config.legacy_system_id=$SYSTEM_ID
76+
set ns-plug.config.legacy_secret=$SYSTEM_SECRET
77+
set ns-plug.config.system_id=$new_key
78+
set ns-plug.config.secret=$new_secret
79+
set ns-plug.config.collect_url=https://my.nethesis.it/collect/api/systems
80+
set ns-plug.config.migrated=1
81+
commit ns-plug
82+
EOI
83+
84+
logger -t migrate-to-my "migrated to my collect credentials"
85+
exit 0

packages/ns-plug/files/register

Lines changed: 16 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,12 @@ if [ -z "$secret" ]; then
3131
exit_error "Invalid secret"
3232
fi
3333

34-
# Setup URLs
34+
# Resolve the system identity from the pasted secret.
35+
#
36+
# Enterprise uses the my collect registration API directly — the
37+
# unit lands on the new my with its native credentials and never
38+
# needs the /proxy/credentials rotation helper. Community keeps
39+
# using my.nethserver.com as before.
3540
case "$type" in
3641
community)
3742
url="https://my.nethserver.com/api/"
@@ -42,11 +47,13 @@ case "$type" in
4247
"${url}machine/info" | jq -r ".uuid" 2>/dev/null)
4348
;;
4449
enterprise)
45-
url="https://my.nethesis.it/api/"
50+
url="https://my.nethesis.it/backend/api/"
4651

47-
system_id=$(curl -s -m $timeout --retry 3 -L \
52+
register_resp=$(curl -s -m $timeout --retry 3 -L \
4853
-H "Content-Type: application/json" -H "Accept: application/json" \
49-
-d '{"secret": "'$secret'"}' "${url}systems/info" | jq -r ".uuid" 2>/dev/null)
54+
-d '{"system_secret": "'$secret'"}' "${url}systems/register")
55+
56+
system_id=$(echo "$register_resp" | jq -r '.data.system_key // empty' 2>/dev/null)
5057
;;
5158
*)
5259
exit_error "Invalid type '$type'"
@@ -71,6 +78,11 @@ case "$type" in
7178
uci set ns-plug.config.alerts_url="https://my.nethesis.it/isa/"
7279
uci set ns-plug.config.api_url="$url"
7380
uci set ns-plug.config.inventory_url="https://my.nethesis.it/isa/inventory/store/"
81+
uci set ns-plug.config.collect_url="https://my.nethesis.it/collect/api/systems"
82+
# Native my register: no legacy credentials to rotate, so
83+
# mark the unit as already migrated. migrate-to-my will be a
84+
# no-op on every subsequent run.
85+
uci set ns-plug.config.migrated="1"
7486
;;
7587
esac
7688

packages/ns-plug/files/remote-backup

Lines changed: 94 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -6,64 +6,126 @@
66
#
77

88
#
9-
# Manage remote backup
9+
# Manage configuration backups.
1010
#
11+
# Enterprise units (type=enterprise) talk to my collect after the
12+
# migrate-to-my credential rotation. Community units (type=community)
13+
# keep using the legacy backupd.nethesis.it endpoint with the same
14+
# URL layout they have always used — backupd still accepts both
15+
# tenants behind the $TYPE/api/v2/backup/ path.
16+
#
17+
# Pipefail so the curl exit status survives the jq stage in `list`;
18+
# without it a HTTP error on the server would be masked by a successful
19+
# jq parse and ns.backup would report success to the UI.
20+
#
21+
22+
set -o pipefail
1123

1224
function exit_error {
1325
>&2 echo "[ERROR] $@"
1426
exit 1
1527
}
1628

1729
function help {
18-
>&2 echo "Usage: $0 <list|download|upload>"
30+
>&2 echo "Usage: $0 <list|download|upload|delete>"
1931
>&2 echo "Commands:"
20-
>&2 echo " - list: retrieve the list of available backups from remote server"
21-
>&2 echo " - download <file> [output]: download the given backup, if 'output' is empty downloaded file will be named as as 'file'"
22-
>&2 echo " - upload <file>: upload the given backup"
32+
>&2 echo " - list: fetch the list of backups stored for this system"
33+
>&2 echo " - download <id> [output]: download the backup <id>; defaults to writing to a file named <id>"
34+
>&2 echo " - upload <file>: upload a backup file"
35+
>&2 echo " - delete <id>: remove the backup <id>"
2336
}
2437

2538
SYSTEM_ID=$(uci -q get ns-plug.config.system_id)
2639
SYSTEM_SECRET=$(uci -q get ns-plug.config.secret)
2740
TYPE=$(uci -q get ns-plug.config.type)
28-
URL=$(uci -q get ns-plug.config.backup_url)
2941

30-
if [ -z "$SYSTEM_ID" ] || [ -z "$SYSTEM_SECRET" ] || [ -z "$URL" ]; then
31-
exit_error "System ID, system secret or backup url not found. Please configure ns-plug."
42+
if [ -z "$SYSTEM_ID" ] || [ -z "$SYSTEM_SECRET" ]; then
43+
exit_error "System ID or system secret not found. Please configure ns-plug."
44+
fi
45+
46+
cmd=${1:-list}
47+
48+
if [ "$TYPE" = "enterprise" ]; then
49+
/usr/sbin/migrate-to-my
50+
51+
COLLECT_URL=$(uci -q get ns-plug.config.collect_url)
52+
if [ -z "$COLLECT_URL" ]; then
53+
exit_error "Collect URL not set. Pre-migration unit — retry later."
54+
fi
55+
56+
BASE="$COLLECT_URL/backups"
57+
# --fail-with-body: exit 22 on 4xx/5xx while still writing the body
58+
# so the caller can inspect the error payload.
59+
curl_args="--silent --location-trusted --fail-with-body --user $SYSTEM_ID:$SYSTEM_SECRET"
60+
61+
case "$cmd" in
62+
list)
63+
# my returns {code, message, data: {backups: [...]}} on
64+
# success. Unwrap `data` so ns.backup can pass it through as
65+
# {values: <output>} without double-nesting. Fall back to an
66+
# empty list on failure.
67+
response=$(curl $curl_args "$BASE")
68+
echo "$response" | jq 'if .data and (.data.backups // empty) then .data else {backups: []} end'
69+
;;
70+
download)
71+
file=$2
72+
[ -z "$file" ] && exit_error "No file specified"
73+
output=${3-$file}
74+
curl $curl_args -o "$output" "$BASE/$file"
75+
;;
76+
upload)
77+
file=$2
78+
[ -z "$file" ] && exit_error "No file specified"
79+
curl $curl_args -X POST \
80+
-H "Content-Type: application/octet-stream" \
81+
-H "X-Filename: $(basename "$file")" \
82+
--data-binary "@$file" \
83+
"$BASE"
84+
;;
85+
delete)
86+
file=$2
87+
[ -z "$file" ] && exit_error "No file specified"
88+
curl $curl_args -X DELETE "$BASE/$file"
89+
;;
90+
*)
91+
help
92+
;;
93+
esac
94+
95+
exit $?
96+
fi
97+
98+
# Community (legacy): backupd.nethesis.it with the /$TYPE/api/v2/backup/
99+
# URL layout. Unchanged from the pre-migration behaviour.
100+
URL=$(uci -q get ns-plug.config.backup_url)
101+
if [ -z "$URL" ]; then
102+
exit_error "Backup URL not set. Please configure ns-plug."
32103
fi
33104

34105
curl_args="--silent --location-trusted --user $SYSTEM_ID:$SYSTEM_SECRET"
35106
base_url="$URL/$TYPE/api/v2/backup/"
36107

37-
cmd=${1:-list}
38-
39108
case "$cmd" in
40109
list)
41110
curl $curl_args $base_url
42111
;;
43112
download)
44113
file=$2
45-
if [ -z "$file" ]; then
46-
exit_error "No file specified"
47-
fi
114+
[ -z "$file" ] && exit_error "No file specified"
48115
output=${3-$file}
49116
curl $curl_args $base_url$file -J -o "$output"
50-
;;
51-
upload)
52-
file=$2
53-
if [ -z "$file" ]; then
54-
exit_error "No file specified"
55-
fi
56-
curl $curl_args $base_url --upload-file $file
57-
;;
58-
delete)
59-
file=$2
60-
if [ -z "$file" ]; then
61-
exit_error "No file specified"
62-
fi
63-
curl $curl_args -X DELETE $base_url$file
64-
;;
65-
66-
*)
67-
help
68-
;;
117+
;;
118+
upload)
119+
file=$2
120+
[ -z "$file" ] && exit_error "No file specified"
121+
curl $curl_args $base_url --upload-file $file
122+
;;
123+
delete)
124+
file=$2
125+
[ -z "$file" ] && exit_error "No file specified"
126+
curl $curl_args -X DELETE $base_url$file
127+
;;
128+
*)
129+
help
130+
;;
69131
esac

packages/ns-plug/files/send-backup

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,12 @@ if [ -z "$SYSTEM_ID" ] || [ -z "$SYSTEM_SECRET" ]; then
3838
exit 0
3939
fi
4040

41+
# remote-backup handles the enterprise/community branching and calls
42+
# migrate-to-my when needed; this script just prepares the payload and
43+
# delegates the upload. An enterprise unit still waiting on the
44+
# migration surfaces its error through remote-backup, which is caught
45+
# by set -e above.
46+
4147
# hack: avoid to backup non-config file
4248
if [ -f /etc/acme/http.header ]; then
4349
mv /etc/acme/http.header /tmp

0 commit comments

Comments
 (0)