Skip to content

Commit 0d9da1a

Browse files
shwstpprshwstppr
committed
extensions: sync and download
For: apache/cloudstack#11814 Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
1 parent 7f677cc commit 0d9da1a

File tree

2 files changed

+259
-0
lines changed

2 files changed

+259
-0
lines changed

‎source/adminguide/extensions.rst‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,8 @@ CloudStack provides built-in Orchestrator Extensions for Proxmox, Hyper-V, and M
9494

9595
.. include:: extensions/inbuilt_extensions.rst
9696

97+
.. include:: extensions/sync_download.rst
98+
9799
.. include:: extensions/limitations.rst
98100

99101
.. include:: extensions/troubleshooting.rst

‎source/adminguide/extensions/sync_download.rst‎

Lines changed: 257 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,257 @@
1+
.. Licensed to the Apache Software Foundation (ASF) under one
2+
or more contributor license agreements. See the NOTICE file
3+
distributed with this work for additional information#
4+
regarding copyright ownership. The ASF licenses this file
5+
to you under the Apache License, Version 2.0 (the
6+
"License"); you may not use this file except in compliance
7+
with the License. You may obtain a copy of the License at
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
Unless required by applicable law or agreed to in writing,
10+
software distributed under the License is distributed on an
11+
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
12+
KIND, either express or implied. See the License for the
13+
specific language governing permissions and limitations
14+
under the License.
15+
16+
17+
Synchronization and Download
18+
============================
19+
20+
In clustered Apache CloudStack management server deployments, extension files
21+
must remain consistent across all management nodes. This release introduces
22+
built-in support for:
23+
24+
* Synchronizing extension files across management servers
25+
* Downloading extension files as an archive
26+
* Secure file sharing via Secondary Storage (SSVM) when direct access to a
27+
management server is not possible
28+
29+
These capabilities are available via new APIs and UI actions.
30+
31+
Share Endpoint Configuration
32+
----------------------------
33+
34+
Extension download and synchronization workflows rely on the management
35+
server's **share endpoint**, which enables controlled file sharing.
36+
37+
The share endpoint is configured in ``server.properties`` and allows
38+
administrators to enable or disable file sharing, define the base directory,
39+
configure cache behavior, and secure download links.
40+
41+
Share Endpoint Properties
42+
~~~~~~~~~~~~~~~~~~~~~~~~~
43+
44+
+----------------------+----------------------------------------------+---------------------------------------------+
45+
| Property | Default Value | Description |
46+
+======================+==============================================+=============================================+
47+
| share.enabled | true | Enables or disables the file sharing |
48+
| | | feature. Must be ``true`` for extension |
49+
| | | downloads to function. |
50+
+----------------------+----------------------------------------------+---------------------------------------------+
51+
| share.base.dir | <HOME_DIRECTORY_OF_CLOUD_USER>/share | Base directory from which files can be |
52+
| | | shared. If not explicitly set, the default |
53+
| | | directory under the CloudStack user home |
54+
| | | is used. |
55+
+----------------------+----------------------------------------------+---------------------------------------------+
56+
| share.cache.control | public,max-age=86400,immutable | Cache-Control header value applied to |
57+
| | | shared files. Controls browser/client |
58+
| | | caching behavior. |
59+
+----------------------+----------------------------------------------+---------------------------------------------+
60+
| share.secret | change-me | Secret key used to generate HMAC-signed |
61+
| | | download links. It is strongly recommended |
62+
| | | to change this value in production. If not |
63+
| | | set, links will not be signed. |
64+
+----------------------+----------------------------------------------+---------------------------------------------+
65+
66+
Notes:
67+
68+
* ``share.enabled`` must be ``true`` for extension downloads to function.
69+
* If ``share.secret`` is configured, generated links are signed using HMAC.
70+
* It is strongly recommended to replace the default ``share.secret`` value
71+
in production environments.
72+
* If ``share.secret`` is not configured, download links will not be signed.
73+
74+
Extension Synchronization
75+
^^^^^^^^^^^^^^^^^^^^^^^^^
76+
77+
In a multi-node management server cluster, extension files are stored locally
78+
on each management server. If an extension is installed or modified on one
79+
node, it must be synchronized to peers.
80+
81+
syncExtension API
82+
~~~~~~~~~~~~~~~~~
83+
84+
The ``syncExtension`` API triggers synchronization of a selected extension
85+
across all management servers in the cluster.
86+
87+
How It Works
88+
~~~~~~~~~~~~
89+
90+
1. The source management server:
91+
92+
* Calculates checksums for all files in the extension directory.
93+
* Packages the extension directory (or selected files) into a ``.tgz`` archive.
94+
95+
2. The server sends a ``DownloadAndSyncExtensionFilesCommand`` to peer
96+
management servers.
97+
98+
3. Each peer management server:
99+
100+
* Downloads the archive.
101+
* Stages it temporarily.
102+
* Extracts the files into the extension directory.
103+
* Validates checksums.
104+
* Cleans up temporary files.
105+
106+
Checksum Validation
107+
~~~~~~~~~~~~~~~~~~~
108+
109+
Checksums are calculated for all files in the extension directory and compared
110+
during synchronization. This ensures:
111+
112+
* File integrity
113+
* Detection of missing or modified files
114+
* Cluster-wide consistency
115+
116+
UI Support
117+
~~~~~~~~~~
118+
119+
In the CloudStack UI:
120+
121+
1. Navigate to **Extensions**
122+
2. Select an extension
123+
3. Click **Sync Extension**
124+
125+
This triggers synchronization across all management servers.
126+
127+
Downloading Extension Files
128+
^^^^^^^^^^^^^^^^^^^^^^^^^^^
129+
130+
You can download an extension’s files as an archive for backup, inspection,
131+
or migration.
132+
133+
downloadExtension API
134+
~~~~~~~~~~~~~~~~~~~~~
135+
136+
The ``downloadExtension`` API allows administrators to download the complete
137+
extension directory as a ``.zip`` archive.
138+
139+
How It Works
140+
~~~~~~~~~~~~
141+
142+
1. The management server creates a ``.zip`` archive of the extension directory.
143+
2. A secure share URL is generated.
144+
3. The administrator downloads the archive using the signed URL.
145+
146+
Secure Share URL
147+
~~~~~~~~~~~~~~~~
148+
149+
Download URLs are:
150+
151+
* Signed using HMAC
152+
* Time-bound (expire automatically)
153+
* Cleaned up after expiry
154+
155+
The validity interval is controlled via the global configuration:
156+
157+
.. code-block:: text
158+
159+
extension.share.link.validity.interval
160+
161+
This defines how long (in seconds) the signed URL remains valid before
162+
automatic cleanup.
163+
164+
Download via Secondary Storage (SSVM)
165+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
166+
167+
In some environments, a management server may not be directly accessible from
168+
the administrator’s network.
169+
170+
To support this scenario:
171+
172+
* The archive can be staged on Secondary Storage.
173+
* The signed URL points to the SSVM endpoint.
174+
* The file is served via Secondary Storage instead of directly from the
175+
management server.
176+
177+
Global Configuration
178+
~~~~~~~~~~~~~~~~~~~~
179+
180+
To force downloads via Secondary Storage:
181+
182+
.. code-block:: text
183+
184+
extension.share.download.use.secondary.storage = true
185+
186+
When set to ``true``, extension downloads are always served via Secondary
187+
Storage (SSVM).
188+
189+
When set to ``false`` (default), downloads are served directly from the
190+
selected management server.
191+
192+
Global Configuration Parameters
193+
--------------------------------
194+
195+
+-----------------------------------------------+--------------------------------------------------------------+
196+
| Name | Description |
197+
+===============================================+==============================================================+
198+
| extension.share.download.use.secondary.storage| If ``true``, forces extension downloads via Secondary |
199+
| | Storage (SSVM). |
200+
+-----------------------------------------------+--------------------------------------------------------------+
201+
| extension.share.link.validity.interval | Validity duration (in seconds) of the signed download URL. |
202+
+-----------------------------------------------+--------------------------------------------------------------+
203+
204+
Events and Logging
205+
------------------
206+
207+
The following improvements are included:
208+
209+
* Dedicated events for sync and download operations
210+
* Improved logging for archive creation, transfer, extraction, and cleanup
211+
* Automatic cleanup of temporary share files after expiry
212+
213+
These changes improve traceability and operational visibility.
214+
215+
Archive Formats
216+
---------------
217+
218+
+---------------+----------+
219+
| Operation | Format |
220+
+===============+==========+
221+
| Synchronize | ``.tgz`` |
222+
+---------------+----------+
223+
| Download | ``.zip`` |
224+
+---------------+----------+
225+
226+
The use of ``.tgz`` for synchronization ensures efficient transfer between
227+
management servers, while ``.zip`` is used for administrator downloads for
228+
broad compatibility.
229+
230+
Typical Workflows
231+
^^^^^^^^^^^^^^^^^
232+
233+
Synchronize Extension Across Cluster
234+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
235+
236+
1. Install or update the extension on one management server.
237+
2. Call the ``syncExtension`` API or use **Sync Extension** in the UI.
238+
3. Verify synchronization across all nodes.
239+
240+
Download Extension for Backup or Migration
241+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
242+
243+
1. Call the ``downloadExtension`` API or use **Download Extension** in the UI.
244+
2. Use the generated signed URL.
245+
3. Download the ``.zip`` archive before the URL expires.
246+
247+
Best Practices
248+
^^^^^^^^^^^^^^
249+
250+
* Always synchronize extensions after modifying files on a single management
251+
server in a clustered setup.
252+
* Use Secondary Storage download mode when management servers are behind
253+
private networks.
254+
* Monitor management server logs for sync failures due to permission or
255+
filesystem issues.
256+
* Adjust ``extension.share.link.validity.interval`` according to your
257+
operational security requirements.

0 commit comments

Comments
 (0)