Adding in support for making any extension pipeline a "nightly" publishing pipeline #8740

Most of this comes from the discussion I had with Daniel Jurek. Yes _the_ Daniel Jurek.

Basically:
- We know it's a "nightly" build because it's scheduled. That's it.
- We don't push GitHub releases for nightlies (it's a lot of clutter). They only get pushed to our blob storage account.
- Nightlies have their own branch for the registry ('nightly') and their own registry (cli/azd/extensions/registry.nightly.json).

To enable this for an extension, just add a nightly publish trigger in Azure DevOps (not as part of the YAML itself). I've tested this with a few builds, and even did an upgrade after the extension was pushed:

Here's the latest build of the azd-demo that produced a nightly build: [dev.azure.com, publishing microsoft.azd.demo as a nightly build](https://dev.azure.com/azure-sdk/internal/_build/results?buildId=6460907&view=results)

First part of the work for #8729

Author: richardpark-msft

cli/azd/docs/extensions/extension-framework.md

@@ -84,6 +84,15 @@ azd extension source add -n dev -t url -l "https://aka.ms/azd/extensions/registr
 
 Extensions installed from the dev registry are automatically promoted to the main registry when a newer version becomes available there. See the [Dev/Experimental Extension Registry](./extension-resolution-and-versioning.md#devexperimental-extension-registry) section for full details on stability expectations, submission guidelines, promotion behavior, and troubleshooting.
 
+A separate **nightly** registry distributes always-latest, automatically built snapshots of first-party extensions (signed on Windows/macOS, built from `main`). To opt in:
+
+```bash
+# Add a new extension source name 'nightly' to your `azd` configuration.
+azd extension source add -n nightly -t url -l "https://raw.githubusercontent.com/Azure/azure-dev/nightly/cli/azd/extensions/registry.nightly.json"
+```
+
+See the [Nightly Extension Registry](./extension-resolution-and-versioning.md#nightly-extension-registry) section for version semantics, promotion behavior, and caveats.
+
 #### `azd extension source list`
 
 Displays a list of installed extension sources.

cli/azd/docs/extensions/extension-resolution-and-versioning.md

@@ -618,6 +618,54 @@ If the dev registry URL is unreachable (network issue, DNS failure), operations
 azd extension source remove dev
 ```
 
+## Nightly Extension Registry
+
+The nightly registry contains **automatically built, always-latest** development snapshots of first-party extensions. Each scheduled pipeline run rebuilds an extension from `main`, signs the Windows and macOS binaries, uploads them to an always-latest storage folder, and updates a single entry in the nightly registry. Installing a nightly always gives you the most recent nightly build available at that time.
+
+| Property | Main Registry | Nightly Registry |
+|----------|---------------|------------------|
+| URL | `https://aka.ms/azd/extensions/registry` | `https://raw.githubusercontent.com/Azure/azure-dev/nightly/cli/azd/extensions/registry.nightly.json` |
+| Source file | `cli/azd/extensions/registry.json` (on `main`) | `cli/azd/extensions/registry.nightly.json` (on the `nightly` branch) |
+| Source name | `azd` (built-in default) | `nightly` (opt-in) |
+| Version shape | `1.2.3` | `1.2.3-nightly.<buildId>` (or `1.2.3-preview.nightly.<buildId>`) |
+| Signed binaries | Yes | Windows/macOS signed; Linux unsigned |
+| History retained | Yes | No — only the latest nightly per extension |
+| Support | Covered by Azure support | **Not covered** |
+
+> [!CAUTION]
+> Nightly extensions are built from `main` and come with **no stability guarantees**. Only the current nightly version is retained - older nightly versions are not installable.
+
+### Adding the Nightly Registry
+
+The nightly registry must be added, manually. To opt in:
+
+```bash
+# Add the nightly registry as a source named "nightly"
+azd extension source add -n nightly -t url -l "https://raw.githubusercontent.com/Azure/azure-dev/nightly/cli/azd/extensions/registry.nightly.json"
+```
+
+Then, to install a nightly-built extension:
+
+```bash
+azd extension install <extension-id> --source nightly
+```
+
+To remove the nightly registry later:
+
+```bash
+azd extension source remove nightly
+```
+
+### Upgrade and Nightly→Main Promotion
+
+Nightly versions use semver prerelease labels, so the standard `azd extension upgrade` flow works:
+
+- A newer nightly (higher build id, or a higher base version) supersedes an older one, so `azd extension upgrade` pulls the latest nightly.
+- When the extension ships a **stable** release whose base version matches your nightly (for example stable `1.2.3` versus `1.2.3-nightly.200`), the stable release outranks the nightly and you are **automatically promoted** to the `azd` registry on your next upgrade.
+
+> [!NOTE]
+> If your nightly was built from a **prerelease** base (for example `1.2.3-preview.nightly.60`), it sorts **above** the matching stable prerelease `1.2.3-preview`. In that case you are not promoted until the stable registry advances to a higher base version. This is expected semver precedence behavior.
+
 ## Related Documentation
 
 | Document | Description |

cli/azd/pkg/extensions/nightly_versioning_test.go

@@ -0,0 +1,167 @@
+// Copyright (c) Microsoft Corporation. All rights reserved.
+// Licensed under the MIT License.
+
+package extensions
+
+import (
+	"testing"
+
+	"github.com/stretchr/testify/require"
+)
+
+// Test_UpdateChecker_NightlyVersions verifies that the nightly version scheme
+// (X.Y.Z-nightly.<buildId> and X.Y.Z-preview.nightly.<buildId>) produces the
+// expected "has update" results through the same semver comparison azd uses at
+// runtime. These pin the upgrade behavior we rely on for nightlies.
+func Test_UpdateChecker_NightlyVersions(t *testing.T) {
+	tests := []struct {
+		name       string
+		installed  string
+		available  []string // available versions in the source (latest is the max semver)
+		wantUpdate bool
+	}{
+		{
+			name:       "newer nightly supersedes older nightly",
+			installed:  "1.2.3-nightly.100",
+			available:  []string{"1.2.3-nightly.100", "1.2.3-nightly.200"},
+			wantUpdate: true,
+		},
+		{
+			name:       "same nightly is not an update",
+			installed:  "1.2.3-nightly.222",
+			available:  []string{"1.2.3-nightly.222"},
+			wantUpdate: false,
+		},
+		{
+			name:       "base version bump supersedes older nightly",
+			installed:  "1.2.3-nightly.999",
+			available:  []string{"1.2.3-nightly.999", "1.2.4-nightly.1"},
+			wantUpdate: true,
+		},
+		{
+			name:       "stable release supersedes nightly of same base",
+			installed:  "1.2.3-nightly.200",
+			available:  []string{"1.2.3-nightly.200", "1.2.3"},
+			wantUpdate: true,
+		},
+		{
+			// Documents the prerelease-base caveat: a nightly built off a
+			// prerelease base sorts ABOVE the matching stable prerelease, so a
+			// user on the stable preview sees the nightly as an update.
+			name:       "nightly off preview base outranks stable preview",
+			installed:  "1.2.3-preview",
+			available:  []string{"1.2.3-preview", "1.2.3-preview.nightly.60"},
+			wantUpdate: true,
+		},
+		{
+			name:       "newer preview nightly supersedes older preview nightly",
+			installed:  "1.2.3-preview.nightly.50",
+			available:  []string{"1.2.3-preview.nightly.50", "1.2.3-preview.nightly.60"},
+			wantUpdate: true,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			t.Setenv("AZD_CONFIG_DIR", t.TempDir())
+
+			cacheManager, err := NewRegistryCacheManager()
+			require.NoError(t, err)
+
+			ctx := t.Context()
+			sourceName := "nightly"
+
+			versions := make([]ExtensionVersion, 0, len(tt.available))
+			for _, v := range tt.available {
+				versions = append(versions, ExtensionVersion{Version: v})
+			}
+
+			err = cacheManager.Set(ctx, sourceName, []*ExtensionMetadata{
+				{
+					Id:          "test.extension",
+					DisplayName: "Test Extension",
+					Versions:    versions,
+				},
+			})
+			require.NoError(t, err)
+
+			updateChecker := NewUpdateChecker(cacheManager)
+
+			result, err := updateChecker.CheckForUpdate(ctx, &Extension{
+				Id:          "test.extension",
+				DisplayName: "Test Extension",
+				Version:     tt.installed,
+				Source:      sourceName,
+			})
+			require.NoError(t, err)
+			require.Equal(t, tt.wantUpdate, result.HasUpdate)
+		})
+	}
+}
+
+// Test_ResolveUpgradeSource_NightlyPromotion verifies how a nightly-sourced
+// install promotes (or not) to the stable "azd" registry given the chosen
+// version strings. Promotion happens only when the stable registry's latest
+// version is strictly greater (semver) than the installed nightly's.
+func Test_ResolveUpgradeSource_NightlyPromotion(t *testing.T) {
+	makeExt := func(source string, versions ...string) *ExtensionMetadata {
+		ext := &ExtensionMetadata{Id: "test.extension", Source: source}
+		for _, v := range versions {
+			ext.Versions = append(ext.Versions, ExtensionVersion{Version: v})
+		}
+		return ext
+	}
+
+	tests := []struct {
+		name          string
+		nightlyLatest string
+		mainLatest    string // empty => extension not in the stable registry
+		wantPromotion bool
+		wantSource    string
+	}{
+		{
+			name:          "stable release promotes nightly of same base",
+			nightlyLatest: "1.2.3-nightly.200",
+			mainLatest:    "1.2.3",
+			wantPromotion: true,
+			wantSource:    MainRegistryName,
+		},
+		{
+			name:          "nightly off preview base stays on nightly (outranks stable preview)",
+			nightlyLatest: "1.2.3-preview.nightly.60",
+			mainLatest:    "1.2.3-preview",
+			wantPromotion: false,
+			wantSource:    "nightly",
+		},
+		{
+			name:          "higher stable base promotes preview nightly",
+			nightlyLatest: "1.2.3-preview.nightly.60",
+			mainLatest:    "1.2.4",
+			wantPromotion: true,
+			wantSource:    MainRegistryName,
+		},
+		{
+			name:          "no stable entry keeps user on nightly",
+			nightlyLatest: "1.2.3-nightly.200",
+			mainLatest:    "",
+			wantPromotion: false,
+			wantSource:    "nightly",
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			installed := &Extension{Id: "test.extension", Source: "nightly"}
+
+			allMatches := []*ExtensionMetadata{makeExt("nightly", tt.nightlyLatest)}
+			if tt.mainLatest != "" {
+				allMatches = append(allMatches, makeExt(MainRegistryName, tt.mainLatest))
+			}
+
+			result := ResolveUpgradeSource(installed, allMatches, "")
+			require.NotNil(t, result)
+			require.Equal(t, tt.wantPromotion, result.IsPromotion)
+			require.Equal(t, tt.wantSource, result.NewSource)
+		})
+	}
+}