Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
278 changes: 278 additions & 0 deletions docs/development/changelog-bundle-registry.md

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions docs/development/toc.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
toc:
- file: index.md
- file: changelog-bundle-registry.md
- folder: ingest
children:
- file: index.md
Expand Down
66 changes: 66 additions & 0 deletions docs/syntax/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ The directive supports the following options:
| `:description-visibility: value` | Visibility of changelog **record** descriptions (YAML `description` on each entry) | `auto` |
| `:dropdowns:` | Render breaking changes, deprecations, known issues, and highlights as expandable dropdowns instead of flattened bulleted lists | false |
| `:config: path` | Path to `changelog.yml` configuration | auto-discover |
| `:cdn: [product]` | Render bundles for a product that is declared under `release_notes` in `docset.yml` and prefetched from the public changelog CDN. The product is optional and inferred from the current repository when omitted | (local folder) |
| `:version: target` | Render only the single bundle matching this target/version | (all versions) |

### Example with options

Expand Down Expand Up @@ -161,6 +163,70 @@ Explicit path to a `changelog.yml` or `changelog.yaml` configuration file, relat

Both explicit and auto-discovered paths must resolve within the repository checkout directory and must not traverse symlinks.

#### `:cdn:` [cdn]

Renders bundles for a single **product** that the docset sources from the public changelog CDN, so a docset can show release notes without vendoring bundle YAML. The directive is a *selector*: it renders bundles that docs-builder prefetched at startup, so the product must first be declared under [`release_notes`](#declaring-cdn-backed-products) in `docset.yml`.
Comment thread
cotti marked this conversation as resolved.

```yaml
# docset.yml
release_notes:
- product: elasticsearch
```

```markdown
:::{changelog}
:cdn: elasticsearch
:::
```

The value names a product defined in [`products.yml`](https://github.com/elastic/docs-builder/blob/main/config/products.yml) (syntactically it must match `[a-zA-Z0-9_-]+`). The value is **optional**: leave it blank to infer the product from the repository that holds the doc. The repository name is mapped to its canonical product id via `products.yml` (for example the `elastic-otel-java` repo renders the `edot-java` product).

```markdown
:::{changelog}
:cdn:
:::
```

If the product cannot be inferred, or is not declared under `release_notes`, the block emits an error rather than rendering empty. When `:cdn:` is set, the local-folder argument is ignored. All other options (`:type:`, `:link-visibility:`, `:description-visibility:`, `:dropdowns:`, `:subsections:`) and `hide-features` apply identically to CDN-sourced bundles.

The CDN base URL is build configuration, not authored per page: it defaults to the public changelog bundles distribution and can be overridden with the `DOCS_BUILDER_CHANGELOG_CDN` environment variable (an absolute `http`/`https` URL) for staging or local testing.

Bundles are fetched **once at build startup** for every declared product, not per directive. If a declared product's registry cannot be fetched the build fails; an individual bundle that is missing from the CDN is skipped with a warning. For the full design — including the manifest format and infrastructure — see [Changelog bundle registry and CDN delivery](/development/changelog-bundle-registry.md).

##### Declaring CDN-backed products [declaring-cdn-backed-products]

List each CDN-sourced product under `release_notes` in `docset.yml`. Every entry must reference a product id from `products.yml` that participates in the release-notes system:

```yaml
# docset.yml
release_notes:
- product: elasticsearch
- product: edot-java
```

docs-builder prefetches the registry and bundles for each declared product at startup. A `:cdn:` directive that names an undeclared product is an error, which keeps the set of network sources auditable in one place rather than discovered dynamically across pages.

#### `:version:` [version]

Renders only the **single** bundle whose target matches the given value, instead of every bundle for the source. A bundle matches when the value equals its declared `target` (for example `9.4.0`, or a date like `2026-04-09`) or its file name (with or without extension). Matching is case-insensitive.

```markdown
:::{changelog}
:version: 9.4.0
:::
```

This works for both local-folder and `:cdn:` sources. In `:cdn:` mode it filters the prefetched bundles down to the matching target at render time.

```markdown
:::{changelog}
:cdn: elasticsearch
:version: 9.4.0
:::
```

If no bundle matches, the directive renders nothing and emits a warning (it does not fall back to showing all versions).

## Filtering entries with bundle rules

You can filter changelog entries at bundle time using the `rules.bundle` configuration in your `changelog.yml` file. This is evaluated during `changelog bundle` and `changelog gh-release`, before the bundle is written. Entries that don't match are excluded from the bundle entirely.
Expand Down
6 changes: 5 additions & 1 deletion src/Elastic.Codex/Building/CodexBuildService.cs
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
using Elastic.Documentation.Configuration;
using Elastic.Documentation.Configuration.Builder;
using Elastic.Documentation.Configuration.Codex;
using Elastic.Documentation.Configuration.ReleaseNotes;
using Elastic.Documentation.Diagnostics;
using Elastic.Documentation.Isolated;
using Elastic.Documentation.LinkIndex;
Expand Down Expand Up @@ -221,8 +222,11 @@ public async Task<CodexBuildResult> BuildAll(
crossLinkResolver = new CrossLinkResolver(FetchedCrossLinks.Empty, uriResolver);
}

// Prefetch CDN-hosted release notes for products declared under `release_notes` in docset.yml.
var releaseNotesResolver = await ReleaseNotesFetcher.PrefetchAsync(buildContext, logFactory, ctx);

// Create documentation set
var documentationSet = new DocumentationSet(buildContext, logFactory, crossLinkResolver);
var documentationSet = new DocumentationSet(buildContext, logFactory, crossLinkResolver, releaseNotesResolver);
await documentationSet.ResolveDirectoryTree(ctx);

return new CodexDocumentationSetBuildContext(checkout, buildContext, documentationSet);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,13 @@ public record ConfigurationFile
/// </summary>
public CrossLinkEntry[] CrossLinkEntries { get; } = [];

/// <summary>
/// Canonical product ids declared under <c>release_notes</c> whose changelog content is sourced from
/// the public CDN. Validated against products.yml; drives startup prefetch for the <c>{changelog}</c>
/// directive and CDN sourcing for <c>changelog bundle</c>.
/// </summary>
public string[] ReleaseNotesProducts { get; } = [];

/// The maximum depth `toc.yml` files may appear
public int MaxTocDepth { get; } = 1;

Expand Down Expand Up @@ -138,6 +145,9 @@ public ConfigurationFile(DocumentationSetFile docSetFile, IDocumentationSetConte

CrossLinkRepositories = CrossLinkEntries.Select(e => e.Repository).ToArray();

// Parse and validate CDN-backed release-notes product declarations
ReleaseNotesProducts = ParseReleaseNotesProducts(docSetFile.ReleaseNotes, productsConfig, context);

// Extensions - assuming they're not in DocumentationSetFile yet
Extensions = new EnabledExtensions(docSetFile.Extensions);

Expand Down Expand Up @@ -371,6 +381,57 @@ private static BrandingConfiguration ValidateBranding(BrandingConfiguration bran
return imagePath;
}

private static string[] ParseReleaseNotesProducts(
IReadOnlyList<ReleaseNotesProductReference> references,
ProductsConfiguration productsConfig,
IDocumentationSetContext context)
{
if (references.Count == 0)
return [];

var products = new List<string>(references.Count);
foreach (var reference in references)
{
var product = reference.Product?.Trim();
if (string.IsNullOrEmpty(product))
{
context.EmitError(context.ConfigurationPath, "A 'release_notes' entry is missing a 'product' value.");
continue;
}

if (!IsValidProductId(product))
{
context.EmitError(context.ConfigurationPath,
$"Invalid 'release_notes' product '{product}'. Product ids must match [a-zA-Z0-9_-]+.");
continue;
}

// products.yml keys are hyphenated; accept the underscore variant for parity with `products`.
var normalized = product.Replace('_', '-');
if (!productsConfig.Products.TryGetValue(normalized, out var resolved))
{
context.EmitError(context.ConfigurationPath,
$"Unknown 'release_notes' product '{product}'. It must be a product id defined in products.yml.");
continue;
}

if (!resolved.Features.ReleaseNotes)
{
context.EmitError(context.ConfigurationPath,
$"Product '{product}' declared in 'release_notes' does not participate in the release-notes system (it lacks the 'release-notes' feature in products.yml).");
continue;
}

if (!products.Contains(resolved.Id, StringComparer.Ordinal))
products.Add(resolved.Id);
}

return [.. products];
}

private static bool IsValidProductId(string product) =>
product.Length > 0 && product.All(c => char.IsAsciiLetterOrDigit(c) || c is '_' or '-');

private static CrossLinkEntry? ParseCrossLinkEntry(string raw, DocSetRegistry docsetRegistry, IFileInfo configPath, IDocumentationContext context)
{
DocSetRegistry entryRegistry;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,12 @@
<ProjectReference Include="..\Elastic.Documentation.Tooling\Elastic.Documentation.Tooling.csproj" />
</ItemGroup>

<ItemGroup>
<AssemblyAttribute Include="System.Runtime.CompilerServices.InternalsVisibleTo">
<_Parameter1>Elastic.Markdown.Tests</_Parameter1>
</AssemblyAttribute>
</ItemGroup>

<ItemGroup>
<PackageReference Include="DotNet.Glob" />
<PackageReference Include="NetEscapades.EnumGenerators" />
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,64 @@ public IReadOnlyList<LoadedBundle> LoadBundles(
return loadedBundles;
}

/// <summary>
/// Loads bundles from in-memory YAML content rather than a folder. Used by the <c>changelog</c>
/// directive in <c>cdn:</c> mode, where bundle files are fetched over HTTP. CDN bundles are
/// self-contained (entries are inline), so no entry-file resolution against the filesystem occurs;
/// any file-only entry reference is skipped with a warning. Amend files are still merged by name.
/// </summary>
/// <param name="bundles">Bundle file name and raw YAML content pairs.</param>
/// <param name="emitWarning">Callback to emit warnings during loading.</param>
/// <returns>A list of successfully loaded bundles.</returns>
public IReadOnlyList<LoadedBundle> LoadBundlesFromContent(
IReadOnlyList<(string FileName, string Content)> bundles,
Action<string> emitWarning)
{
var loadedBundles = new List<LoadedBundle>(bundles.Count);

foreach (var (fileName, content) in bundles)
{
Bundle bundleData;
try
{
bundleData = ReleaseNotesSerialization.DeserializeBundle(content);
}
catch (YamlException e)
{
emitWarning($"Failed to parse changelog bundle '{fileName}': {e.Message}");
continue;
}

var version = GetVersionFromBundle(bundleData) ?? fileSystem.Path.GetFileNameWithoutExtension(fileName);
var repo = GetRepoFromBundle(bundleData);
var owner = GetOwnerFromBundle(bundleData);
var entries = ResolveInlineEntries(bundleData, fileName, emitWarning);

loadedBundles.Add(new LoadedBundle(version, repo, owner, bundleData, fileName, entries));
}

return MergeAmendFiles(loadedBundles);
}

/// <summary>Resolves only inline entries; file-only references (unresolvable without the changelog dir) are skipped with a warning.</summary>
private static List<ChangelogEntry> ResolveInlineEntries(Bundle bundleData, string fileName, Action<string> emitWarning)
{
var entries = new List<ChangelogEntry>(bundleData.Entries.Count);
foreach (var entry in bundleData.Entries)
{
if (!string.IsNullOrWhiteSpace(entry.Title) && entry.Type != null)
{
entries.Add(ReleaseNotesSerialization.ConvertBundledEntry(entry));
continue;
}

emitWarning(
$"Bundle '{fileName}' has a non-self-contained entry (file '{entry.File?.Name}'); CDN bundles must inline their entries. Skipping.");
}

return entries;
}

/// <summary>
/// Resolves entries from a bundle, loading from file references if needed.
/// </summary>
Expand Down Expand Up @@ -132,7 +190,7 @@ public IReadOnlyList<ChangelogEntry> FilterEntries(
/// </summary>
/// <param name="bundles">The sorted list of bundles to merge.</param>
/// <returns>A list of bundles where same-target bundles are merged.</returns>
public IReadOnlyList<LoadedBundle> MergeBundlesByTarget(IReadOnlyList<LoadedBundle> bundles)
public static IReadOnlyList<LoadedBundle> MergeBundlesByTarget(IReadOnlyList<LoadedBundle> bundles)
{
if (bundles.Count <= 1)
return bundles;
Expand Down
Loading
Loading