feat(2397): add AI-generated "What's New" for release notes (#2417)

Author: ycanales

docs/commands.md

@@ -12,6 +12,8 @@
   - [`import_commits`](#import_commits)
   - [`update_issues`](#update_issues)
   - [`import_beta_release`](#import_beta_release)
+  - [`import_release_notes`](#import_release_notes)
+  - [`generate_whats_new`](#generate_whats_new)
   - [`sync_mailinglist_stats`](#sync_mailinglist_stats)
   - [`update_library_version_dependencies`](#update_library_version_dependencies)
   - [`release_tasks`](#release_tasks)
@@ -246,6 +248,51 @@ If both the `--release` and the `--library-name` are passed, the command will lo
 | `--delete-versions`  | bool  | If passed, all existing beta Version records will be deleted before the new beta release is imported. |
 
 
+## `import_release_notes`
+
+**Purpose**: Fetch the rendered release notes for Boost versions and store them in the `RenderedContent` cache (keyed `release_notes_boost-X-XX-X`). Tries the AsciiDoc source on S3 first, falls back to the legacy HTML in the `boostorg/website` GitHub repo. Also fetches the in-progress release notes.
+
+When a release note is freshly stored and the `Version.whats_new` field is empty, this command also queues the AI "What's New" summary task — see [`generate_whats_new`](#generate_whats_new).
+
+**Example**
+
+```bash
+./manage.py import_release_notes
+```
+
+**Options**
+
+| Options | Format | Description                                                                                  |
+|---------|--------|----------------------------------------------------------------------------------------------|
+| `--new` | bool   | Default: `true`. If `true`, only imports notes for the most recent version. Set to `false` to import for all active versions. |
+
+## `generate_whats_new`
+
+**Purpose**: Generate the AI-powered "What's New" draft summary for one or more Boost releases. The summary is a short, fixed-rubric bullet list (new libraries, performance, dependencies, security & reliability, developer experience) saved on the `Version` model as `whats_new` (markdown bullets). The public site parses the bullets into `whats_new_items` and renders them in the release-highlights card. Drafts are not shown on the public site until an admin sets `whats_new_approved=True` (also available as a Django admin action).
+
+This command is opt-in. Auto-generation only runs as a side-effect of `import_release_notes` when a version's `whats_new` is empty. Use this command to backfill historical versions or to regenerate after editing the prompt.
+
+The LLM call is a Celery task; the worker must be running and `OPENROUTER_API_KEY` must be set (see [Environment Variables](./env_vars.md)).
+
+**Example**
+
+```bash
+./manage.py generate_whats_new --all-missing
+./manage.py generate_whats_new --version boost-1-90-0 --force
+./manage.py generate_whats_new --validate --limit 10
+```
+
+**Options**
+
+| Options          | Format | Description                                                                                                |
+|------------------|--------|------------------------------------------------------------------------------------------------------------|
+| `--all-missing`  | bool   | Queue generation for every active version that has stored release notes but no `whats_new` summary yet.    |
+| `--version`      | string | Slug of a single version to (re)generate. Format: `boost-1-90-0`.                                          |
+| `--force`        | bool   | Regenerate even when a summary already exists. The chained save task overwrites `whats_new` and resets `whats_new_approved` to `False`, so regenerated content goes back through admin moderation. |
+| `--dry-run`      | bool   | List the versions that would be queued without queuing them.                                               |
+| `--validate`     | bool   | Run the prompt synchronously against the latest `--limit` versions (that have release notes) and print the LLM output. No DB writes. Use to review prompt changes before sign-off. |
+| `--limit`        | int    | Number of versions to process when `--validate` is set. Default: 10.                                       |
+
 ## `sync_mailinglist_stats`
 
 **Purpose**: Build EmailData objects from the hyperkitty email archive database.

docs/env_vars.md

@@ -82,3 +82,15 @@ This project uses environment variables to configure certain aspects of the appl
 
 ### `SLACK_BOT_TOKEN`
 - Used to authenticate with the Slack API for pulling data for release reports.
+
+## AI Summarization (OpenRouter)
+
+### `OPENROUTER_API_KEY`
+
+- API key for [OpenRouter](https://openrouter.ai), used by the `openai` SDK to reach the LLM that powers two features:
+  - News/blogpost/link entry summaries (`news/tasks.py`)
+  - The Boost release-notes "What's New" draft summary (`versions/tasks.py`)
+- Default model is `gpt-oss-120b`. To use a different model (e.g. a Claude model via OpenRouter), change `WHATS_NEW_MODEL` in `versions/tasks.py` and the per-handler model strings in `news/tasks.py`.
+- For **local development**, set this in your `.env` file. Note: docker compose only loads `env_file` at container creation, so after adding the variable run `docker compose up -d --force-recreate web celery-worker celery-beat` to pick it up.
+- In **deployed environments**, set as a kube secret in `kube/boost/values.yaml` (or the environment-specific yaml file).
+- Without this variable set, OpenRouter responds with `401 No cookie auth credentials found` and Celery retries the task up to 3 times before giving up.

docs/news.md

@@ -37,3 +37,9 @@ Users can moderate if:
 - The user posses the `change_entry` permission to the News Entry model
 - The user is in a group which posses the `change_entry` permission to the News Entry model
 - The user is a Superuser
+
+## AI-generated entry summaries
+
+When an `Entry` is saved without a `summary`, `news/tasks.py` dispatches a Celery task that asks an LLM (via [OpenRouter](https://openrouter.ai), default model `gpt-oss-120b`) to produce a short plain-text summary, then writes it back to the entry. Clearing the `summary` field and saving triggers regeneration.
+
+This is the same OpenRouter integration used by the Boost release-notes "What's New" summary in `versions/tasks.py`. Both share `OPENROUTER_API_KEY` — see [Environment Variables](./env_vars.md).

scripts/seed_whats_new_qa.py

@@ -0,0 +1,49 @@
+"""QA helper: seed a Boost Version + release-notes RenderedContent row so
+the `generate_whats_new` command has something to summarize.
+
+Run from the project root:
+
+    docker compose exec web ./manage.py shell < scripts/seed_whats_new_qa.py
+
+To target a different release, edit NAME / RELEASE_NOTES_URL below.
+"""
+
+import requests
+
+from core.models import RenderedContent
+from versions.models import Version
+
+NAME = "boost-1.89.0"
+RELEASE_NOTES_URL = (
+    "https://raw.githubusercontent.com/boostorg/website/master/"
+    "users/history/version_1_89_0.html"
+)
+
+version, _ = Version.objects.update_or_create(
+    name=NAME,
+    defaults={
+        "active": True,
+        "fully_imported": True,
+        "full_release": True,
+        "beta": False,
+        "github_url": f"https://github.com/boostorg/boost/releases/tag/{NAME}",
+    },
+)
+
+response = requests.get(RELEASE_NOTES_URL, timeout=30)
+response.raise_for_status()
+
+RenderedContent.objects.update_or_create(
+    cache_key=version.release_notes_cache_key,
+    defaults={
+        "content_type": "text/html",
+        "content_html": response.text,
+        "content_original": "",
+    },
+)
+
+print(f"Seeded {version.name} (slug={version.slug}, pk={version.pk})")
+print(f"Cache key: {version.release_notes_cache_key}")
+print(
+    f"Next:  docker compose exec web ./manage.py generate_whats_new --version={version.slug} --force"
+)

versions/admin.py

@@ -3,11 +3,13 @@
 from django.db.models.query import QuerySet
 from django.http import HttpRequest, HttpResponseRedirect
 from django.urls import path
+from django.utils.html import format_html, format_html_join
 
 from libraries.tasks import import_new_versions_tasks
 
 from . import models
 from .models import Version
+from .tasks import dispatch_whats_new
 
 
 class VersionFileInline(admin.StackedInline):
@@ -27,13 +29,66 @@ class VersionAdmin(admin.ModelAdmin):
         "beta",
         "fully_imported",
         "full_release",
+        "whats_new_approved",
     ]
-    list_filter = ["active", "full_release", "beta"]
+    list_filter = ["active", "full_release", "beta", "whats_new_approved"]
     ordering = ["-release_date", "-name"]
     search_fields = ["name", "description"]
     date_hierarchy = "release_date"
     inlines = [VersionFileInline]
     change_list_template = "admin/version_change_list.html"
+    readonly_fields = ["whats_new_items_display", "whats_new_generated_at"]
+    fieldsets = (
+        (
+            None,
+            {
+                "fields": (
+                    "name",
+                    "slug",
+                    "release_date",
+                    "description",
+                    "active",
+                    "github_url",
+                    "beta",
+                    "full_release",
+                    "data",
+                    "fully_imported",
+                )
+            },
+        ),
+        (
+            "What's New",
+            {
+                "fields": (
+                    "whats_new",
+                    "whats_new_items_display",
+                    "whats_new_approved",
+                    "whats_new_generated_at",
+                ),
+                "description": (
+                    "AI-generated draft summary. Edit `whats_new` (markdown bullets) "
+                    "and re-save to refresh the parsed items shown below, or use the "
+                    "'Regenerate What's New' action. Only bullets matching the "
+                    "`- **Label** — text` pattern are surfaced on the public site."
+                ),
+            },
+        ),
+    )
+    actions = ["approve_whats_new", "regenerate_whats_new"]
+
+    @admin.display(description="Parsed items (rendered on the site)")
+    def whats_new_items_display(self, obj: Version) -> str:
+        items = obj.whats_new_items
+        if not items:
+            return "(no parseable bullets — site will not render a What's New card)"
+        return format_html(
+            "<ul>{}</ul>",
+            format_html_join(
+                "",
+                "<li><strong>{}</strong> — {}</li>",
+                ((item["title"], item["description"]) for item in items),
+            ),
+        )
 
     def get_queryset(self, request: HttpRequest) -> QuerySet:
         # we want all versions here, including not fully_imported
@@ -56,6 +111,19 @@ def import_new_releases(self, request):
         self.message_user(request, msg)
         return HttpResponseRedirect("../")
 
+    @admin.action(description="Approve What's New (publish)")
+    def approve_whats_new(self, request, queryset):
+        updated = queryset.exclude(whats_new="").update(whats_new_approved=True)
+        self.message_user(request, f"Approved What's New for {updated} version(s).")
+
+    @admin.action(description="Regenerate What's New (queue task)")
+    def regenerate_whats_new(self, request, queryset):
+        queued = 0
+        for version in queryset:
+            dispatch_whats_new(version.pk)
+            queued += 1
+        self.message_user(request, f"Queued regeneration for {queued} version(s).")
+
 
 class ResultInline(admin.StackedInline):
     model = models.ReviewResult