feat: Add support for topics to Search Contexts (#1028)

* feat: Add support for `topics` to Search Contexts

This commit updates Sourcebot to include support for using `topics` as
part of the Search Context definition.

As part of this:
* Updated `repoMetadataSchema` to store `topics` for `github` and
  `gitlab` host types
* Populate the topic list when compiling GitHub and Gitlab repos
* Updated schemas to support `includeTopics/excludeTopics`
* Expanded test coverage
* Updated Docs

* Replace some references to `GitLab` with `repository`

* One more replace

* Add CHANGELOG.md entry

---------

Co-authored-by: Gavin Williams <gavin.williams@getchip.uk>

Author: fatmcgav

CHANGELOG.md

@@ -18,6 +18,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Added `list_tree` tool to the ask agent. [#1014](https://github.com/sourcebot-dev/sourcebot/pull/1014)
 - Added input & output token breakdown in ask details card. [#1014](https://github.com/sourcebot-dev/sourcebot/pull/1014)
 - Added `path` parameter to the `/api/commits` api to allow filtering commits by paths. [#1014](https://github.com/sourcebot-dev/sourcebot/pull/1014)
+- Search contexts now support topic-based filtering with new `includeTopics` and `excludeTopics` fields, enabling repository filtering by GitHub/GitLab topics with glob pattern support and case-insensitive matching.[#1028](https://github.com/sourcebot-dev/sourcebot/pull/1028)
 
 ### Fixed
 - Fixed issue where ask responses would sometimes appear in the details panel while generating. [#1014](https://github.com/sourcebot-dev/sourcebot/pull/1014)

docs/docs/features/search/search-contexts.mdx

@@ -106,6 +106,38 @@ Like other prefixes, contexts can be negated using `-` or combined using `or`:
 
 See [this doc](/docs/features/search/syntax-reference) for more details on the search query syntax.
 
+## Filtering by topic
+
+If your repositories are tagged with topics on [GitHub](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/classifying-your-repository-with-topics) or [GitLab](https://docs.gitlab.com/ee/user/project/topics.html), you can use `includeTopics` and `excludeTopics` to filter repositories by topic instead of (or in addition to) specifying individual repo URLs. Glob patterns are supported.
+
+```json
+{
+    "$schema": "https://raw.githubusercontent.com/sourcebot-dev/sourcebot/main/schemas/v3/index.json",
+    "contexts": {
+        "backend": {
+            "includeTopics": ["backend", "core-*"],
+            "excludeTopics": ["deprecated", "archived-*"],
+            "description": "Active backend services."
+        }
+    },
+    "connections": {
+        /* ...connection definitions... */
+    }
+}
+```
+
+`includeTopics` and `excludeTopics` follow the same additive semantics as `include` and `exclude`:
+
+- `includeTopics` adds all repos tagged with a matching topic to the context.
+- `excludeTopics` removes repos tagged with a matching topic from the context.
+- Both can be freely combined with `include`, `includeConnections`, `exclude`, and `excludeConnections`.
+
+Topic matching is case-insensitive, so `"Backend"` matches the pattern `"backend"`.
+
+<Note>
+Topics are populated when a connection syncs. If you add topics to your repositories after the last sync, trigger a re-sync for the new topics to take effect.
+</Note>
+
 ## Schema reference
 
 <Accordion title="Reference">

docs/snippets/schemas/v3/index.schema.mdx

@@ -151,6 +151,32 @@
             "type": "string"
           }
         },
+        "includeTopics": {
+          "type": "array",
+          "description": "List of repository topics to include in the search context. Only repositories matching at least one topic are included. Glob patterns are supported.",
+          "items": {
+            "type": "string"
+          },
+          "examples": [
+            [
+              "backend",
+              "core-*"
+            ]
+          ]
+        },
+        "excludeTopics": {
+          "type": "array",
+          "description": "List of repository topics to exclude from the search context. Repositories matching any of these topics are excluded. Glob patterns are supported.",
+          "items": {
+            "type": "string"
+          },
+          "examples": [
+            [
+              "deprecated",
+              "archived-*"
+            ]
+          ]
+        },
         "description": {
           "type": "string",
           "description": "Optional description of the search context that surfaces in the UI."
@@ -313,6 +339,32 @@
                 "type": "string"
               }
             },
+            "includeTopics": {
+              "type": "array",
+              "description": "List of repository topics to include in the search context. Only repositories matching at least one topic are included. Glob patterns are supported.",
+              "items": {
+                "type": "string"
+              },
+              "examples": [
+                [
+                  "backend",
+                  "core-*"
+                ]
+              ]
+            },
+            "excludeTopics": {
+              "type": "array",
+              "description": "List of repository topics to exclude from the search context. Repositories matching any of these topics are excluded. Glob patterns are supported.",
+              "items": {
+                "type": "string"
+              },
+              "examples": [
+                [
+                  "deprecated",
+                  "archived-*"
+                ]
+              ]
+            },
             "description": {
               "type": "string",
               "description": "Optional description of the search context that surfaces in the UI."

docs/snippets/schemas/v3/searchContext.schema.mdx

@@ -46,6 +46,32 @@
         "type": "string"
       }
     },
+    "includeTopics": {
+      "type": "array",
+      "description": "List of repository topics to include in the search context. Only repositories matching at least one topic are included. Glob patterns are supported.",
+      "items": {
+        "type": "string"
+      },
+      "examples": [
+        [
+          "backend",
+          "core-*"
+        ]
+      ]
+    },
+    "excludeTopics": {
+      "type": "array",
+      "description": "List of repository topics to exclude from the search context. Repositories matching any of these topics are excluded. Glob patterns are supported.",
+      "items": {
+        "type": "string"
+      },
+      "examples": [
+        [
+          "deprecated",
+          "archived-*"
+        ]
+      ]
+    },
     "description": {
       "type": "string",
       "description": "Optional description of the search context that surfaces in the UI."