Rel 1.6.3 (#179)

* add project filter

* add bucket filter

* prep for 1.6.3

Author: xiangshen-dk

CHANGELOG.md

@@ -1,4 +1,9 @@
 # Changelog
+## 1.6.3 (2026-04-05)
+* Add Project List Filter to restrict which projects appear in dropdowns using regex patterns
+* Add Log Bucket Filter with include/exclude support — prefix patterns with `!` to exclude matching buckets (e.g., `!.*/_Default` to hide Default buckets)
+* Fix race condition in variable query where default project could be unresolved before bucket/view queries run
+
 ## 1.6.2 (2026-03-18)
 * Fix project dropdown search failing with "contains global restriction" error
 * Fix `useEffect` dependency causing excessive API calls for log buckets and views

README.md

@@ -49,19 +49,65 @@ Once Grafana is configured with Google authentication for signing in, ensure tha
 
 You can then configure the data source with the `OAuth Passthrough` authentication method. Ensure that you provide a default project ID otherwise the health-check will fail.
 
+### Universe Domain
+
+If you are using a Google Cloud environment that uses a custom universe domain (e.g., a sovereign or isolated cloud), you can configure the **Universe Domain** in the data source settings. This tells the plugin to use a different API endpoint instead of the default `googleapis.com`.
+
+Leave this field empty to use the default `googleapis.com` domain.
+
 ### Grafana Configuration
 
 1. With Grafana restarted, navigate to `Configuration -> Data sources` (or the route `/datasources`)
 2. Click "Add data source"
 3. Select "Google Cloud Logging"
 4. Provide credentials from your JWT file, either by uploading it using the file selector or by pasting its contents directly into the designated field
-5. Click "Save & test" to test that logs can be queried from Cloud Logging
+5. Optionally, configure the **Universe Domain** if you are using a non-default GCP environment
+6. Optionally, configure the **Project List Filter** to restrict which projects appear in the project dropdown (see [Project List Filter](#project-list-filter) below)
+7. Optionally, configure the **Log Bucket Filter** to include or exclude specific log buckets from the bucket dropdown (see [Log Bucket Filter](#log-bucket-filter) below)
+8. Click "Save & test" to test that logs can be queried from Cloud Logging
 
 ![image info](https://github.com/GoogleCloudPlatform/cloud-logging-data-source-plugin/blob/main/src/img/cloud_logging_config.png?raw=true)
 
+### Project List Filter
+
+If you have access to many GCP projects, you can restrict which projects appear in the project dropdown by configuring a **Project List Filter** in the data source settings.
+
+Enter project IDs or regex patterns in the text area, one per line. Only projects matching at least one pattern will appear in the dropdown. Leave the field empty to show all projects (the default behavior).
+
+Each line is treated as a [regular expression](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_expressions) anchored to the full project ID. For example:
+
+| Pattern | Matches |
+| --- | --- |
+| `my-project-123` | Only the exact project `my-project-123` |
+| `team-alpha-.*` | All projects starting with `team-alpha-` |
+| `prod-.*-logging` | Projects like `prod-us-logging`, `prod-eu-logging`, etc. |
+
+You can combine multiple patterns (one per line) to match the union of all patterns. If a pattern contains invalid regex syntax, it is treated as a literal string match.
+
+### Log Bucket Filter
+
+You can control which log buckets appear in the bucket dropdown by configuring a **Log Bucket Filter** in the data source settings. This supports both **include** and **exclude** patterns, which is especially useful for excluding default system buckets from initiative projects.
+
+Enter patterns in the text area, one per line:
+- **Include patterns** (no prefix): Only buckets matching at least one include pattern are shown.
+- **Exclude patterns** (prefixed with `!`): Buckets matching any exclude pattern are removed.
+
+When both include and exclude patterns are present, include patterns are applied first, then exclude patterns remove from the result. Leave the field empty to show all buckets (the default behavior).
+
+Patterns are [regular expressions](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_expressions) anchored to the **full bucket path** as returned by the API (e.g., `global/buckets/_Default` or `locations/us-central1/buckets/my-bucket`).
+
+| Pattern | Effect |
+| --- | --- |
+| `!.*/_Default` | Exclude all `_Default` buckets across all locations |
+| `!.*/_Default`<br>`!.*/_Required` | Exclude both `_Default` and `_Required` buckets |
+| `.*my-app-logs` | Include only buckets ending with `my-app-logs` |
+| `global/buckets/.*`<br>`!.*/_Default` | Include only `global/buckets/*` buckets, but exclude `_Default` |
+
+If a pattern contains invalid regex syntax, it is treated as a literal string match.
+
 ### An alternative way to provision the data source
 
-After the plugin is installed, you can define and configure the data source in YAML files as part of Grafana’s provisioning system, similar to [the Google Cloud Monitoring plugin](https://grafana.com/docs/grafana/latest/datasources/google-cloud-monitoring/#provision-the-data-source). For more information about provisioning, and for available configuration options, refer to [Provisioning Grafana](https://grafana.com/docs/grafana/latest/administration/provisioning/#data-sources).
+After the plugin is installed, you can define and configure the data source in YAML files as part of Grafana's provisioning system, similar to [the Google Cloud Monitoring plugin](https://grafana.com/docs/grafana/latest/datasources/google-cloud-monitoring/#provision-the-data-source). For more information about provisioning, and for available configuration options, refer to [Provisioning Grafana](https://grafana.com/docs/grafana/latest/administration/provisioning/#data-sources).
 
 The following YAML is an example.
 
@@ -74,6 +120,15 @@ datasources:
     access: proxy
     jsonData:
       authenticationType: gce
+      # Optional: restrict the project dropdown to matching projects (regex supported)
+      # projectListFilter: |
+      #   my-project-123
+      #   team-alpha-.*
+      # Optional: filter log buckets (prefix with ! to exclude)
+      # logBucketFilter: |
+      #   !.*/_Default
+      # Optional: custom universe domain for sovereign cloud environments
+      # universeDomain: googleapis.com
 ```
 
 ### Supported variables

package.json

@@ -1,6 +1,6 @@
 {
   "name": "googlecloud-logging-datasource",
-  "version": "1.6.2",
+  "version": "1.6.3",
   "description": "Backend Grafana plugin that enables visualization of GCP Cloud Logging logs in Grafana.",
   "scripts": {
     "build": "webpack -c ./.config/webpack/webpack.config.ts --env production",
@@ -23,9 +23,9 @@
     "@grafana/tsconfig": "^2.0.1",
     "@playwright/test": "^1.57.0",
     "@stylistic/eslint-plugin-ts": "^4.4.0",
-    "@swc/core": "^1.15.0",
+    "@swc/core": "^1.15.24",
     "@swc/helpers": "^0.5.0",
-    "@swc/jest": "^0.2.0",
+    "@swc/jest": "^0.2.39",
     "@testing-library/jest-dom": "^6.6.0",
     "@testing-library/react": "^16.3.0",
     "@types/jest": "^29.5.0",

src/CloudLoggingVariableFindQuery.ts

@@ -25,7 +25,7 @@ export default class CloudLoggingVariableFindQuery {
     async execute(query: CloudLoggingVariableQuery) {
         try {
             if (!query.projectId) {
-                this.datasource.getDefaultProject().then(r => query.projectId = r);
+                query.projectId = await this.datasource.getDefaultProject();
             }
             switch (query.selectedQueryType) {
                 case LogFindQueryScopes.Projects:
@@ -44,7 +44,7 @@ export default class CloudLoggingVariableFindQuery {
     }
 
     async handleProjectsQuery() {
-        const projects = await this.datasource.getProjects();
+        const projects = await this.datasource.getFilteredProjects();
         return (projects).map((s) => ({
             text: s,
             value: s,
@@ -58,7 +58,7 @@ export default class CloudLoggingVariableFindQuery {
         if (projectId.startsWith('$')) {
             p = getTemplateSrv().replace(projectId)
         }
-        buckets = await this.datasource.getLogBuckets(p);
+        buckets = await this.datasource.getFilteredBuckets(p);
         return (buckets).map((s) => ({
             text: s,
             value: s,

src/ConfigEditor.tsx

@@ -16,7 +16,7 @@
 
 import { DataSourcePluginOptionsEditorProps, SelectableValue } from '@grafana/data';
 import { ConnectionConfig, GoogleAuthType } from '@grafana/google-sdk';
-import { Checkbox, Field, Input, SecretInput, Select } from '@grafana/ui';
+import { Checkbox, Field, Input, SecretInput, Select, TextArea } from '@grafana/ui';
 import React, { PureComponent } from 'react';
 import { authTypes, CloudLoggingOptions, DataSourceSecureJsonData } from './types';
 
@@ -206,6 +206,48 @@ const defaultProject = (props: Props) => {
           onPointerLeaveCapture={undefined}
         />
       </Field>
+      <Field
+        label="Project List Filter"
+        description="Enter project IDs or regex patterns, one per line. Only matching projects will appear in the project dropdown. Leave empty to show all projects."
+      >
+        <TextArea
+          value={options.jsonData.projectListFilter || ''}
+          placeholder={'my-project-id\nteam-.*\nprod-.*-logging'}
+          rows={4}
+          onChange={(e: React.ChangeEvent<HTMLTextAreaElement>) => {
+            onOptionsChange({
+              ...options,
+              jsonData: {
+                ...options.jsonData,
+                projectListFilter: e.target.value,
+              },
+            });
+          }}
+          onPointerEnterCapture={undefined}
+          onPointerLeaveCapture={undefined}
+        />
+      </Field>
+      <Field
+        label="Log Bucket Filter"
+        description="Filter log buckets in dropdowns. Enter patterns one per line. Prefix a line with ! to exclude matching buckets. Without !, only matching buckets are included. Patterns are regex matched against the full bucket path. Leave empty to show all buckets."
+      >
+        <TextArea
+          value={options.jsonData.logBucketFilter || ''}
+          placeholder={'!.*/_Default\nmy-custom-bucket-.*'}
+          rows={4}
+          onChange={(e: React.ChangeEvent<HTMLTextAreaElement>) => {
+            onOptionsChange({
+              ...options,
+              jsonData: {
+                ...options.jsonData,
+                logBucketFilter: e.target.value,
+              },
+            });
+          }}
+          onPointerEnterCapture={undefined}
+          onPointerLeaveCapture={undefined}
+        />
+      </Field>
     </>
   );
 };

src/QueryEditor.tsx

@@ -34,9 +34,21 @@ export function LoggingQueryEditor({ datasource, query, range, onChange, onRunQu
     }
   };
 
-  // Apply defaults if needed
+  // Apply defaults if needed, and validate against project list filter
   if (!query.projectId) {
-    datasource.getDefaultProject().then(r => query.projectId = r);
+    datasource.getDefaultProject().then(r => {
+      if (datasource.filterProjects([r]).length > 0) {
+        query.projectId = r;
+      }
+    });
+  } else if (!query.projectId.startsWith('$') && datasource.filterProjects([query.projectId]).length === 0) {
+    // Previously selected project no longer passes the filter — clear it
+    query.projectId = '';
+  }
+
+  if (query.bucketId && !query.bucketId.startsWith('$') && datasource.filterBuckets([query.bucketId]).length === 0) {
+    // Previously selected bucket no longer passes the filter — clear it
+    query.bucketId = '';
   }
 
   // Check query field from query params to support default way of propagating query from other parts of grafana.
@@ -71,7 +83,7 @@ export function LoggingQueryEditor({ datasource, query, range, onChange, onRunQu
   };
 
   const loadProjects = useCallback((inputValue: string): Promise<Array<SelectableValue<string>>> => {
-    return datasource.getProjects(inputValue || undefined).then(res => {
+    return datasource.getFilteredProjects(inputValue || undefined).then(res => {
       setFetchError(undefined);
       return res.map(project => ({
         label: project,
@@ -88,15 +100,15 @@ export function LoggingQueryEditor({ datasource, query, range, onChange, onRunQu
     if (!query.projectId) {
       datasource.getDefaultProject().then(r => {
         query.projectId = r;
-        datasource.getLogBuckets(query.projectId).then(res => {
+        datasource.getFilteredBuckets(query.projectId).then(res => {
           setBuckets(res.map(bucket => ({
             label: bucket,
             value: bucket,
           })));
         }).catch(err => setFetchError(sanitizeFetchError(err)));
       });
     } else if (!query.projectId.startsWith('$')) {
-      datasource.getLogBuckets(query.projectId).then(res => {
+      datasource.getFilteredBuckets(query.projectId).then(res => {
         setBuckets(res.map(bucket => ({
           label: bucket,
           value: bucket,

src/VariableQueryEditor.tsx

@@ -49,10 +49,10 @@ export class CloudLoggingVariableQueryEditor extends PureComponent<Props, Variab
     async componentDidMount() {
         await this.props.datasource.ensureGCEDefaultProject();
         const projectId = this.props.query.projectId || (await this.props.datasource.getDefaultProject());
-        const projects = (await this.props.datasource.getProjects());
+        const projects = (await this.props.datasource.getFilteredProjects());
         let buckets: string[] = [];
         if (!projectId.startsWith('$')) {
-            buckets = await this.props.datasource.getLogBuckets(projectId);
+            buckets = await this.props.datasource.getFilteredBuckets(projectId);
         }
 
         const state: any = {
@@ -80,7 +80,7 @@ export class CloudLoggingVariableQueryEditor extends PureComponent<Props, Variab
     async onProjectChange(projectId: string) {
         let buckets: string[] = [];
         if (!projectId.startsWith('$')) {
-            buckets = await this.props.datasource.getLogBuckets(projectId);
+            buckets = await this.props.datasource.getFilteredBuckets(projectId);
         }
         const state: any = {
             buckets,