feat: enhance notification system and migrate to pubsub/v2 to fix CI

docs/_sidebar.md

@@ -7,6 +7,7 @@
     - [Debug Console](flagr_debugging.md)
 - Server Configuration
     - [Env](flagr_env.md)
+    - [Notifications](flagr_notifications.md)
 - Client SDKs
     - [Ruby SDK 🔗](https://github.com/openflagr/rbflagr)
     - [Go SDK 🔗](https://github.com/openflagr/goflagr)

docs/flagr_notifications.md

@@ -0,0 +1,76 @@
+# Notifications
+
+Flagr provides an integrated notification system that allows you to monitor changes and updates to your operational resources in real-time. You can configure Flagr to send HTTP `POST` webhooks whenever a flag is created, updated, deleted, or restored.
+
+## Tracked Operations
+
+Flagr monitors changes to **flags** and their related configuration. All notifications have `EntityType: "flag"` in the payload.
+
+The following operations trigger notifications:
+
+| Operation | Description |
+|-----------|-------------|
+| `create` | A new flag is created |
+| `update` | Any change to a flag's metadata, enabled state, or any of its associated entities (segments, variants, constraints, distributions, tags) |
+| `delete` | A flag is soft-deleted |
+| `restore` | A soft-deleted flag is restored |
+
+**Note**: Operations such as adding/removing tags, updating segment rollout percentages, modifying constraints, or changing variant attachments all trigger an `update` notification for the parent flag. Enabling or disabling a flag is also considered an update.
+
+## Configuration
+
+To enable notifications, set the following environment variables:
+
+- `FLAGR_NOTIFICATION_WEBHOOK_ENABLED=true` (Default: `false`) — Enable webhook notifications.
+- `FLAGR_NOTIFICATION_WEBHOOK_URL=https://api.your-org.com/webhooks/flagr` — HTTP destination endpoint for POST requests.
+- `FLAGR_NOTIFICATION_WEBHOOK_HEADERS=Authorization: Bearer secret-token, X-Custom-Header: value` — (Optional) Custom comma-separated HTTP headers, often utilized for securing your webhook receiver with an API token.
+- `FLAGR_NOTIFICATION_TIMEOUT=10s` (Default: `10s`) — Configures the timeout window for dialing the webhook endpoint.
+- `FLAGR_NOTIFICATION_DETAILED_DIFF_ENABLED=true` (Default: `false`) — When enabled, Flagr will embed the precise visual JSON diff of the modified flag within the notification payload.
+- `FLAGR_NOTIFICATION_MAX_RETRIES=3` (Default: `3`) — Maximum number of retry attempts for transient HTTP failures (5xx errors). Set to `0` to disable retries.
+- `FLAGR_NOTIFICATION_RETRY_BASE=1s` (Default: `1s`) — Base delay for exponential backoff between retries.
+- `FLAGR_NOTIFICATION_RETRY_MAX=10s` (Default: `10s`) — Maximum delay between retries.
+
+### Concurrency & Observability
+
+- Notifications are sent asynchronously with a default concurrency limit of 100 to prevent resource exhaustion under load.
+- Metric `notification.sent` is emitted when statsd is enabled, tagged with `provider`, `operation`, `entity_type`, and `status` (`success`/`failure`).
+
+### Important Notes
+
+- **Asynchronous delivery**: Notifications are sent in background goroutines. Failures are logged but **do not affect the API response**.
+- **Startup validation**: Flagr validates the notification configuration at startup and logs a warning if `FLAGR_NOTIFICATION_WEBHOOK_URL` is not set while webhooks are enabled.
+- **Silent fallback**: If webhooks are enabled but the URL is missing, notifications will be silently dropped. A warning is logged at startup to help diagnose misconfiguration.
+
+## Webhook Payload Format
+
+The target endpoint receives a structured JSON payload:
+
+```json
+{
+  "operation": "update",
+  "flag_id": 123,
+  "flag_key": "my-feature-flag",
+  "component_type": "segment",
+  "component_id": 7,
+  "component_key": "power-users",
+  "pre_value": "...",
+  "post_value": "...",
+  "diff": "--- Previous\n+++ Current\n@@ ...",
+  "user": "admin@example.com",
+  "timestamp": "2026-04-26T18:51:03Z"
+}
+```
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `operation` | string | `create`, `update`, `delete`, or `restore` |
+| `flag_id` | uint | Database ID of the parent flag |
+| `flag_key` | string | Unique key of the parent flag |
+| `component_type` | string | What part of the flag changed: `flag`, `segment`, `variant`, `constraint`, `distribution`, or `tag` |
+| `component_id` | uint | Database ID of the changed component |
+| `component_key` | string | Key/name of the changed component (e.g. variant key, tag value) |
+| `pre_value` | string | Previous flag snapshot JSON (only if `FLAGR_NOTIFICATION_DETAILED_DIFF_ENABLED=true`)
+| `post_value` | string | Current flag snapshot JSON (only if `FLAGR_NOTIFICATION_DETAILED_DIFF_ENABLED=true`)
+| `diff` | string | Unified diff between previous and current (only if `FLAGR_NOTIFICATION_DETAILED_DIFF_ENABLED=true`)
+| `user` | string | Identity of the user who made the change |
+| `timestamp` | string | UTC timestamp of the change in RFC 3339 format |

go.mod

@@ -13,7 +13,7 @@ require (
 	github.com/bsm/ratelimit v2.0.0+incompatible
 	github.com/caarlos0/env v3.5.0+incompatible
 	github.com/certifi/gocertifi v0.0.0-20210507211836-431795d63e8d // indirect
-	github.com/davecgh/go-spew v1.1.1
+	github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc
 	github.com/dchest/uniuri v1.2.0
 	github.com/evalphobia/logrus_sentry v0.8.2
 	github.com/form3tech-oss/jwt-go v3.2.5+incompatible
@@ -47,13 +47,12 @@ require (
 	github.com/zhouzhuojie/conditions v0.2.3
 	github.com/zhouzhuojie/withtimeout v0.0.0-20190405051827-12b39eb2edd5
 	golang.org/x/net v0.48.0
-	google.golang.org/api v0.247.0
+	google.golang.org/api v0.257.0
 	google.golang.org/grpc v1.79.3
 	gopkg.in/DataDog/dd-trace-go.v1 v1.46.0
 )
 
 require (
-	cloud.google.com/go/pubsub v1.49.0
 	github.com/glebarez/sqlite v1.6.0
 	github.com/newrelic/go-agent v2.1.0+incompatible
 	gorm.io/driver/mysql v1.4.5
@@ -62,35 +61,38 @@ require (
 )
 
 require (
-	github.com/aws/aws-sdk-go-v2/config v1.31.20
+	cloud.google.com/go/pubsub/v2 v2.0.0
+	github.com/aws/aws-sdk-go-v2/config v1.32.5
 	github.com/aws/aws-sdk-go-v2/service/kinesis v1.42.3
+	github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2
 )
 
 require (
-	cloud.google.com/go/auth v0.16.4 // indirect
+	cloud.google.com/go/auth v0.17.0 // indirect
 	cloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect
 	cloud.google.com/go/compute/metadata v0.9.0 // indirect
-	cloud.google.com/go/iam v1.5.2 // indirect
+	cloud.google.com/go/iam v1.5.3 // indirect
 	github.com/DataDog/datadog-agent/pkg/obfuscate v0.41.1 // indirect
 	github.com/DataDog/datadog-agent/pkg/remoteconfig/state v0.42.0-rc.5 // indirect
 	github.com/DataDog/datadog-go/v5 v5.2.0 // indirect
 	github.com/DataDog/go-tuf v0.3.0--fix-localmeta-fork // indirect
 	github.com/DataDog/sketches-go v1.4.1 // indirect
 	github.com/Microsoft/go-winio v0.6.0 // indirect
 	github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 // indirect
-	github.com/aws/aws-sdk-go-v2 v1.39.6 // indirect
+	github.com/aws/aws-sdk-go-v2 v1.41.0 // indirect
 	github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.3 // indirect
-	github.com/aws/aws-sdk-go-v2/credentials v1.18.24 // indirect
-	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.13 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.13 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.13 // indirect
+	github.com/aws/aws-sdk-go-v2/credentials v1.19.5 // indirect
+	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.16 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.16 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.16 // indirect
 	github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.3 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.13 // indirect
-	github.com/aws/aws-sdk-go-v2/service/sso v1.30.3 // indirect
-	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.7 // indirect
-	github.com/aws/aws-sdk-go-v2/service/sts v1.40.2 // indirect
-	github.com/aws/smithy-go v1.23.2 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.4 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.16 // indirect
+	github.com/aws/aws-sdk-go-v2/service/signin v1.0.4 // indirect
+	github.com/aws/aws-sdk-go-v2/service/sso v1.30.7 // indirect
+	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.12 // indirect
+	github.com/aws/aws-sdk-go-v2/service/sts v1.41.5 // indirect
+	github.com/aws/smithy-go v1.24.0 // indirect
 	github.com/beorn7/perks v1.0.1 // indirect
 	github.com/cespare/xxhash/v2 v2.3.0 // indirect
 	github.com/dgraph-io/ristretto v0.1.1 // indirect
@@ -122,7 +124,7 @@ require (
 	github.com/google/go-cmp v0.7.0 // indirect
 	github.com/google/s2a-go v0.1.9 // indirect
 	github.com/google/uuid v1.6.0 // indirect
-	github.com/googleapis/enterprise-certificate-proxy v0.3.6 // indirect
+	github.com/googleapis/enterprise-certificate-proxy v0.3.7 // indirect
 	github.com/googleapis/gax-go/v2 v2.15.0 // indirect
 	github.com/hashicorp/errwrap v1.1.0 // indirect
 	github.com/hashicorp/go-multierror v1.1.1 // indirect
@@ -139,26 +141,26 @@ require (
 	github.com/jinzhu/inflection v1.0.0 // indirect
 	github.com/jinzhu/now v1.1.5 // indirect
 	github.com/klauspost/compress v1.18.0 // indirect
-	github.com/mattn/go-isatty v0.0.17 // indirect
+	github.com/mattn/go-isatty v0.0.20 // indirect
 	github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
 	github.com/oklog/ulid v1.3.1 // indirect
 	github.com/pierrec/lz4/v4 v4.1.17 // indirect
 	github.com/pkg/errors v0.9.1 // indirect
-	github.com/pmezard/go-difflib v1.0.0 // indirect
 	github.com/prometheus/common v0.66.1 // indirect
 	github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475 // indirect
 	github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
 	github.com/secure-systems-lab/go-securesystemslib v0.4.0 // indirect
-	go.einride.tech/aip v0.68.1 // indirect
+	github.com/stretchr/objx v0.5.3 // indirect
+	go.einride.tech/aip v0.73.0 // indirect
 	go.mongodb.org/mongo-driver v1.17.4 // indirect
 	go.opencensus.io v0.24.0 // indirect
 	go.opentelemetry.io/auto/sdk v1.2.1 // indirect
-	go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.61.0 // indirect
-	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 // indirect
-	go.opentelemetry.io/otel v1.41.0 // indirect
-	go.opentelemetry.io/otel/metric v1.41.0 // indirect
+	go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.64.0 // indirect
+	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.64.0 // indirect
+	go.opentelemetry.io/otel v1.40.0 // indirect
+	go.opentelemetry.io/otel/metric v1.40.0 // indirect
 	go.opentelemetry.io/otel/sdk v1.40.0 // indirect
-	go.opentelemetry.io/otel/trace v1.41.0 // indirect
+	go.opentelemetry.io/otel/trace v1.40.0 // indirect
 	go.uber.org/atomic v1.10.0 // indirect
 	go.yaml.in/yaml/v2 v2.4.2 // indirect
 	go.yaml.in/yaml/v3 v3.0.4 // indirect
@@ -170,10 +172,10 @@ require (
 	golang.org/x/sync v0.19.0 // indirect
 	golang.org/x/sys v0.40.0 // indirect
 	golang.org/x/text v0.32.0 // indirect
-	golang.org/x/time v0.12.0 // indirect
+	golang.org/x/time v0.14.0 // indirect
 	golang.org/x/tools v0.39.0 // indirect
 	golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
-	google.golang.org/genproto v0.0.0-20250603155806-513f23925822 // indirect
+	google.golang.org/genproto v0.0.0-20251202230838-ff82c1b0f217 // indirect
 	google.golang.org/genproto/googleapis/api v0.0.0-20251202230838-ff82c1b0f217 // indirect
 	google.golang.org/genproto/googleapis/rpc v0.0.0-20251202230838-ff82c1b0f217 // indirect
 	google.golang.org/protobuf v1.36.10 // indirect