Skip to content
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion api/v1/receiver_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,11 @@ type ReceiverSpec struct {
ResourceFilter string `json:"resourceFilter,omitempty"`

// SecretRef specifies the Secret containing the token used
// to validate the payload authenticity.
// to validate the payload authenticity. The Secret must contain a 'token'
// key. For GCR receivers, the Secret must also contain an 'email' key
// with the IAM service account email configured on the Pub/Sub push
// subscription, and may optionally contain an 'audience' key with the
// expected OIDC token audience.
// +required
SecretRef meta.LocalObjectReference `json:"secretRef"`

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,11 @@ spec:
secretRef:
description: |-
SecretRef specifies the Secret containing the token used
to validate the payload authenticity.
to validate the payload authenticity. The Secret must contain a 'token'
key. For GCR receivers, the Secret must also contain an 'email' key
with the IAM service account email configured on the Pub/Sub push
subscription, and may optionally contain an 'audience' key with the
expected OIDC token audience.
properties:
name:
description: Name of the referent.
Expand Down
12 changes: 10 additions & 2 deletions docs/api/v1/notification.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,11 @@ github.com/fluxcd/pkg/apis/meta.LocalObjectReference
</td>
<td>
<p>SecretRef specifies the Secret containing the token used
to validate the payload authenticity.</p>
to validate the payload authenticity. The Secret must contain a &lsquo;token&rsquo;
key. For GCR receivers, the Secret must also contain an &lsquo;email&rsquo; key
with the IAM service account email configured on the Pub/Sub push
subscription, and may optionally contain an &lsquo;audience&rsquo; key with the
expected OIDC token audience.</p>
</td>
</tr>
<tr>
Expand Down Expand Up @@ -366,7 +370,11 @@ github.com/fluxcd/pkg/apis/meta.LocalObjectReference
</td>
<td>
<p>SecretRef specifies the Secret containing the token used
to validate the payload authenticity.</p>
to validate the payload authenticity. The Secret must contain a &lsquo;token&rsquo;
key. For GCR receivers, the Secret must also contain an &lsquo;email&rsquo; key
with the IAM service account email configured on the Pub/Sub push
subscription, and may optionally contain an &lsquo;audience&rsquo; key with the
expected OIDC token audience.</p>
</td>
</tr>
<tr>
Expand Down
61 changes: 52 additions & 9 deletions docs/spec/v1/receivers.md
Original file line number Diff line number Diff line change
Expand Up @@ -545,15 +545,53 @@ spec:
#### GCR

When a Receiver's `.spec.type` is set to `gcr`, the controller will respond to
an [HTTP webhook event payload](https://cloud.google.com/container-registry/docs/configuring-notifications#notification_examples)
from Google Cloud Registry to the generated [`.status.webhookPath`](#webhook-path),
while verifying the payload is legitimate using [JWT](https://cloud.google.com/pubsub/docs/push#authentication).
an [HTTP webhook event payload](https://cloud.google.com/artifact-registry/docs/configure-notifications)
from Google Container Registry (GCR) or Google Artifact Registry (GAR) to the
generated [`.status.webhookPath`](#webhook-path), while verifying the payload is
legitimate using [OIDC ID token validation](https://cloud.google.com/pubsub/docs/authenticate-push-subscriptions).

The controller authenticates the request by performing the following checks on
the OIDC ID token from the `Authorization` header:

1. **Signature verification**: The token signature is validated against Google's
public keys.
2. **Audience verification**: The `aud` claim is verified against the expected
audience (see below).
3. **Issuer verification**: The `iss` claim must be `accounts.google.com` or
`https://accounts.google.com`.
4. **Email verification**: The `email` claim must match the service account
email specified in the referenced Secret's `email` key, and `email_verified`
must be `true`.

For this to work, [authentication must be enabled on the Pub/Sub push
subscription](https://cloud.google.com/pubsub/docs/push#configure_for_push_authentication),
with the OIDC service account set to the same service account specified in the
Secret's `email` key.

##### Secret format for GCR

The Secret referenced by `.spec.secretRef.name` must contain the following keys:

| Key | Required | Description |
|--------------|----------|----------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `token` | Yes | Random string used to salt the generated [webhook path](#webhook-path). |
| `email` | Yes | The email of the IAM service account configured on the Pub/Sub push subscription for OIDC authentication. |
| `audience` | No | The expected `aud` claim in the OIDC token. If omitted, the controller reconstructs it from the incoming request URL, which matches the Pub/Sub default behavior of using the push endpoint URL as the audience. Set this if you configured a custom audience on the Pub/Sub subscription. |

The controller verifies the request originates from Google by validating the
token from the [`Authorization` header](https://cloud.google.com/pubsub/docs/push#validate_tokens).
For this to work, authentication must be enabled for the Pub/Sub subscription,
refer to the [Google Cloud documentation](https://cloud.google.com/pubsub/docs/push#configure_for_push_authentication)
for more information.
Example:

```yaml
---
apiVersion: v1
kind: Secret
metadata:
name: gcr-webhook-token
namespace: default
type: Opaque
stringData:
token: <random token>
email: <service-account>@<project>.iam.gserviceaccount.com
```

When the verification succeeds, the request payload is unmarshalled to the
expected format. If this is successful, the controller will request a
Expand All @@ -574,7 +612,7 @@ metadata:
spec:
type: gcr
secretRef:
name: webhook-token
name: gcr-webhook-token
resources:
- apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageRepository
Expand Down Expand Up @@ -777,6 +815,11 @@ This token is used to salt the generated [webhook path](#webhook-path), and
depending on the Receiver [type](#supported-receiver-types), to verify the
authenticity of a request.

**Note:** Some receiver types require additional keys in the Secret. For
example, the [GCR](#gcr) type requires an `email` key and optionally an
`audience` key. Refer to the documentation for the specific receiver type for
details.

Example:

```yaml
Expand Down
19 changes: 17 additions & 2 deletions internal/server/receiver_handler_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import (
"encoding/base64"
"encoding/hex"
"encoding/json"
"fmt"
"hash"
"net/http"
"net/http/httptest"
Expand Down Expand Up @@ -1211,7 +1212,7 @@ func Test_handlePayload(t *testing.T) {
Spec: apiv1.ReceiverSpec{
Type: apiv1.GCRReceiver,
SecretRef: meta.LocalObjectReference{
Name: "token",
Name: "gcr-token",
},
Resources: []apiv1.CrossNamespaceObjectReference{
{
Expand All @@ -1227,7 +1228,15 @@ func Test_handlePayload(t *testing.T) {
Conditions: []metav1.Condition{{Type: meta.ReadyCondition, Status: metav1.ConditionTrue}},
},
},
secret: testSecretWithToken,
secret: &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Name: "gcr-token",
},
Data: map[string][]byte{
"token": []byte("token"),
"email": []byte("test@example.iam.gserviceaccount.com"),
},
},
resources: []client.Object{testReceiverResource},
expectedResourcesAnnotated: 1,
expectedResponseCode: http.StatusOK,
Expand Down Expand Up @@ -1401,6 +1410,12 @@ func Test_handlePayload(t *testing.T) {
logger: logger.NewLogger(logger.Options{}),
kubeClient: client,
noCrossNamespaceRefs: tt.noCrossNamespaceRefs,
gcrTokenValidator: func(_ context.Context, bearer string, expectedEmail string, expectedAudience string) error {
if bearer == "" {
return fmt.Errorf("missing authorization header")
}
return nil
},
}

data, err := json.Marshal(tt.payload)
Expand Down
121 changes: 88 additions & 33 deletions internal/server/receiver_handlers.go
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ import (
"github.com/fluxcd/pkg/runtime/conditions"
"github.com/go-logr/logr"
"github.com/google/go-github/v64/github"
"google.golang.org/api/idtoken"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime/schema"
Expand Down Expand Up @@ -223,12 +224,23 @@ func (s *ReceiverServer) validate(ctx context.Context, receiver apiv1.Receiver,
}
r.Body = io.NopCloser(bytes.NewReader(b))

// Fetch the token.
token, err := s.token(ctx, receiver)
// Fetch the secret.
secret, err := s.secret(ctx, receiver)
if err != nil {
return fmt.Errorf("unable to read token, error: %w", err)
return fmt.Errorf("unable to read secret, error: %w", err)
}

// Extract the token from the secret.
secretName := types.NamespacedName{
Namespace: receiver.GetNamespace(),
Name: receiver.Spec.SecretRef.Name,
}
tokenBytes, ok := secret.Data["token"]
if !ok {
return fmt.Errorf("invalid %q secret data: required field 'token'", secretName)
}
token := string(tokenBytes)

logger := s.logger.WithValues(
"reconciler kind", apiv1.ReceiverKind,
"name", receiver.Name,
Expand Down Expand Up @@ -398,8 +410,6 @@ func (s *ReceiverServer) validate(ctx context.Context, receiver apiv1.Receiver,
r.Body = io.NopCloser(bytes.NewReader(b))
return nil
case apiv1.GCRReceiver:
const tokenIndex = len("Bearer ")

type data struct {
Action string `json:"action"`
Digest string `json:"digest"`
Expand All @@ -415,8 +425,34 @@ func (s *ReceiverServer) validate(ctx context.Context, receiver apiv1.Receiver,
} `json:"message"`
}

err := authenticateGCRRequest(&http.Client{}, r.Header.Get("Authorization"), tokenIndex)
if err != nil {
expectedEmail, ok := secret.Data["email"]
_ = ok
// TODO: in Flux 2.9, require the email. this will be a breaking change.
// if !ok {
// return fmt.Errorf("invalid secret data: required field 'email' for GCR receiver")
// }

// Determine the expected audience. If explicitly set in the secret, use
// that. Otherwise, reconstruct the webhook URL from the request, which is
// the default audience used by GCR when it sends the webhook.
audience := string(secret.Data["audience"])
if audience == "" {
scheme := "https"
if r.TLS == nil {
if proto := r.Header.Get("X-Forwarded-Proto"); proto != "" {
scheme = proto
} else {
scheme = "http"
}
}
audience = scheme + "://" + r.Host + r.URL.Path
}

authenticate := authenticateGCRRequest
if s.gcrTokenValidator != nil {
authenticate = s.gcrTokenValidator
}
if err := authenticate(ctx, r.Header.Get("Authorization"), string(expectedEmail), audience); err != nil {
return fmt.Errorf("cannot authenticate GCR request: %w", err)
}

Expand Down Expand Up @@ -498,26 +534,18 @@ func (s *ReceiverServer) validate(ctx context.Context, receiver apiv1.Receiver,
return fmt.Errorf("recevier type %q not supported", receiver.Spec.Type)
}

func (s *ReceiverServer) token(ctx context.Context, receiver apiv1.Receiver) (string, error) {
token := ""
func (s *ReceiverServer) secret(ctx context.Context, receiver apiv1.Receiver) (*corev1.Secret, error) {
secretName := types.NamespacedName{
Namespace: receiver.GetNamespace(),
Name: receiver.Spec.SecretRef.Name,
}

var secret corev1.Secret
err := s.kubeClient.Get(ctx, secretName, &secret)
if err != nil {
return "", fmt.Errorf("unable to read token from secret %q error: %w", secretName, err)
if err := s.kubeClient.Get(ctx, secretName, &secret); err != nil {
return nil, fmt.Errorf("unable to read secret %q: %w", secretName, err)
}

if val, ok := secret.Data["token"]; ok {
token = string(val)
} else {
return "", fmt.Errorf("invalid %q secret data: required field 'token'", secretName)
}

return token, nil
return &secret, nil
}

// requestReconciliation requests reconciliation of all the resources matching the given CrossNamespaceObjectReference by annotating them accordingly.
Expand Down Expand Up @@ -577,26 +605,53 @@ func (s *ReceiverServer) annotate(ctx context.Context, resource *metav1.PartialO
return nil
}

func authenticateGCRRequest(c *http.Client, bearer string, tokenIndex int) (err error) {
type auth struct {
Aud string `json:"aud"`
// authenticateGCRRequest validates the OIDC ID token according to
// https://docs.cloud.google.com/pubsub/docs/authenticate-push-subscriptions#go.
func authenticateGCRRequest(ctx context.Context, bearer string, expectedEmail string, expectedAudience string) error {
const bearerPrefix = "Bearer "
if !strings.HasPrefix(bearer, bearerPrefix) {
return fmt.Errorf("the Authorization header is missing or malformed")
}

if len(bearer) < tokenIndex {
return fmt.Errorf("the Authorization header is missing or malformed: %v", bearer)
}
token := bearer[len(bearerPrefix):]

token := bearer[tokenIndex:]
url := fmt.Sprintf("https://oauth2.googleapis.com/tokeninfo?id_token=%s", token)

resp, err := c.Get(url)
// Validate the OIDC ID token signature and claims using Google's public keys.
v, err := idtoken.NewValidator(ctx)
if err != nil {
return fmt.Errorf("cannot create ID token validator: %w", err)
}
payload, err := v.Validate(ctx, token, expectedAudience)
if err != nil {
return fmt.Errorf("cannot verify authenticity of payload: %w", err)
// Extract the actual audience from the token for logging.
gotAudience := "<unknown>"
if parts := strings.Split(token, "."); len(parts) == 3 {
if claimsJSON, decErr := base64.RawURLEncoding.DecodeString(parts[1]); decErr == nil {
var claims struct {
Aud string `json:"aud"`
}
if json.Unmarshal(claimsJSON, &claims) == nil && claims.Aud != "" {
gotAudience = claims.Aud
}
}
}
return fmt.Errorf("invalid ID token: audience is '%s', want '%s': %w", gotAudience, expectedAudience, err)
}

var p auth
if err := json.NewDecoder(resp.Body).Decode(&p); err != nil {
return fmt.Errorf("cannot decode auth payload: %w", err)
// Verify the token issuer.
issuer, _ := payload.Claims["iss"].(string)
if issuer != "accounts.google.com" && issuer != "https://accounts.google.com" {
return fmt.Errorf("token issuer is '%s', want 'accounts.google.com' or 'https://accounts.google.com'", issuer)
}

// Verify the token was issued for the expected service account.
email, _ := payload.Claims["email"].(string)
emailVerified, _ := payload.Claims["email_verified"].(bool)
// TODO: in Flux 2.9, require the email (remove `expectedEmail != "" &&`). this will be a breaking change.
if expectedEmail != "" && email != expectedEmail {
return fmt.Errorf("token email is '%s', want '%s'", email, expectedEmail)
}
if !emailVerified {
return fmt.Errorf("token email '%s' is not verified", email)
}

return nil
Expand Down
3 changes: 3 additions & 0 deletions internal/server/receiver_server.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,9 @@ type ReceiverServer struct {
kubeClient client.Client
noCrossNamespaceRefs bool
exportHTTPPathMetrics bool
// gcrTokenValidator overrides the default GCR OIDC token validation function.
// Used in tests to avoid calling Google's servers.
gcrTokenValidator func(ctx context.Context, bearer string, expectedEmail string, expectedAudience string) error
}

// NewReceiverServer returns an HTTP server that handles webhooks
Expand Down
Loading