Skip to content

Commit d9283aa

Browse files
authored
KNOX-3365: New docs for k8s ServiceAccount validator (#1283)
1 parent 2fdbda0 commit d9283aa

1 file changed

Lines changed: 112 additions & 1 deletion

File tree

knox-site/docs/config_preauth_sso_provider.md

Lines changed: 112 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,7 @@ The following table describes the configuration options for the web app security
5757

5858
Name | Description | Default
5959
---------|-----------|--------
60-
preauth.validation.method | Optional parameter that indicates the types of trust validation to perform on incoming requests. There could be one or more comma-separated validators defined in this property. If there are multiple validators, Apache Knox validates each validator in the same sequence as it is configured. This works similar to short-circuit AND operation i.e. if any validator fails, Knox does not perform further validation and returns overall failure immediately. Possible values are: null, preauth.default.validation, preauth.ip.validation, custom validator (details described in [Custom Validator](dev-guide.html#Validator)). Failure results in a 403 forbidden HTTP status response.| null - which means 'preauth.default.validation' that is no validation will be performed and that we are assuming that the network security and external authentication system is sufficient.
60+
preauth.validation.method | Optional parameter that indicates the types of trust validation to perform on incoming requests. There could be one or more comma-separated validators defined in this property. If there are multiple validators, Apache Knox validates each validator in the same sequence as it is configured. This works similar to short-circuit AND operation i.e. if any validator fails, Knox does not perform further validation and returns overall failure immediately. Possible values are: null, preauth.default.validation, preauth.ip.validation, preauth.k8s.service.account.validation (see [Kubernetes ServiceAccount Validator](#kubernetes-serviceaccount-validator)), custom validator (details described in [Custom Validator](dev-guide.html#Validator)). Failure results in a 403 forbidden HTTP status response.| null - which means 'preauth.default.validation' that is no validation will be performed and that we are assuming that the network security and external authentication system is sufficient.
6161
preauth.ip.addresses | Optional parameter that indicates the list of trusted ip addresses. When preauth.ip.validation is indicated as the validation method this parameter must be provided to indicate the trusted ip address set. Wildcarded IPs may be used to indicate subnet level trust. ie. 127.0.* | null - which means that no validation will be performed.
6262
preauth.custom.header | Required parameter for indicating a custom header to use for extracting the preauthenticated principal. The value extracted from this header is utilized as the PrimaryPrincipal within the established Subject. An incoming request that is missing the configured header will be refused with a 401 unauthorized HTTP status. | SM_USER for SiteMinder usecase
6363
preauth.custom.group.header | Optional parameter for indicating a HTTP header name that contains a comma separated list of groups. These are added to the authenticated Subject as group principals. A missing group header will result in no groups being extracted from the incoming request and a log entry but processing will continue. | null - which means that there will be no group principals extracted from the request and added to the established Subject.
@@ -101,3 +101,114 @@ The following curl command can be used to request a directory listing from HDFS
101101
curl -k -i --header "iv_user: guest" --header "iv_group: admin" -v https://localhost:8443/gateway/sandbox/webhdfs/v1/tmp?op=LISTSTATUS
102102

103103
Omitting the `--header "iv_user: guest"` above will result in a rejected request.
104+
105+
#### Kubernetes ServiceAccount Validator ####
106+
107+
The Kubernetes ServiceAccount Validator (`preauth.k8s.service.account.validation`) is a `PreAuthValidator` implementation shipped in the `gateway-provider-security-k8s` module. It is intended for deployments where Knox runs inside a Kubernetes cluster and accepts pre-authenticated requests from workloads whose identity is asserted via a [SPIFFE](https://spiffe.io/) ID.
108+
109+
The validator enforces a binding between:
110+
111+
1. the calling workload's SPIFFE identity (its Kubernetes namespace and ServiceAccount), and
112+
2. the user the caller is asserting in the `HeaderPreAuth` custom user header.
113+
114+
The asserted user is only accepted if the corresponding ServiceAccount in Kubernetes carries an annotation that explicitly authorizes that ServiceAccount to assert that user. This prevents a compromised or misconfigured workload from spoofing arbitrary identities through the pre-auth header.
115+
116+
##### How it works #####
117+
118+
For each incoming request the validator:
119+
120+
1. Reads the SPIFFE header (default `x-spiffe-id`). Its value must be a Kubernetes-style SPIFFE ID of the form `spiffe://<trust-domain>/ns/<namespace>/sa/<service-account>`. Anything else is rejected.
121+
2. Reads the asserted user header (default `x-knoxidf-obo.username`). A missing or empty value is rejected.
122+
3. Looks up the `ServiceAccount` named `<service-account>` in namespace `<namespace>` via the Kubernetes API and reads the annotation configured by `preauth.k8s.sa.user.annotation` (default `knox.apache.org/owner-username`).
123+
4. Accepts the request only if the annotation is present and its value equals the asserted user. Otherwise the request is rejected and `HeaderPreAuth` returns 403.
124+
125+
ServiceAccount annotation lookups are cached in-memory (Caffeine, write-expiring). Successful lookups (including "annotation missing") are cached; transient errors talking to the Kubernetes API are not cached so they will be retried on the next request.
126+
127+
##### Prerequisites #####
128+
129+
- Knox must run inside a Kubernetes pod (or otherwise have a kubeconfig discoverable by the fabric8 `KubernetesClient`).
130+
- The ServiceAccount Knox runs as must have permission to `get` `serviceaccounts` in any namespace whose workloads it must authorize. Grant this with a `ClusterRole`/`ClusterRoleBinding` (or namespace-scoped `Role`/`RoleBinding`).
131+
- A trust path must inject the SPIFFE header into incoming requests. The network/mesh configuration MUST prevent clients from setting this header themselves; otherwise this validator can be bypassed.
132+
- Each ServiceAccount permitted to assert a user MUST be annotated with the configured annotation key, e.g.:
133+
134+
apiVersion: v1
135+
kind: ServiceAccount
136+
metadata:
137+
name: my-workload-sa
138+
namespace: analytics
139+
annotations:
140+
knox.apache.org/owner-username: alice
141+
142+
##### Configuration parameters #####
143+
144+
All parameters are set as `<param>` entries on the `HeaderPreAuth` provider.
145+
146+
Name | Description | Default
147+
---------|-----------|--------
148+
preauth.validation.method | Must include `preauth.k8s.service.account.validation` to enable this validator. May be combined with other validators (comma-separated, short-circuit AND). | n/a
149+
preauth.k8s.sa.spiffe.header | HTTP header carrying the caller's SPIFFE ID. Must be set/overwritten by a trusted hop — never by clients directly. | `x-spiffe-id`
150+
preauth.k8s.sa.custom.header | HTTP header carrying the asserted user that the caller is requesting to act as. This is the value validated against the ServiceAccount annotation. | `x-knoxidf-obo.username`
151+
preauth.k8s.sa.user.annotation | Annotation key on the `ServiceAccount` whose value, when equal to the asserted user, authorizes the assertion. | `knox.apache.org/owner-username`
152+
preauth.k8s.sa.cache.ttl.seconds | Write-expiry TTL (in seconds) for cached ServiceAccount annotation lookups. Must be `> 0`. Lower values pick up annotation changes faster at the cost of more API calls. | `60`
153+
preauth.k8s.sa.cache.max.size | Maximum number of distinct `(namespace, service-account)` entries kept in the lookup cache. Must be `> 0`. | `1000`
154+
155+
Note: the user header default (`x-knoxidf-obo.username`) is independent of the `HeaderPreAuth` `preauth.custom.header` parameter — `preauth.custom.header` controls which header `HeaderPreAuth` extracts as the primary principal, while `preauth.k8s.sa.custom.header` controls which header this validator checks against the ServiceAccount annotation. Configure both to the same header name in deployments where the same value plays both roles.
156+
157+
##### Example topology #####
158+
159+
<provider>
160+
<role>federation</role>
161+
<name>HeaderPreAuth</name>
162+
<enabled>true</enabled>
163+
<param>
164+
<name>preauth.validation.method</name>
165+
<value>preauth.k8s.service.account.validation</value>
166+
</param>
167+
<param>
168+
<name>preauth.custom.header</name>
169+
<value>x-knoxidf-obo.username</value>
170+
</param>
171+
<param>
172+
<name>preauth.k8s.sa.spiffe.header</name>
173+
<value>x-spiffe-id</value>
174+
</param>
175+
<param>
176+
<name>preauth.k8s.sa.custom.header</name>
177+
<value>x-knoxidf-obo.username</value>
178+
</param>
179+
<param>
180+
<name>preauth.k8s.sa.user.annotation</name>
181+
<value>knox.apache.org/owner-username</value>
182+
</param>
183+
<param>
184+
<name>preauth.k8s.sa.cache.ttl.seconds</name>
185+
<value>60</value>
186+
</param>
187+
<param>
188+
<name>preauth.k8s.sa.cache.max.size</name>
189+
<value>1000</value>
190+
</param>
191+
</provider>
192+
193+
##### Example request #####
194+
195+
Assuming the namespace `analytics` contains a ServiceAccount `my-workload-sa` annotated with `knox.apache.org/owner-username: alice`, and the service mesh injects `x-spiffe-id` for that workload:
196+
197+
curl -k -i \
198+
--header "x-spiffe-id: spiffe://example.org/ns/analytics/sa/my-workload-sa" \
199+
--header "x-knoxidf-obo.username: alice" \
200+
-v https://knox.example.org:8443/gateway/sandbox/webhdfs/v1/tmp?op=LISTSTATUS
201+
202+
The request is accepted and processed as user `alice`. Sending the same request with `x-knoxidf-obo.username: bob` (or with a SPIFFE ID pointing to a ServiceAccount that is not annotated for `alice`) yields a 403 response.
203+
204+
##### Failure modes #####
205+
206+
The validator rejects the request (causing `HeaderPreAuth` to return 403) and emits a WARN log entry when:
207+
208+
- the SPIFFE header is missing or empty;
209+
- the user header is missing or empty;
210+
- the SPIFFE header value is not a parseable Kubernetes SPIFFE ID (`spiffe://.../ns/<ns>/sa/<sa>`);
211+
- the target `ServiceAccount` does not exist or carries no annotation under the configured key;
212+
- the annotation value does not match the asserted user.
213+
214+
Errors talking to the Kubernetes API are logged at ERROR and result in rejection but are not cached, so transient API outages do not produce sticky 403s once the API recovers.

0 commit comments

Comments
 (0)