Reworked RBAC machinery for APIServiceExports#451
Conversation
| // to be able to figure out what to bind. | ||
| { | ||
| APIGroups: []string{kubebindv1alpha2.GroupName}, | ||
| Resources: []string{"boundschemas"}, |
There was a problem hiding this comment.
^ These rules are already part of the kube-binder ClusterRole:
Probably no need to re-create them?
| return nil, err | ||
| } | ||
|
|
||
| err = kuberesources.EnsureBinderRoleBinding(ctx, c, ns, sa.Name) |
There was a problem hiding this comment.
Previously, kube-binder RoleBinding was created here:
IMHO it makes sense to move it to here to the /bind handler, since this is the place where kube-binder ClusterRole as well as the kube-bind-<Cluster> namespace is created.
|
|
||
| errs = append(errs, | ||
| r.ensureAggregatingClusteRoleAndClusterRoleBinding(ctx, client, cache, kubebinderSubject), | ||
| // We always use cluster-scoped RBACs for exported resources. |
There was a problem hiding this comment.
RBACs for exported resources always use cluster-scoped permissions. This is how it was before too.
We could in theory use namespaced Roles & RoleBindings for resources too when Isolation==Namespaced && Scope==Namespaced.
There was a problem hiding this comment.
does this means that if I have prefix isolation, and claim secrets, I get access to all secrets in the cluster?
There was a problem hiding this comment.
@mjudeikis that decision (whether konnector has cluster- or namespace-scoped access for claimed resources) is based on:
| Resource claim scope | Informer scope | What happens |
|---|---|---|
| cluster | cluster | Creates ClusterRole & ClusterRoleBinding |
| namespace | cluster | Creates ClusterRole & ClusterRoleBinding |
| --- | --- | --- |
| cluster | namespace | Creates ClusterRole & ClusterRoleBinding |
| namespace | namespace | Creates Role & RoleBinding |
So konnector gets namespaced access to Secrets only if the informer scope itself is namespaced.
And this is the behavior we already have btw:
If c.scope == kubebindv1alpha2.ClusterScope:
... otherwise (namespaced):
There was a problem hiding this comment.
My comment above ( #451 (comment) ) was just a suggestion that we could add (yet another :/) mode of operation -- this time for exported resources (APIServiceExport.Spec.Resources), where also backend isolation would be considered.
Right now exported resources always get cluster-scoped access, unconditionally.
There was a problem hiding this comment.
Can we add the table to the docs somewhere?
ntnn
left a comment
There was a problem hiding this comment.
Otherwise lgtm, but leaving final approve to @mjudeikis
| err = kuberesources.EnsureBinderRoleBinding(ctx, c, ns, sa.Name) | ||
| if err != nil { | ||
| return nil, err | ||
| } |
There was a problem hiding this comment.
| err = kuberesources.EnsureBinderRoleBinding(ctx, c, ns, sa.Name) | |
| if err != nil { | |
| return nil, err | |
| } | |
| if err := kuberesources.EnsureBinderRoleBinding(ctx, c, ns, sa.Name); err != nil { | |
| return nil, err | |
| } |
| func (r *APIServiceExportRBACReconciler) SetupWithManager(mgr mcmanager.Manager) error { | ||
| return mcbuilder.ControllerManagedBy(mgr). | ||
| For(&kubebindv1alpha2.APIServiceExport{}). | ||
| Watches( |
There was a problem hiding this comment.
Actually I've ended up removing Controller: true in the owner reference ( c4472ee ).
I'm not sure Owns() would do much: the owner is the Namespace object, so we wouldn't get notified :/
There was a problem hiding this comment.
So that means we don't get notifications on RBACs at all. This is a bit meh, on the other hand we didn't have that before either.
We could add it if it's necessary -- maybe in a follow-up PR? If a care-free admin messes up the RBACs they could trigger a reconcile by nudging the APIServiceExport or APIServiceNamespace so it's not that critical...
|
|
||
| errs = append(errs, | ||
| r.ensureAggregatingClusteRoleAndClusterRoleBinding(ctx, client, cache, kubebinderSubject), | ||
| // We always use cluster-scoped RBACs for exported resources. |
There was a problem hiding this comment.
does this means that if I have prefix isolation, and claim secrets, I get access to all secrets in the cluster?
| } | ||
|
|
||
| func compareMap[V any](t *testing.T, kind string, a, b map[string]V) { | ||
| require.Equal(t, slices.Sorted(maps.Keys(a)), slices.Sorted(maps.Keys(b)), "got unexpected entries after %s reconciliation", kind) |
There was a problem hiding this comment.
Should call t.Helper().
2b5d05c to
b426cb5
Compare
| } | ||
| } | ||
|
|
||
| if err := c.cleanupRBACResources(ctx, client, cache, sns, apiServiceExports); err != nil { |
There was a problem hiding this comment.
are we dropping cleanup of RBAC resources completely?
There was a problem hiding this comment.
All created RBAC-related objects have the owner set to kube-binder-<Consumer's cluster name>. So when that namespace is gone, the GC will do the clean up.
There was a problem hiding this comment.
Do you think it's not enough? It's not perfect, I agree: there is one caveat in that if a consumer un-binds an export, they'd retain their RBACs until the ClusterBinding goes away (and with that the whole namespace, which cleans up the RBACs).
The question is if this actually does have any impact, and I'd hazard to say no, because the consumer/konnector could gain these permissions again by simply re-binding the service export.
| } | ||
| if clusterRole == nil { | ||
| err = r.createClusterRole(ctx, client, &expectedClusterRole) | ||
| if err != nil && apierrors.IsAlreadyExists(err) { |
There was a problem hiding this comment.
| if err != nil && apierrors.IsAlreadyExists(err) { | |
| if err != nil && !apierrors.IsAlreadyExists(err) { |
| Rules: rules, | ||
| } | ||
|
|
||
| role, err := r.getRole(ctx, cache, namespace, name) |
There was a problem hiding this comment.
| role, err := r.getRole(ctx, cache, namespace, name) | |
| role, err := r.getRole(ctx, cache, namespace, name) | |
| if err != nil { | |
| if !apierrors.IsNotFound(err) { | |
| return err | |
| } | |
| if err := r.createRole(ctx, client, &expectedRole); err != nil && !apierrors.IsAlreadyExists(err) { | |
| return err | |
| } | |
| role = &expectedRole | |
| } |
let's maybe simplify it to something like
There was a problem hiding this comment.
I've moved this whole mess into getOrCreate function: 5d08fa1
Added new APIServiceExportRBAC controller for managing APIServiceExport RBACs. Additionally: * kube-binder ClusterRoleBinding creation/update was moved to /bind handler * Redundant boundschemas and boundschemas/status rules in `kube-binder-<ClusterBinding.Namespace>` ClusterRole were removed, because they are already present in `kube-binder` ClusterRole. On-behalf-of: @SAP robert.vasek@sap.com Signed-off-by: Robert Vasek <robert.vasek@clyso.com>
b426cb5 to
27f9533
Compare
* Reworked RBAC machinery for APIServiceExports Added new APIServiceExportRBAC controller for managing APIServiceExport RBACs. Additionally: * kube-binder ClusterRoleBinding creation/update was moved to /bind handler * Redundant boundschemas and boundschemas/status rules in `kube-binder-<ClusterBinding.Namespace>` ClusterRole were removed, because they are already present in `kube-binder` ClusterRole. * e2e: adjusted tests for the new APIServiceExport RBAC machinery * cleanup manager * serviceexportrbac reconciler: don't set Controller=True in owner refs * serviceexportrbac reconciler unit test: add missing t.Helper() * code cleaning * added docs into developers/architecture.md On-behalf-of: @SAP robert.vasek@sap.com Signed-off-by: Robert Vasek <robert.vasek@clyso.com>
Summary
Added new APIServiceExportRBAC controller for managing APIServiceExport RBACs.
<APIServiceExport.Namespace>/kube-binderServiceAccount.kube-binder-exportsaggregates all ClusterRoles of all service exports.kube-binder-resources-<APIServiceExport.Namespace>-<APIServiceExport.Name>ClusterRole for its exported resources,kube-binder-claims-<APIServiceExport.Namespace>-<APIServiceExport.Name>ClusterRole for its claimed resources.kube-binder-claims-<APIServiceExport.Name>is created in each respective<APIServiceNamespace.Status.Namespace>namespace.Additionally:
kube-binderClusterRoleBinding creation/update was moved to /bind handlerkube-binder-<ClusterBinding.Namespace>ClusterRole were removed, because they are already present inkube-binderClusterRole.What Type of PR Is This?
/kind feature
Related Issue(s)
Fixes #344
Release Notes