mirror of
https://github.com/fluxcd/flux2.git
synced 2026-02-07 16:27:27 +00:00
Refer to authorisation model in RFC-0001
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
This commit is contained in:
Stefan Prodan
parent
a67d19317b
commit
e0bc754ad0
1 changed files with 37 additions and 103 deletions
|
|
@ -1,4 +1,10 @@
|
|||
# RFC-0001 Flux Multi-Tenancy
|
||||
# RFC-0004 Flux Multi-Tenancy
|
||||
|
||||
**Status:** provisional
|
||||
|
||||
**Creation date:** 2021-11-15
|
||||
|
||||
**Last update:** 2021-12-17
|
||||
|
||||
## Summary
|
||||
|
||||
|
|
@ -7,10 +13,10 @@ models for multi-tenancy, and gives reference implementations for those models.
|
|||
|
||||
## Motivation
|
||||
|
||||
To this point, the Flux project has provided [examples of
|
||||
multi-tenancy](https://github.com/fluxcd/flux2-multi-tenancy/tree/v0.1.0), but not explained exactly
|
||||
To this point, the Flux project has provided [examples of multi-tenancy][mt], but not explained exactly
|
||||
how they relate to Flux's authorisation model. This RFC explains two multi-tenancy implementations,
|
||||
their security properties, and how they are implemented within the authorisation model.
|
||||
their security properties, and how they are implemented within the authorisation model
|
||||
as defined in [RFC-0001](https://github.com/fluxcd/flux2/pull/2212).
|
||||
|
||||
### Goals
|
||||
|
||||
|
|
@ -22,7 +28,7 @@ their security properties, and how they are implemented within the authorisation
|
|||
### Non-Goals
|
||||
|
||||
- Give an exhaustive account of multi-tenancy implementations in general.
|
||||
- Provide an [end-to-end workflow](](https://github.com/fluxcd/flux2-multi-tenancy/tree/v0.1.0)) of
|
||||
- Provide an [end-to-end workflow][mt] of
|
||||
how to set up multi-tenancy with Flux.
|
||||
|
||||
## Introduction
|
||||
|
|
@ -54,14 +60,18 @@ to a cluster. These controllers are subject to authorisation on two counts:
|
|||
- when accessing Kubernetes resources that are needed for a
|
||||
particular "apply" operation -- for example, a secret referenced in
|
||||
the field `.spec.valuesFrom` in a `HelmRelease`;
|
||||
- when creating, updating and deleting Kubernetes resources in the process of applying a piece of
|
||||
configuration.
|
||||
- when creating, watching, updating and deleting Kubernetes resources
|
||||
in the process of applying a piece of configuration.
|
||||
|
||||
To give users control over this authorisation, these two controllers will _impersonate_ (assume the
|
||||
identity of) a service account mentioned in the apply specification (e.g., [the field
|
||||
`.spec.serviceAccountName` in a `Kustomization` object][serviceAccountName]) for both accessing
|
||||
resources and applying configuration. This lets a user constrain the operations mentioned above with
|
||||
RBAC.
|
||||
identity of) a service account mentioned in the apply specification (e.g., the field
|
||||
`.spec.serviceAccountName` in a [`Kustomization` object][kcsa]
|
||||
or in a [`HelmRelease` object][hcsa]) for both accessing resources and applying configuration.
|
||||
This lets a user constrain the operations mentioned above with RBAC.
|
||||
|
||||
As stated in [RFC-0003](https://github.com/fluxcd/flux2/pull/2093),
|
||||
the platform admins can configure Flux to enforce service account impersonation
|
||||
by setting a default service account name when `.spec.serviceAccountName` is not specified.
|
||||
|
||||
#### Remote apply
|
||||
|
||||
|
|
@ -72,9 +82,6 @@ the client used to apply the specified set of configuration. The effect of this
|
|||
configuration will be applied as the user given in the kubeconfig; often this is a user with the
|
||||
`cluster-admin` role bound to it, but not necessarily so.
|
||||
|
||||
[serviceAccountName]: https://fluxcd.io/docs/components/kustomize/api/#kustomize.toolkit.fluxcd.io/v1beta2.KustomizationSpec
|
||||
[kubeconfig]: https://fluxcd.io/docs/components/kustomize/api/#kustomize.toolkit.fluxcd.io/v1beta2.KubeConfig
|
||||
|
||||
## Assumptions made by the multi-tenancy models
|
||||
|
||||
### User Roles
|
||||
|
|
@ -146,9 +153,9 @@ To restrict the reconciliation of tenant's sources, a Kubernetes service account
|
|||
in Flux `Kustomizations` and `HelmReleases` under `.spec.serviceAccountName`. Please consult the Flux
|
||||
documentation for more details:
|
||||
|
||||
- [Kustomization API: Role-based access control](https://fluxcd.io/docs/components/kustomize/kustomization/#role-based-access-control)
|
||||
- [HelmRelease API: Role-based access control](https://fluxcd.io/docs/components/helm/helmreleases/#role-based-access-control)
|
||||
- [Flux multi-tenancy example repository](https://github.com/fluxcd/flux2-multi-tenancy)
|
||||
- [Kustomization API: Role-based access control][kcsa]
|
||||
- [HelmRelease API: Role-based access control][hcsa]
|
||||
- [Flux multi-tenancy example repository][mt]
|
||||
|
||||
Note that with soft multi-tenancy, true tenant isolation requires security measures beyond Kubernetes RBAC.
|
||||
Please refer to the Kubernetes [security considerations documentation](https://kubernetes.io/blog/2021/04/15/three-tenancy-models-for-kubernetes/#security-considerations)
|
||||
|
|
@ -167,100 +174,22 @@ The Flux CLI offers an easy way of generating all the Kubernetes manifests neede
|
|||
- `flux create source git` command generates the configuration that tells Flux which repositories belong to tenants.
|
||||
- `flux create kustomization` command generates the configuration that tells Flux how to reconcile the manifests found in the tenants repositories.
|
||||
|
||||
All the above commands have an `--export` flag for generating the Kubernetes resources in YAML format.
|
||||
The platform admins should place the generated manifests in the repository that defines the cluster(s) desired state.
|
||||
|
||||
Here is an example of the generated manifests:
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: tenant1
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: flux
|
||||
namespace: tenant1
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: flux
|
||||
namespace: tenant1
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: cluster-admin
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: flux
|
||||
namespace: tenant1
|
||||
---
|
||||
apiVersion: source.toolkit.fluxcd.io/v1beta1
|
||||
kind: GitRepository
|
||||
metadata:
|
||||
name: tenant1
|
||||
namespace: tenant1
|
||||
spec:
|
||||
interval: 5m0s
|
||||
ref:
|
||||
branch: main
|
||||
secretRef:
|
||||
name: tenant1-git-auth
|
||||
url: ssh://git@github.com/org/tenant1
|
||||
---
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: tenant1
|
||||
namespace: tenant1
|
||||
spec:
|
||||
interval: 10m0s
|
||||
path: ./
|
||||
prune: true
|
||||
serviceAccountName: flux
|
||||
sourceRef:
|
||||
kind: GitRepository
|
||||
name: tenant1
|
||||
```
|
||||
|
||||
Note that the [cluster-admin](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles)
|
||||
role is used in a `RoleBinding`; this only gives full control over every resource in the role binding's namespace.
|
||||
|
||||
Once the tenants main repositories are registered on the cluster(s), the tenants can configure their app delivery
|
||||
in Git using Kubernetes namespace-scoped resources such as `Deployments`, `Services`, Flagger `Canaries`,
|
||||
Flux `GitRepositories`, `Kustomizations`, `HelmRepositories`, `HelmReleases`, `ImageUpdateAutomations`,
|
||||
`Alerts`, `Receivers`, etc.
|
||||
|
||||
#### Caveats
|
||||
|
||||
As of v0.23.0, Flux does not enforce a service account to be specified on Flux `Kustomizations` and `HelmReleases`.
|
||||
When a service account is not specified, Flux defaults to cluster-admin.
|
||||
In order to enforce the tenant isolation, an admission controller such as Kyverno or OPA Gatekeeper must be used
|
||||
to make the `.spec.serviceAccountName` a required field for the Flux custom resources created by tenants.
|
||||
|
||||
We provide an [example](https://github.com/fluxcd/flux2-multi-tenancy/blob/main/infrastructure/kyverno-policies/flux-multi-tenancy.yaml)
|
||||
for enforcing service accounts using a Kyverno cluster policy.
|
||||
|
||||
As of v0.23.0, Flux allows for `Kustomizations` and `HelmReleases` to reference sources
|
||||
(`GitRepositories`, `HelmRepositories` and `Buckets`) across namespaces.
|
||||
In order to prevent tenants from accessing each other sources, an admission controller such as Kyverno or OPA Gatekeeper
|
||||
must be used to block cross-namespace references.
|
||||
|
||||
We provide an [example](https://github.com/fluxcd/flux2-multi-tenancy/blob/main/infrastructure/kyverno-policies/flux-multi-tenancy.yaml)
|
||||
for blocking source cross-namespace references using a Kyverno cluster policy.
|
||||
|
||||
### Hard Multi-Tenancy
|
||||
|
||||
With hard multi-tenancy, the platform admins use Kubernetes Cluster API to create dedicated clusters for each tenant.
|
||||
The Flux instance installed on the management cluster is responsible
|
||||
for reconciling the cluster definitions belonging to tenants.
|
||||
With hard multi-tenancy, the platform admins create dedicated clusters for each tenant.
|
||||
|
||||
When the tenants's clusters are created with Kubernetes Cluster API, the Flux instance
|
||||
installed on the management cluster is responsible for reconciling the cluster
|
||||
definitions belonging to tenants.
|
||||
|
||||
To enable GitOps for the tenant's clusters, the platform admins can configure the Flux instance running on the
|
||||
management cluster to connect to the tenant's cluster using the `kubeConfig` generated by the Cluster API provider.
|
||||
management cluster to connect to the tenant's cluster using the kubeconfig generated by the Cluster API provider
|
||||
or by creating kubeconfig secrets for the clusters created by other means than Cluster API.
|
||||
|
||||
To configure Flux reconciliation of remote clusters, a Kubernetes secret containing a `kubeConfig` can be specified
|
||||
in Flux `Kustomizations` and `HelmReleases` under `.spec.kubeConfig.secretRef`. Please consult the Flux API
|
||||
|
|
@ -278,7 +207,7 @@ When using a Kubernetes Cluster API provider, the `kubeConfig` secret is automat
|
|||
make use of it without any manual actions. For clusters created by other means than Cluster API, the
|
||||
platform team has to create the `kubeConfig` secrets to allow Flux access to the remote clusters.
|
||||
|
||||
As of Flux v0.23.0, we don't provide any guidance for cluster admins on how to generate the `kubeConfig` secrets.
|
||||
As of Flux v0.24 (Nov 2021), we don't provide any guidance for cluster admins on how to generate the `kubeConfig` secrets.
|
||||
|
||||
## Implementation History
|
||||
|
||||
|
|
@ -289,3 +218,8 @@ As of Flux v0.23.0, we don't provide any guidance for cluster admins on how to g
|
|||
[fluxcd/flux2-multi-tenancy](https://github.com/fluxcd/flux2-multi-tenancy).
|
||||
- Soft multi-tenancy [CVE-2021-41254](https://github.com/fluxcd/kustomize-controller/security/advisories/GHSA-35rf-v2jv-gfg7)
|
||||
"Privilege escalation to cluster admin on multi-tenant environments" was fixed in flux2 **v0.15.0**.
|
||||
|
||||
[mt]: https://github.com/fluxcd/flux2-multi-tenancy/tree/v0.1.0
|
||||
[kcsa]: https://fluxcd.io/docs/components/kustomize/kustomization/#role-based-access-control
|
||||
[hcsa]: https://fluxcd.io/docs/components/helm/helmreleases/#role-based-access-control
|
||||
[kubeconfig]: https://fluxcd.io/docs/components/kustomize/api/#kustomize.toolkit.fluxcd.io/v1beta2.KubeConfig
|
||||
Loading…
Reference in a new issue