-
Notifications
You must be signed in to change notification settings - Fork 0
mto argocd extension first commit #365
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,249 @@ | ||
| # MTO ArgoCD Extension | ||
|
|
||
| ## ArgoCD multi-tenancy | ||
|
|
||
| > **ArgoCD multi-tenancy is implemented exclusively through `AppProject`.** | ||
|
|
||
| There are **no other tenancy primitives** in ArgoCD. | ||
|
|
||
| An `AppProject` defines: | ||
|
|
||
| * which namespaces a tenant can deploy to | ||
| * which clusters are allowed | ||
| * which Git repositories are allowed | ||
| * which users/groups can access and sync | ||
| * whether cluster-scoped resources are allowed | ||
|
|
||
| If a tenant has its **own AppProject**, it is isolated. | ||
| If not, it is not. | ||
|
|
||
| Everything else (applications, bootstrap, repo creds) is **supporting plumbing**, not tenancy. | ||
|
|
||
| ## What MTO ArgoCD Extension is responsible for? | ||
|
|
||
| MTO’s ArgoCD Extension **only required responsibility** for ArgoCD is: | ||
|
|
||
| > **For each Tenant, create and maintain exactly one AppProject.** | ||
|
|
||
| That’s it. | ||
|
|
||
| Optional responsibilities (UX / automation): | ||
|
|
||
| * creating an initial Application (bootstrap) | ||
| * validating repo credentials exist | ||
| * cleaning up resources on tenant deletion | ||
|
|
||
| ## Current implementation (what exists today) (this section to be removed later) | ||
|
|
||
| ### A) IntegrationConfig (deprecated direction) | ||
|
|
||
| ArgoCD configuration exists inside a [central IntegrationConfig](https://docs.stakater.com/mto/latest/kubernetes-resources/integration-config.html#argocd), living in the tenant-operator repo. | ||
|
|
||
| Problems: | ||
|
|
||
| * tight coupling between tenant core and integrations | ||
| * hard to test in isolation | ||
| * hard to release independently | ||
| * unclear ownership boundaries | ||
|
|
||
| ### B) Extensions CR (per-tenant) – current state | ||
|
|
||
| [Extensions CR](https://docs.stakater.com/mto/latest/integrations/argocd.html) per tenant | ||
|
|
||
| ```yaml | ||
| apiVersion: tenantoperator.stakater.com/v1alpha1 | ||
| kind: Extensions | ||
| spec: | ||
| tenantName: tenant-sample | ||
| argoCD: | ||
| onDeletePurgeAppProject: true | ||
| appProject: | ||
| sourceRepos: [...] | ||
| clusterResourceWhitelist: [...] | ||
| namespaceResourceBlacklist: [...] | ||
| ``` | ||
|
|
||
| Problems: | ||
|
|
||
| * one CR **per tenant per integration** | ||
| * CRD explosion | ||
| * heavy reconciliation | ||
| * allowlist/denylist leaks ArgoCD internals | ||
| * difficult upgrade/migration story | ||
|
|
||
| ## New ArgoCDExtension` CR (v1alpha1) Proposal | ||
|
|
||
| ```yaml | ||
| # ArgoCDExtension | ||
| # Purpose: | ||
| # - Enable ArgoCD multi-tenancy for MTO tenants by creating ONE AppProject per tenant. | ||
| # - AppProject is the ONLY ArgoCD multi-tenancy primitive (security boundary). | ||
| # - Optional: create one "bootstrap" Application per tenant to auto-connect Git repo (UX only). | ||
| # | ||
| # Key design decisions: | ||
| # - No per-tenant CRs (no CR explosion). | ||
| # - Namespace destinations are derived from Namespace labels (clean + stable). | ||
| apiVersion: extensions.mto.stakater.com/v1alpha1 | ||
| kind: ArgoCDExtension | ||
| metadata: | ||
| # Convention: one instance per ArgoCD server/instance. | ||
| # Keep this in a platform namespace (e.g., mto-system). | ||
| name: cluster-default | ||
| namespace: mto-system | ||
| spec: | ||
| # Global on/off for this extension instance. | ||
| enabled: true | ||
|
|
||
| # Reference to the existing/shared ArgoCD instance this extension manages. | ||
| # The controller will create AppProjects (and optional bootstrap Applications) that ArgoCD consumes. | ||
| server: | ||
| name: argocd | ||
| namespace: openshift-operators | ||
| # Optional: if you run multiple ArgoCD instances in the same namespace. | ||
| # instanceName: argocd | ||
|
|
||
| # Which tenants should be managed by this ArgoCD extension? | ||
| # Recommended: opt-in via Tenant label (simple, GitOps-friendly). | ||
| scaffolding: | ||
|
rasheedamir marked this conversation as resolved.
Outdated
|
||
| mode: TenantSelector | ||
| tenantSelector: | ||
| matchLabels: | ||
| # Tenant CR must carry this label to be managed by ArgoCD extension. | ||
| # Example: | ||
| # metadata.labels["mto.stakater.io/integrations.argocd"]="enabled" | ||
| mto.stakater.io/integrations.argocd: "enabled" | ||
|
|
||
| # Lifecycle behavior for resources CREATED by this extension. | ||
| # - Retain: leave AppProjects/Bootstrap apps in cluster when tenant opts out or is deleted. | ||
| # - Delete: clean them up (typical if MTO is the owner-of-truth). | ||
| deletionPolicy: Retain # Retain | Delete | ||
|
|
||
| tenancy: | ||
| # Core multi-tenancy behavior: | ||
| # Create ONE AppProject per tenant (this is the tenancy boundary). | ||
| projectPerTenant: true | ||
|
rasheedamir marked this conversation as resolved.
Outdated
|
||
|
|
||
| # How we decide where tenant apps are allowed to deploy. | ||
| # We DO NOT rely on namespace name prefixes/postfixes, and we DO NOT require Tenant.status. | ||
| # Instead we select namespaces by labels (Kubernetes-native + stable). | ||
| destinations: | ||
| strategy: NamespaceLabelSelector | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is also redundant. There's only 1 strategy. If needed, we can expand and add more keys instead of a 2 layered approach for filtering
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. we can read namespaces from tenant.status as well but maybe that is not a good strategy at all |
||
|
|
||
| # Include all namespaces that belong to the tenant (template resolved per tenant). | ||
| # Requirement: tenant namespaces must be labeled like: | ||
| # mto.stakater.io/tenant: <tenant-name> | ||
| namespaceSelectorTemplate: | ||
| matchLabels: | ||
| mto.stakater.io/tenant: "{{ .tenant.name }}" | ||
|
|
||
| # Exclude system namespaces (platform-owned) even if they have the tenant label. | ||
| # Requirement: system namespaces must be labeled like: | ||
| # mto.stakater.io/purpose: system | ||
| excludeSelector: | ||
| matchLabels: | ||
| mto.stakater.io/purpose: "system" | ||
|
|
||
| guardrails: | ||
| # High-signal safety knob: | ||
| # - false: tenants cannot deploy cluster-scoped resources via ArgoCD (recommended default) | ||
| # - true: tenants MAY deploy cluster-scoped resources (use with extreme care) | ||
| allowClusterScopedResources: false | ||
|
|
||
| # Advanced escape hatch (optional): | ||
| # If you ever need to tune AppProject policies beyond what the standard knobs cover, | ||
| # you can apply a structured patch. This keeps the CR extensible without exploding fields. | ||
| # | ||
| # IMPORTANT: MTO API uses allowlist/denylist terminology. | ||
| # Controller will translate to ArgoCD internal fields when applying. | ||
| appProjectPolicyPatch: | ||
| enabled: false | ||
| patch: {} | ||
| # Example (only if you ever need it; not required for v1): | ||
| # patch: | ||
| # clusterResourceAllowlist: | ||
| # - group: "" | ||
| # kind: "Pod" | ||
| # namespaceResourceDenylist: | ||
| # - group: "" | ||
| # kind: "ResourceQuota" | ||
|
|
||
|
rasheedamir marked this conversation as resolved.
Outdated
|
||
| # How tenant users/groups get permissions in ArgoCD. | ||
| # The extension should map tenant membership to AppProject roles/policies. | ||
| # (Exact group source depends on how your Tenant CR represents membership.) | ||
| tenantRoleMapping: | ||
|
rasheedamir marked this conversation as resolved.
|
||
| # Source of truth for tenant identities (matches current Tenant CR) | ||
| # Tenant.spec.accessControl: | ||
| # owners/editors/viewers each contain users[] and groups[] | ||
| source: TenantSpecAccessControl | ||
|
|
||
| # Map Tenant roles -> ArgoCD project roles | ||
| roles: | ||
| admin: | ||
| fromTenantSpec: owners # owners.users + owners.groups | ||
| write: | ||
| fromTenantSpec: editors # editors.users + editors.groups | ||
| read: | ||
| fromTenantSpec: viewers # viewers.users + viewers.groups | ||
|
|
||
| # If accessControl is missing/empty, do not grant access by default | ||
| fallback: deny # deny | read | ||
|
|
||
| # Repo credentials strategy (plumbing, not multi-tenancy): | ||
| # This controls how ArgoCD gets access to private repos for each tenant. | ||
| # Recommended: per-tenant repo creds secret. | ||
| repoCredentials: | ||
| strategy: PerTenantSecret # PerTenantSecret | SharedSecret | None | ||
|
rasheedamir marked this conversation as resolved.
|
||
|
|
||
| perTenantSecret: | ||
| # Where the secret lives: | ||
| # - TenantSystemNamespace: a per-tenant "system" namespace (if you have it) | ||
| # - ArgoCDNamespace: place per-tenant repo secrets in ArgoCD namespace for simpler RBAC | ||
| namespaceStrategy: ArgoCDNamespace # TenantSystemNamespace | ArgoCDNamespace | ||
|
|
||
| # Secret name convention (same for all tenants). | ||
| # The secret content is provided by platform process/ESO/OpenBao/Vault/etc. | ||
| # MTO can validate presence and report status if missing. | ||
| secretName: argocd-repo-creds | ||
|
|
||
| # Bootstrap (optional UX): | ||
| # Creates ONE root Application per tenant so onboarding is "zero-click". | ||
|
rasheedamir marked this conversation as resolved.
|
||
| # This is NOT required for multi-tenancy. Multi-tenancy is AppProject only. | ||
| bootstrap: | ||
| enabled: true | ||
|
|
||
| # Repo URL/path/revision can be overridden per tenant via annotations (small scalar overrides only). | ||
| # Example Tenant annotations: | ||
| # argocd.ext.mto.stakater.com/bootstrap-repo-url: https://github.com/acme/gitops | ||
| # argocd.ext.mto.stakater.com/bootstrap-path: tenants/acme | ||
| # argocd.ext.mto.stakater.com/bootstrap-revision: main | ||
| repoURL: | ||
| defaultTemplate: "https://github.com/{{ .tenant.name }}/gitops" | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This can be set to just
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. how do you mean exactly? |
||
| overrideFromAnnotation: "argocd.ext.mto.stakater.com/bootstrap-repo-url" | ||
|
rasheedamir marked this conversation as resolved.
Outdated
|
||
|
|
||
| path: | ||
| defaultTemplate: "tenants/{{ .tenant.name }}" | ||
| overrideFromAnnotation: "argocd.ext.mto.stakater.com/bootstrap-path" | ||
|
|
||
| revision: | ||
| default: "main" | ||
| overrideFromAnnotation: "argocd.ext.mto.stakater.com/bootstrap-revision" | ||
|
|
||
| # Behavior if repoURL/path cannot be resolved (missing annotations + no usable defaults): | ||
| # - SkipBootstrap: do not create bootstrap Application; still create AppProject (tenancy is intact) | ||
| # - DegradedTenant: mark tenant as degraded in extension status | ||
| onMissingRepoOrPath: SkipBootstrap # SkipBootstrap | DegradedTenant | ||
|
rasheedamir marked this conversation as resolved.
Outdated
|
||
| ``` | ||
|
|
||
| ## Supported per-tenant overrides (v1alpha1) | ||
|
|
||
| Only these annotations are supported: | ||
|
|
||
| ```yaml | ||
| argocd.ext.mto.stakater.com/bootstrap-repo-url | ||
| argocd.ext.mto.stakater.com/bootstrap-path | ||
| argocd.ext.mto.stakater.com/bootstrap-revision | ||
| ``` | ||
|
|
||
| Nothing else. | ||
|
|
||
| Security, isolation, and topology are **platform-owned**. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Its better to delete / comment out the Extension itself rather than disabling it through the controller. Unless we just want to disable the ingress or something instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was confused about this as well but GPT kept suggesting that I should have it! but its good you don't see its necessary like me
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so, the verdict is we remove it?