-
Notifications
You must be signed in to change notification settings - Fork 0
mto argocd extension first commit #365
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,247 @@ | ||
| # MTO ArgoCD Extension | ||
|
|
||
| ## ArgoCD multi-tenancy | ||
|
|
||
| > **ArgoCD multi-tenancy is implemented exclusively through `AppProject`.** | ||
|
|
||
| There are **no other tenancy primitives** in ArgoCD. | ||
|
|
||
| An `AppProject` defines: | ||
|
|
||
| * which namespaces a tenant can deploy to | ||
| * which clusters are allowed | ||
| * which Git repositories are allowed | ||
| * which users/groups can access and sync | ||
| * whether cluster-scoped resources are allowed | ||
|
|
||
| If a tenant has its **own AppProject**, it is isolated. | ||
| If not, it is not. | ||
|
|
||
| Everything else (applications, bootstrap, repo creds) is **supporting plumbing**, not tenancy. | ||
|
|
||
| ## What MTO ArgoCD Extension is responsible for? | ||
|
|
||
| MTO’s ArgoCD Extension **only required responsibility** for ArgoCD is: | ||
|
|
||
| > **For each Tenant, create and maintain exactly one AppProject.** | ||
|
|
||
| That’s it. | ||
|
|
||
| Optional responsibilities (UX / automation): | ||
|
|
||
| * creating an initial Application (bootstrap) | ||
| * validating repo credentials exist | ||
| * cleaning up resources on tenant deletion | ||
|
|
||
| ## Current implementation (what exists today) (this section to be removed later) | ||
|
|
||
| ### A) IntegrationConfig (deprecated direction) | ||
|
|
||
| ArgoCD configuration exists inside a [central IntegrationConfig](https://docs.stakater.com/mto/latest/kubernetes-resources/integration-config.html#argocd), living in the tenant-operator repo. | ||
|
|
||
| Problems: | ||
|
|
||
| * tight coupling between tenant core and integrations | ||
| * hard to test in isolation | ||
| * hard to release independently | ||
| * unclear ownership boundaries | ||
|
|
||
| ### B) Extensions CR (per-tenant) – current state | ||
|
|
||
| [Extensions CR](https://docs.stakater.com/mto/latest/integrations/argocd.html) per tenant | ||
|
|
||
| ```yaml | ||
| apiVersion: tenantoperator.stakater.com/v1alpha1 | ||
| kind: Extensions | ||
| spec: | ||
| tenantName: tenant-sample | ||
| argoCD: | ||
| onDeletePurgeAppProject: true | ||
| appProject: | ||
| sourceRepos: [...] | ||
| clusterResourceWhitelist: [...] | ||
| namespaceResourceBlacklist: [...] | ||
| ``` | ||
|
|
||
| Problems: | ||
|
|
||
| * one CR **per tenant per integration** | ||
| * CRD explosion | ||
| * heavy reconciliation | ||
| * allowlist/denylist leaks ArgoCD internals | ||
| * difficult upgrade/migration story | ||
|
|
||
| ## New ArgoCDExtension` CR (v1alpha1) Proposal | ||
|
|
||
| ```yaml | ||
| # ArgoCDExtension | ||
| # Purpose: | ||
| # - Enable ArgoCD multi-tenancy for MTO tenants by creating ONE AppProject per tenant. | ||
| # - AppProject is the ONLY ArgoCD multi-tenancy primitive (security boundary). | ||
| # - Optional: create one "bootstrap" Application per tenant to auto-connect Git repo (UX only). | ||
| # | ||
| # Key design decisions: | ||
| # - No per-tenant CRs (no CR explosion). | ||
| # - Namespace destinations are derived from Namespace labels (clean + stable). | ||
| apiVersion: extensions.mto.stakater.com/v1alpha1 | ||
| kind: ArgoCDExtension | ||
| metadata: | ||
| # Convention: one instance per ArgoCD server/instance. | ||
| # Keep this in a platform namespace (e.g., mto-system). | ||
| name: cluster-default | ||
| namespace: mto-system | ||
| spec: | ||
| # Global on/off for this extension instance. | ||
| enabled: true | ||
|
|
||
| # Reference to the existing/shared ArgoCD instance this extension manages. | ||
| # The controller will create AppProjects (and optional bootstrap Applications) that ArgoCD consumes. | ||
| server: | ||
| name: argocd | ||
| namespace: openshift-operators | ||
| # Optional: if you run multiple ArgoCD instances in the same namespace. | ||
| # instanceName: argocd | ||
|
|
||
| # Which tenants should be managed by this ArgoCD extension? | ||
| # Recommended: opt-in via Tenant label (simple, GitOps-friendly). | ||
| tenantSelector: | ||
| matchLabels: | ||
| # Tenant CR must carry this label to be managed by ArgoCD extension. | ||
| mto.stakater.com/argocd: "enabled" | ||
|
|
||
| # Lifecycle behavior for resources CREATED by this extension. | ||
| # - Retain: leave AppProjects/Bootstrap apps in cluster when tenant opts out or is deleted. | ||
| # - Delete: clean them up (typical if MTO is the owner-of-truth). | ||
| deletionPolicy: Retain # Retain | Delete | ||
|
|
||
| tenancy: | ||
| # How we decide where tenant apps are allowed to deploy. | ||
| # We DO NOT rely on namespace name prefixes/postfixes, and we DO NOT require Tenant.status. | ||
| # Instead we select namespaces by labels (Kubernetes-native + stable). | ||
| destinations: | ||
| strategy: NamespaceLabelSelector | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is also redundant. There's only 1 strategy. If needed, we can expand and add more keys instead of a 2 layered approach for filtering
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. we can read namespaces from tenant.status as well but maybe that is not a good strategy at all |
||
|
|
||
| # Include all namespaces that belong to the tenant (template resolved per tenant). | ||
| # Requirement: tenant namespaces must be labeled like: | ||
| # mto.stakater.io/tenant: <tenant-name> | ||
| namespaceSelectorTemplate: | ||
| matchLabels: | ||
| mto.stakater.io/tenant: "{{ .tenant.name }}" | ||
|
|
||
| # Exclude system namespaces (platform-owned) even if they have the tenant label. | ||
| # Requirement: system namespaces must be labeled like: | ||
| # mto.stakater.io/purpose: system | ||
| excludeSelector: | ||
| matchLabels: | ||
| mto.stakater.io/purpose: "system" | ||
|
|
||
| guardrails: | ||
| # High-signal safety knob: | ||
| # - false: tenants cannot deploy cluster-scoped resources via ArgoCD (recommended default) | ||
| # - true: tenants MAY deploy cluster-scoped resources (use with extreme care) | ||
| allowClusterScopedResources: false | ||
|
|
||
| # How tenant users/groups get permissions in ArgoCD. | ||
| # The extension should map tenant membership to AppProject roles/policies. | ||
| tenantRoleMapping: | ||
|
rasheedamir marked this conversation as resolved.
|
||
| owner: admin | ||
| editor: write | ||
| viewer: read | ||
|
|
||
| # Repo credentials strategy (plumbing, not multi-tenancy): | ||
| # This controls how ArgoCD gets access to private repos for each tenant. | ||
| # Recommended: per-tenant repo creds secret. | ||
| repoCredentials: | ||
| strategy: PerTenantSecret # PerTenantSecret | SharedSecret | None | ||
|
rasheedamir marked this conversation as resolved.
|
||
|
|
||
| perTenantSecret: | ||
| # Where the secret lives: | ||
| # - TenantSystemNamespace: a per-tenant "system" namespace (if you have it) | ||
| # - ArgoCDNamespace: place per-tenant repo secrets in ArgoCD namespace for simpler RBAC | ||
| namespaceStrategy: ArgoCDNamespace # TenantSystemNamespace | ArgoCDNamespace | ||
|
|
||
| # Secret name convention (same for all tenants). | ||
| # The secret content is provided by platform process/ESO/OpenBao/Vault/etc. | ||
| # MTO can validate presence and report status if missing. | ||
| secretName: argocd-repo-creds | ||
|
|
||
| sharedSecret: | ||
| name: global-repo-creds | ||
| namespace: argocd | ||
|
|
||
| # Bootstrap (optional UX): | ||
| # Creates ONE root Application per tenant so onboarding is "zero-click". | ||
|
rasheedamir marked this conversation as resolved.
|
||
| # This is NOT required for multi-tenancy. Multi-tenancy is AppProject only. | ||
| bootstrap: | ||
| enabled: true | ||
|
|
||
| # Repo URL/path/revision can be overridden per tenant via annotations (small scalar overrides only). | ||
| # Example Tenant annotations: | ||
| # argocd.ext.mto.stakater.com/bootstrap-repo-url: https://github.com/acme/gitops | ||
| # argocd.ext.mto.stakater.com/bootstrap-path: tenants/acme | ||
| # argocd.ext.mto.stakater.com/bootstrap-revision: main | ||
| repoURL: | ||
| defaultTemplate: "https://github.com/{{ .tenant.name }}/gitops" | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This can be set to just
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. how do you mean exactly? |
||
| overrideAnnotation: "argocd.ext.mto.stakater.com/bootstrap-repo-url" | ||
|
|
||
| path: | ||
| defaultTemplate: "tenants/{{ .tenant.name }}" | ||
| overrideAnnotation: "argocd.ext.mto.stakater.com/bootstrap-path" | ||
|
|
||
| revision: | ||
| default: "main" | ||
| overrideAnnotation: "argocd.ext.mto.stakater.com/bootstrap-revision" | ||
|
|
||
| # Behavior if repoURL/path cannot be resolved (missing annotations + no usable defaults): | ||
| # - SkipBootstrap: do not create bootstrap Application; still create AppProject (tenancy is intact) | ||
| # - DegradedTenant: mark tenant as degraded in extension status | ||
| onMissingRepo: SkipBootstrap # SkipBootstrap | DegradedTenant | ||
|
|
||
| status: | ||
| conditions: | ||
| - type: Ready | ||
| status: "True" | ||
| reason: ReconcileSuccess | ||
| message: "All tenants reconciled successfully" | ||
| lastTransitionTime: "1111-11-11T11:11:11Z" | ||
| observedGeneration: 3 | ||
| - type: ServerConnected | ||
| status: "True" | ||
| reason: Success | ||
| message: "Connected to ArgoCD server" | ||
| lastTransitionTime: "1111-11-11T11:11:11Z" | ||
| - type: SSOConfigured | ||
| status: "True" | ||
| reason: Success | ||
| message: "SSO configuration applied" | ||
| lastTransitionTime: "1111-11-11T11:11:11Z" | ||
| - type: TenantsReconciled | ||
| status: "True" | ||
| reason: Success | ||
| message: "All tenant AppProjects are in sync" | ||
| lastTransitionTime: "1111-11-11T11:11:11Z" | ||
| tenantSummary: | ||
| total: 12 | ||
| reconciled: 11 | ||
| failed: 1 | ||
| lastReconcileTime: "1111-11-11T11:11:11Z" | ||
| failedTenants: | ||
| - name: tenant-foo | ||
| reason: "AppProject creation failed: quota exceeded" | ||
| lastAttempt: "1111-11-11T11:11:11Z" | ||
| observedGeneration: 3 | ||
| ``` | ||
|
|
||
| ## Supported per-tenant overrides (v1alpha1) | ||
|
|
||
| Only these annotations are supported: | ||
|
|
||
| ```yaml | ||
| argocd.ext.mto.stakater.com/bootstrap-repo-url | ||
| argocd.ext.mto.stakater.com/bootstrap-path | ||
| argocd.ext.mto.stakater.com/bootstrap-revision | ||
| ``` | ||
|
|
||
| Nothing else. | ||
|
|
||
| Security, isolation, and topology are **platform-owned**. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Its better to delete / comment out the Extension itself rather than disabling it through the controller. Unless we just want to disable the ingress or something instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was confused about this as well but GPT kept suggesting that I should have it! but its good you don't see its necessary like me
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so, the verdict is we remove it?