feat: new major 2.0 release

.github/workflows/test_pr.yml

@@ -35,6 +35,7 @@ jobs:
               - 'go.mod'
               - 'go.sum'
               - 'cmd/nelm/**'
+              - 'internal/**'
               - 'pkg/**'
               - 'scripts/**'
             lint:

AGENTS.md

@@ -2,7 +2,7 @@
 
 All rules in this document are requirements — not suggestions. ALWAYS follow them.
 
-Nelm is a Go-based Kubernetes deployment tool, which deploys Helm charts, is compatible with Helm releases, and is an alternative to Helm. Nelm is built on top of a Helm fork ([werf/3p-helm](https://github.com/werf/3p-helm)) and is also used as the deployment engine of [werf](https://github.com/werf/werf).
+Nelm is a Go-based Kubernetes deployment tool, which deploys Helm charts, is compatible with Helm releases, and is an alternative to Helm. Nelm is built on top of a Helm fork and is also used as the deployment engine of [werf](https://github.com/werf/werf).
 
 ## Highest-priority rule (MANDATORY)
 
@@ -75,7 +75,7 @@ ALWAYS use these `task` commands. NEVER use raw `go build`, `go test`, `go fmt`,
 - ALWAYS place tests alongside source files, not in a separate directory.
 - Test helpers go in `helpers_test.go` (or `helpers_ai_test.go` for AI-written helpers).
 - Test fixtures go in `testdata/` subdirectory next to the tests.
-- Shared test helpers are in `internal/test/`.
+- Shared test helpers are in `pkg/test/`.
 
 ## PR review guidelines (MANDATORY)
 
@@ -85,7 +85,6 @@ ALWAYS use these `task` commands. NEVER use raw `go build`, `go test`, `go fmt`,
 
 ## Related repositories
 
-- [werf/3p-helm](https://github.com/werf/3p-helm) — Helm fork. Provides chart loading, rendering, and release primitives. Changes to Helm internals go here, not in nelm.
-- [werf/kubedog](https://github.com/werf/kubedog) — Kubernetes resource tracking library. Used by `internal/track/`.
+- [werf/kubedog](https://github.com/werf/kubedog) — Kubernetes resource tracking library. Used by `pkg/track/`.
 - [werf/common-go](https://github.com/werf/common-go) — Shared Go libraries (secrets, CLI utilities, locking).
 - [werf/werf](https://github.com/werf/werf) — CI/CD tool that uses nelm as its deployment engine.

README.md

@@ -5,7 +5,7 @@
 
 **Nelm** is a Helm 4 alternative. It is a Kubernetes deployment tool that manages Helm Charts and deploys them to Kubernetes. It is also the deployment engine of [werf](https://github.com/werf/werf). Nelm does everything that Helm does, but better, and even quite some on top of it. Nelm is based on an improved and partially rewritten Helm codebase, to introduce:
 
-* `terraform plan`-like capabilities;
+* `terraform plan`-like capabilities with two-stage deployment support;
 * improved CRD management;
 * out-of-the-box secrets management;
 * advanced resource ordering capabilities;
@@ -35,7 +35,7 @@ Nelm is production-ready: as the werf deployment engine, it was battle-tested ac
   - [Advanced resource lifecycle capabilities](#advanced-resource-lifecycle-capabilities)
   - [Resource state tracking](#resource-state-tracking)
   - [Printing logs and events during deploy](#printing-logs-and-events-during-deploy)
-  - [Release planning](#release-planning)
+  - [Release planning and two-stage deployment support](#release-planning-and-two-stage-deployment-workflow-support)
   - [Encrypted values and encrypted files](#encrypted-values-and-encrypted-files)
   - [Improved CRD management](#improved-crd-management)
 - [Usage](#usage)
@@ -244,12 +244,23 @@ Nelm has powerful resource tracking built from the ground up, much more advanced
 
 During the deployment, Nelm finds Pods of deploying resources and periodically prints their container logs. With annotation `werf.io/show-service-messages: "true"`, resource events are also printed. Can be configured with CLI flags and annotations.
 
-### Release planning
+### Release planning and two-stage deployment workflow support
 
-`nelm release plan install` shows exactly what's going to happen in the cluster on the next release. It shows 100% accurate diffs between current and to-be resource versions, utilizing robust dry-run Server-Side Apply instead of client-side trickery.
+`nelm release plan install` shows exactly what's going to happen in the cluster on the next release. It shows diffs between the current and to-be resource versions, utilizing robust dry-run Kubernetes Server-Side Apply capabilities.
 
 ![planning](resources/images/nelm-release-plan-install.png)
 
+To ensure that these exact changes will be applied during the release install, you can utilize a two-stage deployment workflow:
+
+1. **Plan:** Generate, review, and save a plan artifact using the `--save-plan` flag:
+```
+nelm release plan install --save-plan=plan.gz
+```
+2. **Apply:** Perform the release install rapidly using the pre-generated reviewed plan:
+```
+nelm release install --use-plan=plan.gz
+```
+
 ### Encrypted values and encrypted files
 
 `nelm chart secret` commands manage encrypted values files such as `secret-values.yaml` or encrypted arbitrary files like `secret/mysecret.txt`. These files are decrypted in-memory during templating and can be used in templates as `.Values.my.secret.value` and `{{ werf_secret_file "mysecret.txt" }}`, respectively.

Taskfile.dist.yaml

@@ -52,7 +52,7 @@ tasks:
     internal: true
     run: once
     cmds:
-      - golangci-lint run {{if eq .fix "true"}}--fix{{end}} --build-tags="{{.tags}}" {{.CLI_ARGS}} {{.paths | default "$(find pkg/ -mindepth 1 -maxdepth 1 -type d ! -name helm -printf './%p/... ' 2>/dev/null || find pkg/ -mindepth 1 -maxdepth 1 -type d ! -name helm | xargs -I{} echo './{}/...') ./cmd/..."}}
+      - golangci-lint run {{if eq .fix "true"}}--fix{{end}} --build-tags="{{.tags}}" {{.CLI_ARGS}} {{.paths | default "$(find pkg/ -mindepth 1 -maxdepth 1 -type d ! -name helm -printf './%p/... ') ./cmd/..."}}
     vars:
       paths: "{{.paths}}"
 
@@ -237,7 +237,7 @@ tasks:
       - task: test:ginkgo
         vars:
           paths: '{{.paths | default "./pkg ./cmd"}}'
-          skipPackage: "pkg/helm{{if .skipPackage}},{{.skipPackage}}{{end}}"
+          skipPackage: "{{.skipPackage}}"
           parallel: "{{.parallel}}"
 
   verify:binaries:dist:all:
@@ -288,6 +288,7 @@ tasks:
   generate:reference:
     desc: "Generate CLI reference documentation with TOC."
     env:
+      # TODO(major): not supported anymore
       HELM_CONFIG_HOME: "~/.config/helm"
       HELM_CACHE_HOME: "~/.cache/helm"
       HELM_DATA_HOME: "~/.local/share/helm"
@@ -491,8 +492,7 @@ tasks:
         sg scan --rule "$intern_rule" -U "{{.forkSnapshotDir}}"
         rm -f "$intern_rule"
 
-        rm -f "{{.forkSnapshotDir}}/cmd/helm/helm.go"
-        find "{{.forkSnapshotDir}}/cmd/helm" -name "*.go" -exec perl -i -p -e 's/^package main$/package helm/' {} +
+        rm -rf "{{.forkSnapshotDir}}/cmd"
 
         git -C "{{.forkSnapshotDir}}" add -A
         git -C "{{.forkSnapshotDir}}" -c user.name="sync" -c user.email="sync@local" \
@@ -504,13 +504,21 @@ tasks:
         git fetch "{{.forkTempRemote}}" prepared
         git checkout -B "{{.forkSyncBranch}}" FETCH_HEAD --no-track
 
-        stripped=0
-        while IFS= read -r sync_path; do
-          if ! git -c core.quotePath=false ls-tree --name-only "${original_ref}" -- "{{.forkPrefix}}/${sync_path}" | grep -q .; then
-            git rm -rf -- "$sync_path"
-            stripped=$((stripped + 1))
-          fi
-        done < <(git ls-tree --name-only HEAD)
+        prev_split=$(git log --grep="git-subtree-dir: {{.forkPrefix}}" --format="%b" "${original_ref}" | grep "^git-subtree-split:" | head -n1 | cut -d' ' -f2)
+        if [ -z "$prev_split" ]; then
+          echo "WARNING: no previous subtree split found for {{.forkPrefix}}, skipping file stripping"
+        else
+          stripped=0
+          while IFS= read -r sync_path; do
+            if ! git -c core.quotePath=false ls-tree "${original_ref}" -- "{{.forkPrefix}}/${sync_path}" | grep -q .; then
+              if git -c core.quotePath=false ls-tree "$prev_split" -- "$sync_path" | grep -q .; then
+                git rm -rf -- "$sync_path"
+                stripped=$((stripped + 1))
+              fi
+            fi
+          done < <(git -c core.quotePath=false ls-tree -r --name-only HEAD)
+          echo "Stripped $stripped files that were removed from our fork"
+        fi
 
         if ! git diff --cached --quiet; then
           git commit -m "sync: strip paths not kept in {{.forkPrefix}}"

cmd/nelm/chart_dependency_download.go

@@ -8,8 +8,9 @@ import (
 	"github.com/spf13/cobra"
 
 	"github.com/werf/common-go/pkg/cli"
-	helm_v3 "github.com/werf/nelm/pkg/helm/cmd/helm"
+	"github.com/werf/nelm/pkg/action"
 	"github.com/werf/nelm/pkg/helm/pkg/chart/loader"
+	helmcmd "github.com/werf/nelm/pkg/helm/pkg/cmd"
 	"github.com/werf/nelm/pkg/log"
 )
 
@@ -31,9 +32,9 @@ func newChartDependencyDownloadCommand(ctx context.Context, afterAllCommandsBuil
 
 	originalRunE := cmd.RunE
 	cmd.RunE = func(cmd *cobra.Command, args []string) error {
-		helmSettings := helm_v3.Settings
+		helmSettings := helmcmd.Settings
 
-		ctx = log.SetupLogging(ctx, lo.Ternary(helmSettings.Debug, log.DebugLevel, log.InfoLevel), log.SetupLoggingOptions{})
+		ctx = action.SetupLogging(ctx, lo.Ternary(helmSettings.Debug, log.DebugLevel, log.InfoLevel), action.SetupLoggingOptions{})
 
 		loader.NoChartLockWarning = ""
 

cmd/nelm/chart_dependency_update.go

@@ -8,8 +8,9 @@ import (
 	"github.com/spf13/cobra"
 
 	"github.com/werf/common-go/pkg/cli"
-	helm_v3 "github.com/werf/nelm/pkg/helm/cmd/helm"
+	"github.com/werf/nelm/pkg/action"
 	"github.com/werf/nelm/pkg/helm/pkg/chart/loader"
+	helmcmd "github.com/werf/nelm/pkg/helm/pkg/cmd"
 	"github.com/werf/nelm/pkg/log"
 )
 
@@ -29,9 +30,9 @@ func newChartDependencyUpdateCommand(ctx context.Context, afterAllCommandsBuiltF
 
 	originalRunE := cmd.RunE
 	cmd.RunE = func(cmd *cobra.Command, args []string) error {
-		helmSettings := helm_v3.Settings
+		helmSettings := helmcmd.Settings
 
-		ctx = log.SetupLogging(ctx, lo.Ternary(helmSettings.Debug, log.DebugLevel, log.InfoLevel), log.SetupLoggingOptions{})
+		ctx = action.SetupLogging(ctx, lo.Ternary(helmSettings.Debug, log.DebugLevel, log.InfoLevel), action.SetupLoggingOptions{})
 
 		loader.NoChartLockWarning = ""
 

cmd/nelm/chart_download.go

@@ -8,8 +8,9 @@ import (
 	"github.com/spf13/cobra"
 
 	"github.com/werf/common-go/pkg/cli"
-	helm_v3 "github.com/werf/nelm/pkg/helm/cmd/helm"
+	"github.com/werf/nelm/pkg/action"
 	"github.com/werf/nelm/pkg/helm/pkg/chart/loader"
+	helmcmd "github.com/werf/nelm/pkg/helm/pkg/cmd"
 	"github.com/werf/nelm/pkg/log"
 )
 
@@ -26,9 +27,9 @@ func newChartDownloadCommand(ctx context.Context, afterAllCommandsBuiltFuncs map
 
 	originalRunE := cmd.RunE
 	cmd.RunE = func(cmd *cobra.Command, args []string) error {
-		helmSettings := helm_v3.Settings
+		helmSettings := helmcmd.Settings
 
-		ctx = log.SetupLogging(ctx, lo.Ternary(helmSettings.Debug, log.DebugLevel, log.InfoLevel), log.SetupLoggingOptions{})
+		ctx = action.SetupLogging(ctx, lo.Ternary(helmSettings.Debug, log.DebugLevel, log.InfoLevel), action.SetupLoggingOptions{})
 
 		loader.NoChartLockWarning = ""
 

cmd/nelm/chart_lint.go

@@ -10,7 +10,6 @@ import (
 	"github.com/werf/common-go/pkg/cli"
 	"github.com/werf/nelm/pkg/action"
 	"github.com/werf/nelm/pkg/common"
-	"github.com/werf/nelm/pkg/featgate"
 	"github.com/werf/nelm/pkg/log"
 )
 
@@ -25,11 +24,7 @@ func newChartLintCommand(ctx context.Context, afterAllCommandsBuiltFuncs map[*co
 	cfg := &chartLintConfig{}
 
 	use := "lint [options...]"
-	if featgate.FeatGateRemoteCharts.Enabled() || featgate.FeatGatePreviewV2.Enabled() {
-		use += " [chart-dir|chart-repo-name/chart-name|chart-archive|chart-archive-url]"
-	} else {
-		use += " [chart-dir]"
-	}
+	use += " [chart-dir|chart-repo-name/chart-name|chart-archive|chart-archive-url]"
 
 	cmd := cli.NewSubCommand(
 		ctx,
@@ -45,16 +40,10 @@ func newChartLintCommand(ctx context.Context, afterAllCommandsBuiltFuncs map[*co
 			},
 		},
 		func(cmd *cobra.Command, args []string) error {
-			ctx = log.SetupLogging(ctx, cmp.Or(log.Level(cfg.LogLevel), action.DefaultChartLintLogLevel), log.SetupLoggingOptions{
-				ColorMode: cfg.LogColorMode,
-			})
+			ctx = action.SetupLogging(ctx, cmp.Or(log.Level(cfg.LogLevel), action.DefaultChartLintLogLevel), action.SetupLoggingOptions{ColorMode: cfg.LogColorMode})
 
 			if len(args) > 0 {
-				if featgate.FeatGateRemoteCharts.Enabled() || featgate.FeatGatePreviewV2.Enabled() {
-					cfg.Chart = args[0]
-				} else {
-					cfg.ChartDirPath = args[0]
-				}
+				cfg.Chart = args[0]
 			}
 
 			if err := action.ChartLint(ctx, cfg.ChartLintOptions); err != nil {
@@ -116,13 +105,11 @@ func newChartLintCommand(ctx context.Context, afterAllCommandsBuiltFuncs map[*co
 			return fmt.Errorf("add flag: %w", err)
 		}
 
-		if featgate.FeatGateRemoteCharts.Enabled() || featgate.FeatGatePreviewV2.Enabled() {
-			if err := cli.AddFlag(cmd, &cfg.ChartVersion, "chart-version", "", "Choose a remote chart version, otherwise the latest version is used", cli.AddFlagOptions{
-				GetEnvVarRegexesFunc: cli.GetFlagGlobalAndLocalEnvVarRegexes,
-				Group:                mainFlagGroup,
-			}); err != nil {
-				return fmt.Errorf("add flag: %w", err)
-			}
+		if err := cli.AddFlag(cmd, &cfg.ChartVersion, "chart-version", "", "Choose a remote chart version, otherwise the latest version is used", cli.AddFlagOptions{
+			GetEnvVarRegexesFunc: cli.GetFlagGlobalAndLocalEnvVarRegexes,
+			Group:                mainFlagGroup,
+		}); err != nil {
+			return fmt.Errorf("add flag: %w", err)
 		}
 
 		// TODO: restrict allowed values

cmd/nelm/chart_pack.go

@@ -9,15 +9,16 @@ import (
 	"github.com/spf13/cobra"
 
 	"github.com/werf/common-go/pkg/cli"
-	helm_v3 "github.com/werf/nelm/pkg/helm/cmd/helm"
+	"github.com/werf/nelm/pkg/action"
+	"github.com/werf/nelm/pkg/common"
 	"github.com/werf/nelm/pkg/helm/pkg/chart/loader"
-	"github.com/werf/nelm/pkg/helm/pkg/werf/helmopts"
+	helmcmd "github.com/werf/nelm/pkg/helm/pkg/cmd"
 	"github.com/werf/nelm/pkg/log"
 	"github.com/werf/nelm/pkg/ts"
 )
 
 func newChartPackCommand(ctx context.Context, afterAllCommandsBuiltFuncs map[*cobra.Command]func(cmd *cobra.Command) error) *cobra.Command {
-	opts := helmopts.TypeScriptOptions{}
+	opts := common.TypeScriptOptions{}
 	cmd := lo.Must(lo.Find(helmRootCmd.Commands(), func(c *cobra.Command) bool {
 		return strings.HasPrefix(c.Use, "package")
 	}))
@@ -31,9 +32,10 @@ func newChartPackCommand(ctx context.Context, afterAllCommandsBuiltFuncs map[*co
 
 	originalRunE := cmd.RunE
 	cmd.RunE = func(cmd *cobra.Command, args []string) error {
-		helmSettings := helm_v3.Settings
+		helmSettings := helmcmd.Settings
 
-		ctx = log.SetupLogging(ctx, lo.Ternary(helmSettings.Debug, log.DebugLevel, log.InfoLevel), log.SetupLoggingOptions{})
+		// TODO(major): should we do it like that everywhere, setting the context?
+		ctx = action.SetupLogging(cmd.Context(), lo.Ternary(helmSettings.Debug, log.DebugLevel, log.InfoLevel), action.SetupLoggingOptions{})
 		ctx = ts.NewContextWithTSOptions(ctx, opts)
 		cmd.SetContext(ctx)
 

cmd/nelm/chart_render.go

@@ -10,7 +10,6 @@ import (
 	"github.com/werf/common-go/pkg/cli"
 	"github.com/werf/nelm/pkg/action"
 	"github.com/werf/nelm/pkg/common"
-	"github.com/werf/nelm/pkg/featgate"
 	"github.com/werf/nelm/pkg/log"
 )
 
@@ -25,11 +24,7 @@ func newChartRenderCommand(ctx context.Context, afterAllCommandsBuiltFuncs map[*
 	cfg := &chartRenderConfig{}
 
 	use := "render [options...]"
-	if featgate.FeatGateRemoteCharts.Enabled() || featgate.FeatGatePreviewV2.Enabled() {
-		use += " [chart-dir|chart-repo-name/chart-name|chart-archive|chart-archive-url]"
-	} else {
-		use += " [chart-dir]"
-	}
+	use += " [chart-dir|chart-repo-name/chart-name|chart-archive|chart-archive-url]"
 
 	cmd := cli.NewSubCommand(
 		ctx,
@@ -45,17 +40,13 @@ func newChartRenderCommand(ctx context.Context, afterAllCommandsBuiltFuncs map[*
 			},
 		},
 		func(cmd *cobra.Command, args []string) error {
-			ctx = log.SetupLogging(ctx, cmp.Or(log.Level(cfg.LogLevel), action.DefaultChartRenderLogLevel), log.SetupLoggingOptions{
+			ctx = action.SetupLogging(ctx, cmp.Or(log.Level(cfg.LogLevel), action.DefaultChartRenderLogLevel), action.SetupLoggingOptions{
 				ColorMode:      cfg.LogColorMode,
 				LogIsParseable: true,
 			})
 
 			if len(args) > 0 {
-				if featgate.FeatGateRemoteCharts.Enabled() || featgate.FeatGatePreviewV2.Enabled() {
-					cfg.Chart = args[0]
-				} else {
-					cfg.ChartDirPath = args[0]
-				}
+				cfg.Chart = args[0]
 			}
 
 			if _, err := action.ChartRender(ctx, cfg.ChartRenderOptions); err != nil {
@@ -113,13 +104,11 @@ func newChartRenderCommand(ctx context.Context, afterAllCommandsBuiltFuncs map[*
 			return fmt.Errorf("add flag: %w", err)
 		}
 
-		if featgate.FeatGateRemoteCharts.Enabled() || featgate.FeatGatePreviewV2.Enabled() {
-			if err := cli.AddFlag(cmd, &cfg.ChartVersion, "chart-version", "", "Choose a remote chart version, otherwise the latest version is used", cli.AddFlagOptions{
-				GetEnvVarRegexesFunc: cli.GetFlagGlobalAndLocalEnvVarRegexes,
-				Group:                mainFlagGroup,
-			}); err != nil {
-				return fmt.Errorf("add flag: %w", err)
-			}
+		if err := cli.AddFlag(cmd, &cfg.ChartVersion, "chart-version", "", "Choose a remote chart version, otherwise the latest version is used", cli.AddFlagOptions{
+			GetEnvVarRegexesFunc: cli.GetFlagGlobalAndLocalEnvVarRegexes,
+			Group:                mainFlagGroup,
+		}); err != nil {
+			return fmt.Errorf("add flag: %w", err)
 		}
 
 		if err := cli.AddFlag(cmd, &cfg.ExtraAPIVersions, "extra-apiversions", nil, "Extra Kubernetes API versions passed to $.Capabilities.APIVersions", cli.AddFlagOptions{
@@ -150,13 +139,6 @@ func newChartRenderCommand(ctx context.Context, afterAllCommandsBuiltFuncs map[*
 			return fmt.Errorf("add flag: %w", err)
 		}
 
-		if err := cli.AddFlag(cmd, &cfg.ForceAdoption, "force-adoption", false, "Always adopt resources, even if they belong to a different Helm release", cli.AddFlagOptions{
-			GetEnvVarRegexesFunc: cli.GetFlagGlobalAndLocalEnvVarRegexes,
-			Group:                mainFlagGroup,
-		}); err != nil {
-			return fmt.Errorf("add flag: %w", err)
-		}
-
 		if err := cli.AddFlag(cmd, &cfg.LocalKubeVersion, "kube-version", common.DefaultLocalKubeVersion, "Kubernetes version stub for non-remote mode", cli.AddFlagOptions{
 			Group: mainFlagGroup,
 		}); err != nil {

cmd/nelm/chart_secret_file_decrypt.go

@@ -38,7 +38,7 @@ func newChartSecretFileDecryptCommand(ctx context.Context, afterAllCommandsBuilt
 			},
 		},
 		func(cmd *cobra.Command, args []string) error {
-			ctx = log.SetupLogging(ctx, cmp.Or(log.Level(cfg.LogLevel), action.DefaultSecretFileDecryptLogLevel), log.SetupLoggingOptions{
+			ctx = action.SetupLogging(ctx, cmp.Or(log.Level(cfg.LogLevel), action.DefaultSecretFileDecryptLogLevel), action.SetupLoggingOptions{
 				ColorMode:      cfg.LogColorMode,
 				LogIsParseable: true,
 			})

cmd/nelm/chart_secret_file_edit.go

@@ -38,9 +38,7 @@ func newChartSecretFileEditCommand(ctx context.Context, afterAllCommandsBuiltFun
 			},
 		},
 		func(cmd *cobra.Command, args []string) error {
-			ctx = log.SetupLogging(ctx, cmp.Or(log.Level(cfg.LogLevel), action.DefaultSecretFileEditLogLevel), log.SetupLoggingOptions{
-				ColorMode: cfg.LogColorMode,
-			})
+			ctx = action.SetupLogging(ctx, cmp.Or(log.Level(cfg.LogLevel), action.DefaultSecretFileEditLogLevel), action.SetupLoggingOptions{ColorMode: cfg.LogColorMode})
 
 			cfg.File = args[0]