Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions docs.md
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,22 @@ Both `minAvailable` and `maxUnavailable` must be a string which represents a non
^([0-9]|[1-9][0-9]|100)%$
```

##### Feature: Webhook Deployment Customization
Comment thread
rohitsakala marked this conversation as resolved.

The `WebhookDeploymentCustomization` field configures the rancher-webhook deployment on downstream clusters. The following sub-fields are validated:

- `replicaCount`: If set, must be at least 1.
- `appendTolerations`: Toleration keys are validated against the upstream apimachinery label name regex:
```regex
([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9]
```
- `overrideAffinity`: Node affinity `nodeSelectorTerms` are validated via label name validation. Pod affinity and pod anti-affinity are validated via label selectors using the [apimachinery label selector validation](https://github.com/kubernetes/apimachinery/blob/02a41040d88da08de6765573ae2b1a51f424e1ca/pkg/apis/meta/v1/validation/validation.go#L56).
- `podDisruptionBudget.minAvailable` and `podDisruptionBudget.maxUnavailable`: Each must be a non-negative whole integer or a whole number percentage between `0%` and `100%`. Only one of the two fields can have a non-zero or non-empty value at a given time. These fields use the following regex when assessing if a given percentage value is valid:
```regex
^([0-9]|[1-9][0-9]|100)%$
```
- `overrideResourceRequirements`: Not validated by the webhook — the Kubernetes API server validates `ResourceRequirements` natively.

## ClusterProxyConfig

### Validation Checks
Expand Down Expand Up @@ -809,6 +825,22 @@ Both `minAvailable` and `maxUnavailable` must be a string which represents a non
^([0-9]|[1-9][0-9]|100)%$
```

##### cluster.spec.webhookDeploymentCustomization

The `WebhookDeploymentCustomization` field configures the rancher-webhook deployment on downstream clusters. The following sub-fields are validated:

- `replicaCount`: If set, must be at least 1.
- `appendTolerations`: Toleration keys are validated against the same upstream apimachinery label name regex used for agent deployment customizations:
```regex
([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9]
```
- `overrideAffinity`: Node affinity `nodeSelectorTerms` are validated via label name validation. Pod affinity and pod anti-affinity are validated via label selectors using the [apimachinery label selector validation](https://github.com/kubernetes/apimachinery/blob/02a41040d88da08de6765573ae2b1a51f424e1ca/pkg/apis/meta/v1/validation/validation.go#L56).
- `podDisruptionBudget.minAvailable` and `podDisruptionBudget.maxUnavailable`: Each must be a non-negative whole integer or a whole number percentage between `0%` and `100%`. Only one of the two fields can have a non-zero or non-empty value at a given time. These fields use the following regex when assessing if a given percentage value is valid:
```regex
^([0-9]|[1-9][0-9]|100)%$
```
- `overrideResourceRequirements`: Not validated by the webhook — the Kubernetes API server validates `ResourceRequirements` natively.

##### NO_PROXY value

Prevent the update of objects with an env var (under `spec.agentEnvVars`) with a name of `NO_PROXY` if its value contains one or more spaces. This ensures that the provided value adheres to
Expand Down
207 changes: 207 additions & 0 deletions pkg/resources/common/deployment_customization.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,207 @@
package common

import (
"fmt"
"net/http"
"strconv"
"strings"

corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/apis/meta/v1/validation"
"k8s.io/apimachinery/pkg/util/validation/field"
)

// PDB holds PodDisruptionBudget string fields, decoupled from API-group-specific types.
type PDB struct {
MinAvailable string
MaxUnavailable string
}

// ValidateWebhookDeploymentCustomization validates the fields of a webhook deployment
// customization spec using standard k8s types.
func ValidateWebhookDeploymentCustomization(replicaCount *int32, tolerations []corev1.Toleration, affinity *corev1.Affinity, pdb *PDB, path *field.Path) field.ErrorList {
var errList field.ErrorList

if replicaCount != nil && *replicaCount < 1 {
errList = append(errList, field.Invalid(path.Child("replicaCount"), *replicaCount, "must be at least 1"))
}

errList = append(errList, ValidateAppendTolerations(tolerations, path.Child("appendTolerations"))...)
errList = append(errList, ValidateAffinity(affinity, path.Child("overrideAffinity"))...)
errList = append(errList, ValidatePDB(pdb, path.Child("podDisruptionBudget"))...)

return errList
}

// ValidatePDB validates PodDisruptionBudget minAvailable/maxUnavailable values.
func ValidatePDB(pdb *PDB, path *field.Path) field.ErrorList {
if pdb == nil {
return nil
}
var errList field.ErrorList

minAvailStr := pdb.MinAvailable
maxUnavailStr := pdb.MaxUnavailable

if (minAvailStr == "" && maxUnavailStr == "") ||
(minAvailStr == "0" && maxUnavailStr == "0") ||
(minAvailStr != "" && minAvailStr != "0") && (maxUnavailStr != "" && maxUnavailStr != "0") {
errList = append(errList, field.Invalid(path, pdb, "both minAvailable and maxUnavailable cannot be set to a non-zero value, at least one must be omitted or set to zero"))
return errList
}

if minAvailStr != "" {
minAvailInt, err := strconv.Atoi(minAvailStr)
if err != nil {
if !PdbPercentageRegex.MatchString(minAvailStr) {
errList = append(errList, field.Invalid(path.Child("minAvailable"), minAvailStr,
fmt.Sprintf("must be a non-negative whole integer or a percentage value between 0 and 100, regex used is '%s'", PdbPercentageRegex.String())))
}
} else if minAvailInt < 0 {
errList = append(errList, field.Invalid(path.Child("minAvailable"), minAvailStr, "cannot be a negative integer"))
}
}

if maxUnavailStr != "" {
maxUnavailInt, err := strconv.Atoi(maxUnavailStr)
if err != nil {
if !PdbPercentageRegex.MatchString(maxUnavailStr) {
errList = append(errList, field.Invalid(path.Child("maxUnavailable"), maxUnavailStr,
fmt.Sprintf("must be a non-negative whole integer or a percentage value between 0 and 100, regex used is '%s'", PdbPercentageRegex.String())))
}
} else if maxUnavailInt < 0 {
errList = append(errList, field.Invalid(path.Child("maxUnavailable"), maxUnavailStr, "cannot be a negative integer"))
}
}

return errList
}

// ValidateAppendTolerations validates that toleration keys follow k8s label name rules.
func ValidateAppendTolerations(tolerations []corev1.Toleration, path *field.Path) field.ErrorList {
var errList field.ErrorList
for k, s := range tolerations {
errList = append(errList, validation.ValidateLabelName(s.Key, path.Index(k))...)
}
return errList
}

// ValidateAffinity validates an Affinity spec including node, pod, and pod anti-affinity rules.
func ValidateAffinity(overrideAffinity *corev1.Affinity, path *field.Path) field.ErrorList {
if overrideAffinity == nil {
return nil
}
var errList field.ErrorList

if affinity := overrideAffinity.NodeAffinity; affinity != nil {
errList = append(errList,
validatePreferredSchedulingTerms(affinity.PreferredDuringSchedulingIgnoredDuringExecution,
path.Child("nodeAffinity").Child("preferredDuringSchedulingIgnoredDuringExecution"))...,
)
errList = append(errList,
validateNodeSelector(affinity.RequiredDuringSchedulingIgnoredDuringExecution,
path.Child("nodeAffinity").Child("requiredDuringSchedulingIgnoredDuringExecution"))...,
)
}

if podAffinity := overrideAffinity.PodAffinity; podAffinity != nil {
errList = append(errList, validatePodAffinityTerms(podAffinity.RequiredDuringSchedulingIgnoredDuringExecution,
path.Child("podAffinity").Child("requiredDuringSchedulingIgnoredDuringExecution"))...)

errList = append(errList, validateWeightedPodAffinityTerms(podAffinity.PreferredDuringSchedulingIgnoredDuringExecution,
path.Child("podAffinity").Child("preferredDuringSchedulingIgnoredDuringExecution"))...)
}

if podAntiAffinity := overrideAffinity.PodAntiAffinity; podAntiAffinity != nil {
errList = append(errList, validatePodAffinityTerms(podAntiAffinity.RequiredDuringSchedulingIgnoredDuringExecution,
path.Child("podAntiAffinity").Child("requiredDuringSchedulingIgnoredDuringExecution"))...)

errList = append(errList, validateWeightedPodAffinityTerms(podAntiAffinity.PreferredDuringSchedulingIgnoredDuringExecution,
path.Child("podAntiAffinity").Child("preferredDuringSchedulingIgnoredDuringExecution"))...)
}
return errList
}

// ErrorListToStatus converts a field.ErrorList to a metav1.Status with bullet-pointed messages.
func ErrorListToStatus(errList field.ErrorList) *metav1.Status {
if len(errList) == 0 {
return nil
}
var builder strings.Builder
builder.WriteString("* ")
for i, fieldErr := range errList {
builder.WriteString(fieldErr.Error())
if i != len(errList)-1 {
builder.WriteString("\n* ")
}
}
return &metav1.Status{
Status: "Failure",
Message: builder.String(),
Reason: metav1.StatusReasonInvalid,
Code: http.StatusUnprocessableEntity,
}
}

func validatePodAffinityTerms(terms []corev1.PodAffinityTerm, path *field.Path) field.ErrorList {
var errList field.ErrorList
for k, v := range terms {
errList = append(errList, validatePodAffinityTerm(v, path.Index(k))...)
}
return errList
}

func validateWeightedPodAffinityTerms(terms []corev1.WeightedPodAffinityTerm, path *field.Path) field.ErrorList {
var errList field.ErrorList
for k, v := range terms {
errList = append(errList, validatePodAffinityTerm(v.PodAffinityTerm, path.Index(k).Child("podAffinityTerm"))...)
}
return errList
}

func validatePodAffinityTerm(term corev1.PodAffinityTerm, path *field.Path) field.ErrorList {
var errList field.ErrorList
errList = append(errList, validateLabelSelector(term.LabelSelector, path.Child("labelSelector"))...)
errList = append(errList, validateLabelSelector(term.NamespaceSelector, path.Child("namespaceSelector"))...)
return errList
}

func validateLabelSelector(labelSelector *metav1.LabelSelector, path *field.Path) field.ErrorList {
return validation.ValidateLabelSelector(labelSelector, validation.LabelSelectorValidationOptions{}, path)
}

func validatePreferredSchedulingTerms(terms []corev1.PreferredSchedulingTerm, path *field.Path) field.ErrorList {
var errList field.ErrorList
for k, v := range terms {
errList = append(errList, validateNodeSelectorTerm(v.Preference, path.Index(k).Child("preferences"))...)
}
return errList
}

func validateNodeSelector(nodeSelector *corev1.NodeSelector, path *field.Path) field.ErrorList {
if nodeSelector == nil {
return nil
}
var errList field.ErrorList
nodeSelectorPath := path.Child("nodeSelectorTerms")
for k, v := range nodeSelector.NodeSelectorTerms {
errList = append(errList, validateNodeSelectorTerm(v, nodeSelectorPath.Index(k))...)
}
return errList
}

func validateNodeSelectorTerm(term corev1.NodeSelectorTerm, path *field.Path) field.ErrorList {
var errList field.ErrorList
errList = append(errList, validateNodeSelectorRequirements(term.MatchFields, path.Child("matchFields"))...)
errList = append(errList, validateNodeSelectorRequirements(term.MatchExpressions, path.Child("matchExpressions"))...)
return errList
}

func validateNodeSelectorRequirements(selector []corev1.NodeSelectorRequirement, path *field.Path) field.ErrorList {
var errList field.ErrorList
for k, s := range selector {
errList = append(errList, validation.ValidateLabelName(s.Key, path.Index(k).Child("key"))...)
}
return errList
}
16 changes: 16 additions & 0 deletions pkg/resources/management.cattle.io/v3/cluster/Cluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,3 +43,19 @@ Both `minAvailable` and `maxUnavailable` must be a string which represents a non
```regex
^([0-9]|[1-9][0-9]|100)%$
```

#### Feature: Webhook Deployment Customization

The `WebhookDeploymentCustomization` field configures the rancher-webhook deployment on downstream clusters. The following sub-fields are validated:

- `replicaCount`: If set, must be at least 1.
- `appendTolerations`: Toleration keys are validated against the upstream apimachinery label name regex:
```regex
([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9]
```
- `overrideAffinity`: Node affinity `nodeSelectorTerms` are validated via label name validation. Pod affinity and pod anti-affinity are validated via label selectors using the [apimachinery label selector validation](https://github.com/kubernetes/apimachinery/blob/02a41040d88da08de6765573ae2b1a51f424e1ca/pkg/apis/meta/v1/validation/validation.go#L56).
- `podDisruptionBudget.minAvailable` and `podDisruptionBudget.maxUnavailable`: Each must be a non-negative whole integer or a whole number percentage between `0%` and `100%`. Only one of the two fields can have a non-zero or non-empty value at a given time. These fields use the following regex when assessing if a given percentage value is valid:
```regex
^([0-9]|[1-9][0-9]|100)%$
```
- `overrideResourceRequirements`: Not validated by the webhook — the Kubernetes API server validates `ResourceRequirements` natively.
13 changes: 13 additions & 0 deletions pkg/resources/management.cattle.io/v3/cluster/validator.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import (
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime/schema"
"k8s.io/apimachinery/pkg/util/validation/field"
authorizationv1 "k8s.io/client-go/kubernetes/typed/authorization/v1"
)

Expand Down Expand Up @@ -130,6 +131,18 @@ func (a *admitter) Admit(request *admission.Request) (*admissionv1.AdmissionResp
return response, err
}

if wdc := newCluster.Spec.WebhookDeploymentCustomization; wdc != nil {
Comment thread
crobby marked this conversation as resolved.
Outdated
var pdb *common.PDB
if wdc.PodDisruptionBudget != nil {
pdb = &common.PDB{MinAvailable: wdc.PodDisruptionBudget.MinAvailable, MaxUnavailable: wdc.PodDisruptionBudget.MaxUnavailable}
}
if response.Result = common.ErrorListToStatus(common.ValidateWebhookDeploymentCustomization(
wdc.ReplicaCount, wdc.AppendTolerations, wdc.OverrideAffinity, pdb,
field.NewPath("spec", "webhookDeploymentCustomization"))); response.Result != nil {
return response, nil
}
}

if a.settingCache != nil {
// The following checks don't make sense for downstream clusters (settingCache == nil)
response, err = a.validateVersionManagementFeature(oldCluster, newCluster, request.Operation)
Expand Down
Loading