diff --git a/documentation/azure-devops/setup-aks-provisioning-pipeline.asciidoc b/documentation/azure-devops/setup-aks-provisioning-pipeline.asciidoc index 2e70283b8..9aa13b597 100644 --- a/documentation/azure-devops/setup-aks-provisioning-pipeline.asciidoc +++ b/documentation/azure-devops/setup-aks-provisioning-pipeline.asciidoc @@ -1,17 +1,25 @@ -= Setting up a Azure AKS provisioning pipeline on Azure DevOps +:provider: Azure Devops +:pipeline_type: pipeline +:trigger_sentence_azure: +:pipeline_type2: pipeline +:path_provider: azure-devops +:aks_variables_path: Azure DevOps > Pipelines > Library > `aks-variables` += Setting up a Azure AKS provisioning {pipeline_type} on {provider} -In this section we will create a pipeline which will provision an Azure AKS cluster. This pipeline will be configured to be manually triggered by the user. As part of AKS cluster provisioning, a NGINX Ingress controller is deployed and a variable group with the name `aks-variables` is created, which contains, among others, the DNS name of the Ingress controller, that you you will need to add as CNAME record on the domains used in your application Ingress manifest files. Refer to the appendix for more details. +In this section we will create a {pipeline_type} which will provision an Azure AKS cluster. This {pipeline_type} will be configured to be manually triggered by the user. As part of AKS cluster provisioning, a NGINX Ingress controller is deployed and a variable group with the name `aks-variables` is created, which contains, among others, the DNS name of the Ingress controller, that you you will need to add as CNAME record on the domains used in your application Ingress manifest files. Refer to the appendix for more details. -The creation of the pipeline will follow the project workflow, so a new branch named `feature/aks-provisioning` will be created, the YAML file for the pipeline and the terraform files for creating the cluster will be pushed to it. +The creation of the {pipeline_type} will follow the project workflow, so a new branch named `feature/aks-provisioning` will be created, the YAML file for the {pipeline_type} and the terraform files for creating the cluster will be pushed to it. Then, a Pull Request (PR) will be created in order to merge the new branch into the appropiate branch (provided in `-b` flag). The PR will be automatically merged if the repository policies are met. If the merge is not possible, either the PR URL will be shown as output, or it will be opened in your web browser if using `-w` flag. -The script located at `/scripts/pipelines/azure-devops/pipeline_generator.sh` will automatically create this new branch, create the AKS provisioning pipeline based on the YAML template, create the Pull Request and, if it is possible, merge this new branch into the specified branch. +The script located at `/scripts/pipelines/{path_provider}/pipeline_generator.sh` will automatically create this new branch, create the AKS provisioning {pipeline_type} based on the YAML template, create the Pull Request and, if it is possible, merge this new branch into the specified branch. == Prerequisites -* Install the https://marketplace.visualstudio.com/items?itemName=ms-devlabs.custom-terraform-tasks[Terraform extension] for Azure DevOps. -* Create a https://docs.microsoft.com/en-us/azure/devops/pipelines/library/service-endpoints?view=azure-devops&tabs=yaml#create-a-service-connection[service connection] to Azure Resource Manager and name it `aks-connection`. If you already have a service connection available or you need a specific connection name, please update `aks-pipeline.cfg` accordingly. + * Install the https://marketplace.visualstudio.com/items?itemName=ms-devlabs.custom-terraform-tasks[Terraform extension] for Azure DevOps. + * Create a https://docs.microsoft.com/en-us/azure/devops/pipelines/library/service-endpoints?view=azure-devops&tabs=yaml#create-a-service-connection[service connection] to Azure Resource Manager and name it `aks-connection`. If you already have a service connection available or you need a specific connection name, please update `aks-pipeline.cfg` accordingly. + + * An Azure resource group in the desired cluster location (e.g. `westeurope`). You can use an existing one or create a new one with the following command: ``` @@ -32,7 +40,7 @@ az storage container create -n --account-name kubectl --kubeconfig= ``` -To get the DNS name of the NGINX Ingress controller on the EKS cluster, go into Azure DevOps > Pipelines > Library > `aks-variables`. +To get the DNS name of the NGINX Ingress controller on the AKS cluster, go into {aks_variables_path}. Rancher, if installed, will be available on `https:///dashboard`. You will be asked for an initial password, which can be retrieved with: @@ -104,4 +112,4 @@ kubectl get secret --namespace cattle-system bootstrap-secret -o go-template='{{ === Appendix: Destroying the cluster -To destroy the provisioned resources, set `operation` pipeline variable value to `destroy` and run the pipeline. \ No newline at end of file +To destroy the provisioned resources, set `operation` {pipeline_type} variable value to `destroy` and run the {pipeline_type}. diff --git a/documentation/github/setup-aks-provisioning-pipeline.asciidoc b/documentation/github/setup-aks-provisioning-pipeline.asciidoc new file mode 100644 index 000000000..439e483de --- /dev/null +++ b/documentation/github/setup-aks-provisioning-pipeline.asciidoc @@ -0,0 +1,114 @@ +:provider: Github +:pipeline_type: workflow +:trigger_sentence_github: +:pipeline_type2: Github action +:path_provider: github +:aks_variables_path: .github > vars > `deployment-env-variables.env` += Setting up a Azure AKS provisioning {pipeline_type} on {provider} + +In this section we will create a {pipeline_type} which will provision an Azure AKS cluster. This {pipeline_type} will be configured to be manually triggered by the user. As part of AKS cluster provisioning, a NGINX Ingress controller is deployed and a variable group with the name `aks-variables` is created, which contains, among others, the DNS name of the Ingress controller, that you you will need to add as CNAME record on the domains used in your application Ingress manifest files. Refer to the appendix for more details. + +The creation of the {pipeline_type} will follow the project workflow, so a new branch named `feature/aks-provisioning` will be created, the YAML file for the {pipeline_type} and the terraform files for creating the cluster will be pushed to it. + +Then, a Pull Request (PR) will be created in order to merge the new branch into the appropiate branch (provided in `-b` flag). The PR will be automatically merged if the repository policies are met. If the merge is not possible, either the PR URL will be shown as output, or it will be opened in your web browser if using `-w` flag. + +The script located at `/scripts/pipelines/{path_provider}/pipeline_generator.sh` will automatically create this new branch, create the AKS provisioning {pipeline_type} based on the YAML template, create the Pull Request and, if it is possible, merge this new branch into the specified branch. + +== Prerequisites + + + * Add AZURE credentials as https://docs.github.com/en/actions/security-guides/encrypted-secrets#creating-encrypted-secrets-for-a-repository[Github Secrets] in your repository and name it `AZURE_USERNAME`, `AZURE_PASSWORD`. If you already have a available credentials or you need a specific credentials connection, please update `aks-provisioning.yml` accordingly. + +* An Azure resource group in the desired cluster location (e.g. `westeurope`). You can use an existing one or create a new one with the following command: + +``` +az group create -n -l +``` + +* An Azure storage account within the previous resource group. You can use an existing one or create a new one with the following command: + +``` +az storage account create -n -g -l +``` + +* An Azure storage container in Azure within the previous storage account. You can use an existing one or create a new one with the following command: + +``` +az storage container create -n --account-name +``` + +* This script will commit and push the corresponding YAML template into your repository, so please be sure your local repository is up-to-date (i.e you have pulled the latest changes with `git pull`). + +== Creating the {pipeline_type} using provided script + +Before executing the script you will need to customize some input variables about the environment. To do so, you can either edit `terraform.tfvars` file or take advantage of the `set-terraform-variables.sh` script located at `/scripts/environment-provisioning/azure/aks`, which allows you to create or update values for the required variables, passing them as flags. As a full example: + +``` +./set-terraform-variables.sh --location --resource_group_name --instance_type --worker_node_count --dns_prefix +``` + +=== Usage +``` +pipeline_generator.sh \ + -c \ + -n \ + -d \ + --cluster-name \ + --resource-group \ + --storage-account \ + --storage-container \ + [--rancher] \ + [-b ] \ + [-w] +``` + +NOTE: The config file for the AKS provisioning {pipeline_type} is located at `/scripts/pipelines/{path_provider}/templates/aks/aks-pipeline.cfg`. + +=== Flags +``` +-c, --config-file [Required] Configuration file containing pipeline definition. +-n, --pipeline-name [Required] Name that will be set to the pipeline. +-d, --local-directory [Required] Local directory of your project (the path should always be using '/' and not '\'). + --cluster-name [Required] Name for the cluster. + --resource-group [Required] Name of the resource group for the cluster. + --storage--account [Required] Name of the storage account for the cluster. + --storage-container [Required] Name of the storage container where the Terraform state of the cluster will be stored. + --rancher Install Rancher to manage the cluster. +-b, --target-branch Name of the branch to which the Pull Request will target. PR is not created if the flag is not provided. +-w Open the Pull Request on the web browser if it cannot be automatically merged. Requires -b flag. +``` + +=== Example + +``` +./pipeline_generator.sh -c ./templates/aks/aks-pipeline.cfg -n aks-provisioning -d C:/Users/$USERNAME/Desktop/quarkus-project --cluster-name devon-hangar --resource-group devonfw --storage-account hangar --storage-container aks-state --rancher -b develop -w +``` + +NOTE: Rancher is installed on the cluster after provisioning when using the above command. + +=== Appendix: Interacting with the cluster + +NOTE: Make sure you have https://kubernetes.io/docs/tasks/tools/#kubectl[kubectl] installed. + +In order to interact with your cluster you will need to download the artifact `kubeconfig` generated by the cluster provisioning {pipeline_type} on the location it is expected by default (`~/.kube/config`) or either: + +``` +# via environment variable (you can add this on your profile) +export KUBECONFIG= +kubectl + +# via command-line flag +kubectl --kubeconfig= +``` + +To get the DNS name of the NGINX Ingress controller on the AKS cluster, go into {aks_variables_path}. + +Rancher, if installed, will be available on `https:///dashboard`. You will be asked for an initial password, which can be retrieved with: + +``` +kubectl get secret --namespace cattle-system bootstrap-secret -o go-template='{{.data.bootstrapPassword|base64decode}}{{"\n"}}' +``` + +=== Appendix: Destroying the cluster + +To destroy the provisioned resources, set `operation` {pipeline_type} variable value to `destroy` and run the {pipeline_type}. diff --git a/documentation/src/azure-devops/setup-aks-provisioning-pipeline.asciidoc b/documentation/src/azure-devops/setup-aks-provisioning-pipeline.asciidoc new file mode 100644 index 000000000..41cf611d2 --- /dev/null +++ b/documentation/src/azure-devops/setup-aks-provisioning-pipeline.asciidoc @@ -0,0 +1,7 @@ +:provider: Azure Devops +:pipeline_type: pipeline +:trigger_sentence_azure: +:pipeline_type2: pipeline +:path_provider: azure-devops +:aks_variables_path: Azure DevOps > Pipelines > Library > `aks-variables` +include::../common_templates/setup-aks-provisioning-pipeline.asciidoc[] diff --git a/documentation/src/azure-devops/setup-package-pipeline.asciidoc b/documentation/src/azure-devops/setup-package-pipeline.asciidoc index c1ca8de6d..afe64c7e3 100644 --- a/documentation/src/azure-devops/setup-package-pipeline.asciidoc +++ b/documentation/src/azure-devops/setup-package-pipeline.asciidoc @@ -1,11 +1,11 @@ -:provider: Azure DevOps -:pipeline_type: pipeline -:trigger_sentence: This pipeline will be configured in order to be triggered every time quality pipeline is executed successfully on a commit for release/* and develop branches, requiring manual launch for other branches but still enforcing that quality pipeline has passed -:pipeline_type2: pipeline -:path_provider: azure-devops -:build-pipeline: -:quality-pipeline: -:extra_args_quarkus: --build-pipeline-name quarkus-project-build --quality-pipeline-name quarkus-project-quality -:extra_args_node: --build-pipeline-name node-project-build --quality-pipeline-name node-project-quality -:extra_args_angular: --build-pipeline-name angular-project-build --quality-pipeline-name angular-project-quality +:provider: Azure DevOps +:pipeline_type: pipeline +:trigger_sentence: This pipeline will be configured in order to be triggered every time quality pipeline is executed successfully on a commit for release/* and develop branches, requiring manual launch for other branches but still enforcing that quality pipeline has passed +:pipeline_type2: pipeline +:path_provider: azure-devops +:build-pipeline: +:quality-pipeline: +:extra_args_quarkus: --build-pipeline-name quarkus-project-build --quality-pipeline-name quarkus-project-quality +:extra_args_node: --build-pipeline-name node-project-build --quality-pipeline-name node-project-quality +:extra_args_angular: --build-pipeline-name angular-project-build --quality-pipeline-name angular-project-quality include::../common_templates/setup-package-pipeline.asciidoc[] \ No newline at end of file diff --git a/documentation/src/azure-devops/setup-quality-pipeline.asciidoc b/documentation/src/azure-devops/setup-quality-pipeline.asciidoc index 17239e63d..1e6d9b66e 100644 --- a/documentation/src/azure-devops/setup-quality-pipeline.asciidoc +++ b/documentation/src/azure-devops/setup-quality-pipeline.asciidoc @@ -1,11 +1,11 @@ -:provider: Azure DevOps -:pipeline_type: pipeline -:trigger_sentence: This pipeline will be configured in order to be triggered every time the test pipeline is executed successfully on a commit -:pipeline_type2: pipeline -:path_provider: azure-devops -:build-pipeline: -:test-pipeline: -:extra_args_quarkus: --build-pipeline-name quarkus-project-build --test-pipeline-name quarkus-project-test -:extra_args_node: --build-pipeline-name node-project-build --test-pipeline-name node-project-test -:extra_args_angular: --build-pipeline-name angular-project-build --test-pipeline-name angular-project-test +:provider: Azure DevOps +:pipeline_type: pipeline +:trigger_sentence: This pipeline will be configured in order to be triggered every time the test pipeline is executed successfully on a commit +:pipeline_type2: pipeline +:path_provider: azure-devops +:build-pipeline: +:test-pipeline: +:extra_args_quarkus: --build-pipeline-name quarkus-project-build --test-pipeline-name quarkus-project-test +:extra_args_node: --build-pipeline-name node-project-build --test-pipeline-name node-project-test +:extra_args_angular: --build-pipeline-name angular-project-build --test-pipeline-name angular-project-test include::../common_templates/setup-quality-pipeline.asciidoc[] \ No newline at end of file diff --git a/documentation/src/common_templates/setup-aks-provisioning-pipeline.asciidoc b/documentation/src/common_templates/setup-aks-provisioning-pipeline.asciidoc new file mode 100644 index 000000000..208c34980 --- /dev/null +++ b/documentation/src/common_templates/setup-aks-provisioning-pipeline.asciidoc @@ -0,0 +1,110 @@ += Setting up a Azure AKS provisioning {pipeline_type} on {provider} + +In this section we will create a {pipeline_type} which will provision an Azure AKS cluster. This {pipeline_type} will be configured to be manually triggered by the user. As part of AKS cluster provisioning, a NGINX Ingress controller is deployed and a variable group with the name `aks-variables` is created, which contains, among others, the DNS name of the Ingress controller, that you you will need to add as CNAME record on the domains used in your application Ingress manifest files. Refer to the appendix for more details. + +The creation of the {pipeline_type} will follow the project workflow, so a new branch named `feature/aks-provisioning` will be created, the YAML file for the {pipeline_type} and the terraform files for creating the cluster will be pushed to it. + +Then, a Pull Request (PR) will be created in order to merge the new branch into the appropiate branch (provided in `-b` flag). The PR will be automatically merged if the repository policies are met. If the merge is not possible, either the PR URL will be shown as output, or it will be opened in your web browser if using `-w` flag. + +The script located at `/scripts/pipelines/{path_provider}/pipeline_generator.sh` will automatically create this new branch, create the AKS provisioning {pipeline_type} based on the YAML template, create the Pull Request and, if it is possible, merge this new branch into the specified branch. + +== Prerequisites + +ifdef::trigger_sentence_azure[ * Install the https://marketplace.visualstudio.com/items?itemName=ms-devlabs.custom-terraform-tasks[Terraform extension] for Azure DevOps.] +ifdef::trigger_sentence_azure[ * Create a https://docs.microsoft.com/en-us/azure/devops/pipelines/library/service-endpoints?view=azure-devops&tabs=yaml#create-a-service-connection[service connection] to Azure Resource Manager and name it `aks-connection`. If you already have a service connection available or you need a specific connection name, please update `aks-pipeline.cfg` accordingly.] + +ifdef::trigger_sentence_github[ * Add AZURE credentials as https://docs.github.com/en/actions/security-guides/encrypted-secrets#creating-encrypted-secrets-for-a-repository[Github Secrets] in your repository and name it `AZURE_USERNAME`, `AZURE_PASSWORD`. If you already have a available credentials or you need a specific credentials connection, please update `aks-provisioning.yml` accordingly.] + +* An Azure resource group in the desired cluster location (e.g. `westeurope`). You can use an existing one or create a new one with the following command: + +``` +az group create -n -l +``` + +* An Azure storage account within the previous resource group. You can use an existing one or create a new one with the following command: + +``` +az storage account create -n -g -l +``` + +* An Azure storage container in Azure within the previous storage account. You can use an existing one or create a new one with the following command: + +``` +az storage container create -n --account-name +``` + +* This script will commit and push the corresponding YAML template into your repository, so please be sure your local repository is up-to-date (i.e you have pulled the latest changes with `git pull`). + +== Creating the {pipeline_type} using provided script + +Before executing the script you will need to customize some input variables about the environment. To do so, you can either edit `terraform.tfvars` file or take advantage of the `set-terraform-variables.sh` script located at `/scripts/environment-provisioning/azure/aks`, which allows you to create or update values for the required variables, passing them as flags. As a full example: + +``` +./set-terraform-variables.sh --location --resource_group_name --instance_type --worker_node_count --dns_prefix +``` + +=== Usage +``` +pipeline_generator.sh \ + -c \ + -n \ + -d \ + --cluster-name \ + --resource-group \ + --storage-account \ + --storage-container \ + [--rancher] \ + [-b ] \ + [-w] +``` + +NOTE: The config file for the AKS provisioning {pipeline_type} is located at `/scripts/pipelines/{path_provider}/templates/aks/aks-pipeline.cfg`. + +=== Flags +``` +-c, --config-file [Required] Configuration file containing pipeline definition. +-n, --pipeline-name [Required] Name that will be set to the pipeline. +-d, --local-directory [Required] Local directory of your project (the path should always be using '/' and not '\'). + --cluster-name [Required] Name for the cluster. + --resource-group [Required] Name of the resource group for the cluster. + --storage--account [Required] Name of the storage account for the cluster. + --storage-container [Required] Name of the storage container where the Terraform state of the cluster will be stored. + --rancher Install Rancher to manage the cluster. +-b, --target-branch Name of the branch to which the Pull Request will target. PR is not created if the flag is not provided. +-w Open the Pull Request on the web browser if it cannot be automatically merged. Requires -b flag. +``` + +=== Example + +``` +./pipeline_generator.sh -c ./templates/aks/aks-pipeline.cfg -n aks-provisioning -d C:/Users/$USERNAME/Desktop/quarkus-project --cluster-name devon-hangar --resource-group devonfw --storage-account hangar --storage-container aks-state --rancher -b develop -w +``` + +NOTE: Rancher is installed on the cluster after provisioning when using the above command. + +=== Appendix: Interacting with the cluster + +NOTE: Make sure you have https://kubernetes.io/docs/tasks/tools/#kubectl[kubectl] installed. + +In order to interact with your cluster you will need to download the artifact `kubeconfig` generated by the cluster provisioning {pipeline_type} on the location it is expected by default (`~/.kube/config`) or either: + +``` +# via environment variable (you can add this on your profile) +export KUBECONFIG= +kubectl + +# via command-line flag +kubectl --kubeconfig= +``` + +To get the DNS name of the NGINX Ingress controller on the AKS cluster, go into {aks_variables_path}. + +Rancher, if installed, will be available on `https:///dashboard`. You will be asked for an initial password, which can be retrieved with: + +``` +kubectl get secret --namespace cattle-system bootstrap-secret -o go-template='{{.data.bootstrapPassword|base64decode}}{{"\n"}}' +``` + +=== Appendix: Destroying the cluster + +To destroy the provisioned resources, set `operation` {pipeline_type} variable value to `destroy` and run the {pipeline_type}. diff --git a/documentation/src/common_templates/setup-package-pipeline.asciidoc b/documentation/src/common_templates/setup-package-pipeline.asciidoc index 840de0a1a..8dcf98eb2 100644 --- a/documentation/src/common_templates/setup-package-pipeline.asciidoc +++ b/documentation/src/common_templates/setup-package-pipeline.asciidoc @@ -1,105 +1,105 @@ -= Setting up a Package {pipeline_type} on {provider} - -In this section we will create a package {pipeline_type} to build and push a container image of the project application into the specified container registry. {trigger_sentence}. It consumes the artifact produced by the build {pipeline_type}. - -The creation of the {pipeline_type2} will follow the project workflow, so a new branch named `feature/package-pipeline` will be created and the YAML file for the {pipeline_type} will be pushed to it. - -Then, a Pull Request (PR) will be created in order to merge the new branch into the appropriate branch (provided in `-b` flag). The PR will be automatically merged if the repository policies are met. If the merge is not possible, either the PR URL will be shown as output, or it will be opened in your web browser if using `-w` flag. - -The script located at `/scripts/pipelines/{path_provider}/pipeline_generator.sh` will automatically create this new branch, create a package {pipeline_type} based on a YAML template, create the Pull Request and, if it is possible, merge this new branch into the specified branch. - -== Prerequisites - -This script will commit and push the corresponding YAML template into your repository, so please be sure your local repository is up-to-date (i.e you have pulled latest changes with `git pull`). - -== Creating the {pipeline_type} using provided script - -=== Usage -[subs=attributes+] -``` -pipeline_generator.sh \ - -c \ - -n <{pipeline_type} name> \ - -l \ - --dockerfile \ - -d \ -ifdef::build-pipeline[ --build-pipeline-name \] -ifdef::quality-pipeline[ --quality-pipeline-name \] -ifdef::ci-pipeline[ --ci-pipeline-name \] - -i \ - [-u ] \ - [-p ] \ - [--aws-access-key ] \ - [--aws-secret-access-key ] \ - [--aws-region ] \ - [-b ] \ - [-w] - -``` - -NOTE: The config file for the package {pipeline_type} is located at `/scripts/pipelines/{path_provider}/templates/package/package-pipeline.cfg`. - - -=== Flags -[subs=attributes+] -``` --c, --config-file [Required] Configuration file containing {pipeline_type} definition. --n, --pipeline-name [Required] Name that will be set to the {pipeline_type}. --l, --language [Required, if dockerfile not set] Language or framework of the project. - --dockerfile [Required, if language not set] Path from the root of the project to its Dockerfile. Takes precedence over the language/framework default one. --d, --local-directory [Required] Local directory of your project. -ifdef::build-pipeline[ --build-pipeline-name [Required] Build {pipeline_type} name.] -ifdef::quality-pipeline[ --quality-pipeline-name [Required] Quality {pipeline_type} name.] -ifdef::ci-pipeline[ --ci-pipeline-name [Required] CI {pipeline_type} name.] --i, --image-name [Required] Name (excluding tag) for the generated container image. --u, --registry-user [Required, unless AWS] Container registry login user. --p, --registry-password [Required, unless AWS] Container registry login password. - --aws-access-key [Required, if AWS] AWS account access key ID. Takes precedence over registry credentials." - --aws-secret-access-key [Required, if AWS] AWS account secret access key." - --aws-region [Required, if AWS] AWS region for ECR." --b, --target-branch Name of the branch to which the Pull Request will target. PR is not created if the flag is not provided. --w Open the Pull Request on the web browser if it cannot be automatically merged. Requires -b flag. -``` - -=== Examples - -==== Quarkus project - -===== Generic container registry -[subs=attributes+] -``` -./pipeline_generator.sh -c ./templates/package/package-pipeline.cfg -n quarkus-project-package -l quarkus -d C:/Users/$USERNAME/Desktop/quarkus-project -i username/quarkus-project -u username -p password {extra_args_quarkus} -b develop -w -``` - -===== AWS ECR -[subs=attributes+] -``` -./pipeline_generator.sh -c ./templates/package/package-pipeline.cfg -n quarkus-project-package -l quarkus -d C:/Users/$USERNAME/Desktop/quarkus-project -i username/quarkus-project --aws-access-key AKIAIOSFODNN7EXAMPLE --aws-secret-access-key wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY --aws-region eu-west-1 {extra_args_quarkus} -b develop -w -``` - -==== Node.js project - -===== Generic container registry -[subs=attributes+] -``` -./pipeline_generator.sh -c ./templates/package/package-pipeline.cfg -n node-project-package -l node -d C:/Users/$USERNAME/Desktop/node-project -i username/node-project -u username -p password {extra_args_node} -b develop -w -``` - -===== AWS ECR -[subs=attributes+] -``` -./pipeline_generator.sh -c ./templates/package/package-pipeline.cfg -n node-project-package -l node -d C:/Users/$USERNAME/Desktop/node-project -i username/node-project --aws-access-key AKIAIOSFODNN7EXAMPLE --aws-secret-access-key wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY --aws-region eu-west-1 {extra_args_node} -b develop -w -``` -==== Angular project - -===== Generic container registry -[subs=attributes+] -``` -./pipeline_generator.sh -c ./templates/package/package-pipeline.cfg -n angular-project-package -l angular -d C:/Users/$USERNAME/Desktop/angular-project --build-pipeline-name angular-project-build --quality-pipeline-name angular-project-quality -i username/angular-project -u username -p password -b develop -w -``` - -===== AWS ECR -[subs=attributes+] -``` -./pipeline_generator.sh -c ./templates/package/package-pipeline.cfg -n angular-project-package -l angular -d C:/Users/$USERNAME/Desktop/angular-project --build-pipeline-name angular-project-build --quality-pipeline-name angular-project-quality -i username/angular-project --aws-access-key AKIAIOSFODNN7EXAMPLE --aws-secret-access-key wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY --aws-region eu-west-1 -b develop -w += Setting up a Package {pipeline_type} on {provider} + +In this section we will create a package {pipeline_type} to build and push a container image of the project application into the specified container registry. {trigger_sentence}. It consumes the artifact produced by the build {pipeline_type}. + +The creation of the {pipeline_type2} will follow the project workflow, so a new branch named `feature/package-pipeline` will be created and the YAML file for the {pipeline_type} will be pushed to it. + +Then, a Pull Request (PR) will be created in order to merge the new branch into the appropriate branch (provided in `-b` flag). The PR will be automatically merged if the repository policies are met. If the merge is not possible, either the PR URL will be shown as output, or it will be opened in your web browser if using `-w` flag. + +The script located at `/scripts/pipelines/{path_provider}/pipeline_generator.sh` will automatically create this new branch, create a package {pipeline_type} based on a YAML template, create the Pull Request and, if it is possible, merge this new branch into the specified branch. + +== Prerequisites + +This script will commit and push the corresponding YAML template into your repository, so please be sure your local repository is up-to-date (i.e you have pulled latest changes with `git pull`). + +== Creating the {pipeline_type} using provided script + +=== Usage +[subs=attributes+] +``` +pipeline_generator.sh \ + -c \ + -n <{pipeline_type} name> \ + -l \ + --dockerfile \ + -d \ +ifdef::build-pipeline[ --build-pipeline-name \] +ifdef::quality-pipeline[ --quality-pipeline-name \] +ifdef::ci-pipeline[ --ci-pipeline-name \] + -i \ + [-u ] \ + [-p ] \ + [--aws-access-key ] \ + [--aws-secret-access-key ] \ + [--aws-region ] \ + [-b ] \ + [-w] + +``` + +NOTE: The config file for the package {pipeline_type} is located at `/scripts/pipelines/{path_provider}/templates/package/package-pipeline.cfg`. + + +=== Flags +[subs=attributes+] +``` +-c, --config-file [Required] Configuration file containing {pipeline_type} definition. +-n, --pipeline-name [Required] Name that will be set to the {pipeline_type}. +-l, --language [Required, if dockerfile not set] Language or framework of the project. + --dockerfile [Required, if language not set] Path from the root of the project to its Dockerfile. Takes precedence over the language/framework default one. +-d, --local-directory [Required] Local directory of your project. +ifdef::build-pipeline[ --build-pipeline-name [Required] Build {pipeline_type} name.] +ifdef::quality-pipeline[ --quality-pipeline-name [Required] Quality {pipeline_type} name.] +ifdef::ci-pipeline[ --ci-pipeline-name [Required] CI {pipeline_type} name.] +-i, --image-name [Required] Name (excluding tag) for the generated container image. +-u, --registry-user [Required, unless AWS] Container registry login user. +-p, --registry-password [Required, unless AWS] Container registry login password. + --aws-access-key [Required, if AWS] AWS account access key ID. Takes precedence over registry credentials." + --aws-secret-access-key [Required, if AWS] AWS account secret access key." + --aws-region [Required, if AWS] AWS region for ECR." +-b, --target-branch Name of the branch to which the Pull Request will target. PR is not created if the flag is not provided. +-w Open the Pull Request on the web browser if it cannot be automatically merged. Requires -b flag. +``` + +=== Examples + +==== Quarkus project + +===== Generic container registry +[subs=attributes+] +``` +./pipeline_generator.sh -c ./templates/package/package-pipeline.cfg -n quarkus-project-package -l quarkus -d C:/Users/$USERNAME/Desktop/quarkus-project -i username/quarkus-project -u username -p password {extra_args_quarkus} -b develop -w +``` + +===== AWS ECR +[subs=attributes+] +``` +./pipeline_generator.sh -c ./templates/package/package-pipeline.cfg -n quarkus-project-package -l quarkus -d C:/Users/$USERNAME/Desktop/quarkus-project -i username/quarkus-project --aws-access-key AKIAIOSFODNN7EXAMPLE --aws-secret-access-key wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY --aws-region eu-west-1 {extra_args_quarkus} -b develop -w +``` + +==== Node.js project + +===== Generic container registry +[subs=attributes+] +``` +./pipeline_generator.sh -c ./templates/package/package-pipeline.cfg -n node-project-package -l node -d C:/Users/$USERNAME/Desktop/node-project -i username/node-project -u username -p password {extra_args_node} -b develop -w +``` + +===== AWS ECR +[subs=attributes+] +``` +./pipeline_generator.sh -c ./templates/package/package-pipeline.cfg -n node-project-package -l node -d C:/Users/$USERNAME/Desktop/node-project -i username/node-project --aws-access-key AKIAIOSFODNN7EXAMPLE --aws-secret-access-key wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY --aws-region eu-west-1 {extra_args_node} -b develop -w +``` +==== Angular project + +===== Generic container registry +[subs=attributes+] +``` +./pipeline_generator.sh -c ./templates/package/package-pipeline.cfg -n angular-project-package -l angular -d C:/Users/$USERNAME/Desktop/angular-project --build-pipeline-name angular-project-build --quality-pipeline-name angular-project-quality -i username/angular-project -u username -p password -b develop -w +``` + +===== AWS ECR +[subs=attributes+] +``` +./pipeline_generator.sh -c ./templates/package/package-pipeline.cfg -n angular-project-package -l angular -d C:/Users/$USERNAME/Desktop/angular-project --build-pipeline-name angular-project-build --quality-pipeline-name angular-project-quality -i username/angular-project --aws-access-key AKIAIOSFODNN7EXAMPLE --aws-secret-access-key wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY --aws-region eu-west-1 -b develop -w ``` \ No newline at end of file diff --git a/documentation/src/github/setup-aks-provisioning-pipeline.asciidoc b/documentation/src/github/setup-aks-provisioning-pipeline.asciidoc new file mode 100644 index 000000000..0b50c4a5d --- /dev/null +++ b/documentation/src/github/setup-aks-provisioning-pipeline.asciidoc @@ -0,0 +1,7 @@ +:provider: Github +:pipeline_type: workflow +:trigger_sentence_github: +:pipeline_type2: Github action +:path_provider: github +:aks_variables_path: .github > vars > `deployment-env-variables.env` +include::../common_templates/setup-aks-provisioning-pipeline.asciidoc[] diff --git a/documentation/src/github/setup-package-pipeline.asciidoc b/documentation/src/github/setup-package-pipeline.asciidoc index b0a1152b8..94bf6e632 100644 --- a/documentation/src/github/setup-package-pipeline.asciidoc +++ b/documentation/src/github/setup-package-pipeline.asciidoc @@ -1,10 +1,10 @@ -:provider: GitHub -:pipeline_type: workflow -:trigger_sentence: This workflow will be configured in order to be triggered every time CI workflow is executed successfully on a commit for release/* and develop branches, requiring manual launch for other branches but still enforcing that CI workflow has passed -:pipeline_type2: GitHub action -:path_provider: github -:ci-pipeline: -:extra_args_quarkus: --ci-pipeline-name quarkus-project-ci -:extra_args_node: --ci-pipeline-name node-project-ci -:extra_args_angular: --ci-pipeline-name angular-project-ci +:provider: GitHub +:pipeline_type: workflow +:trigger_sentence: This workflow will be configured in order to be triggered every time CI workflow is executed successfully on a commit for release/* and develop branches, requiring manual launch for other branches but still enforcing that CI workflow has passed +:pipeline_type2: GitHub action +:path_provider: github +:ci-pipeline: +:extra_args_quarkus: --ci-pipeline-name quarkus-project-ci +:extra_args_node: --ci-pipeline-name node-project-ci +:extra_args_angular: --ci-pipeline-name angular-project-ci include::../common_templates/setup-package-pipeline.asciidoc[] \ No newline at end of file diff --git a/documentation/src/github/setup-quality-pipeline.asciidoc b/documentation/src/github/setup-quality-pipeline.asciidoc index 6e79574f8..1b3c75e92 100644 --- a/documentation/src/github/setup-quality-pipeline.asciidoc +++ b/documentation/src/github/setup-quality-pipeline.asciidoc @@ -1,11 +1,11 @@ -:provider: GitHub -:pipeline_type: workflow -:trigger_sentence: This workflow will be configured to be executed as a job inside a CI workflow after the test (or build, if no test) job -:pipeline_type2: GitHub action -:path_provider: github -:extra_args_quarkus: -:extra_args_node: -:extra_args_angular: -:extra_sentence_ci: Please note that this workflow, although manually triggerable, is designed to be executed as part of a CI workflow, which you can create following this xref:setup-ci-pipeline.asciidoc[guide]. -include::../common_templates/setup-quality-pipeline.asciidoc[] - +:provider: GitHub +:pipeline_type: workflow +:trigger_sentence: This workflow will be configured to be executed as a job inside a CI workflow after the test (or build, if no test) job +:pipeline_type2: GitHub action +:path_provider: github +:extra_args_quarkus: +:extra_args_node: +:extra_args_angular: +:extra_sentence_ci: Please note that this workflow, although manually triggerable, is designed to be executed as part of a CI workflow, which you can create following this xref:setup-ci-pipeline.asciidoc[guide]. +include::../common_templates/setup-quality-pipeline.asciidoc[] + diff --git a/documentation/src/github/setup-test-pipeline.asciidoc b/documentation/src/github/setup-test-pipeline.asciidoc index ba547babe..66da72033 100644 --- a/documentation/src/github/setup-test-pipeline.asciidoc +++ b/documentation/src/github/setup-test-pipeline.asciidoc @@ -1,10 +1,10 @@ -:provider: GitHub -:pipeline_type: workflow -:trigger_sentence: This workflow will be configured to be executed as a job inside a CI workflow after the build job -:pipeline_type2: GitHub action -:path_provider: github -:extra_args_quarkus: -:extra_args_node: -:extra_args_angular: -:extra_sentence_ci: Please note that this workflow, although manually triggerable, is designed to be executed as part of a CI workflow, which you can create following this xref:setup-ci-pipeline.asciidoc[guide]. -include::../common_templates/setup-test-pipeline.asciidoc[] +:provider: GitHub +:pipeline_type: workflow +:trigger_sentence: This workflow will be configured to be executed as a job inside a CI workflow after the build job +:pipeline_type2: GitHub action +:path_provider: github +:extra_args_quarkus: +:extra_args_node: +:extra_args_angular: +:extra_sentence_ci: Please note that this workflow, although manually triggerable, is designed to be executed as part of a CI workflow, which you can create following this xref:setup-ci-pipeline.asciidoc[guide]. +include::../common_templates/setup-test-pipeline.asciidoc[] diff --git a/scripts/pipelines/azure-devops/pipeline_generator.sh b/scripts/pipelines/azure-devops/pipeline_generator.sh index f5ed33453..498884d32 100644 --- a/scripts/pipelines/azure-devops/pipeline_generator.sh +++ b/scripts/pipelines/azure-devops/pipeline_generator.sh @@ -4,40 +4,40 @@ FLAGS=$(getopt -a --options c:n:d:a:b:l:t:i:u:p:hw --long "config-file:,pipeline eval set -- "$FLAGS" while true; do case "$1" in - -c | --config-file) configFile=$2; shift 2;; - -n | --pipeline-name) pipelineName=$2; shift 2;; - -d | --local-directory) localDirectory=$2; shift 2;; - -a | --artifact-path) artifactPath=$2; shift 2;; - -b | --target-branch) targetBranch=$2; shift 2;; - -l | --language) language=$2; shift 2;; - -t | --target-directory) targetDirectory=$2; shift 2;; - --build-pipeline-name) export buildPipelineName=$2; shift 2;; - --sonar-url) sonarUrl=$2; shift 2;; - --sonar-token) sonarToken=$2; shift 2;; - -i | --image-name) imageName=$2; shift 2;; - -u | --registry-user) dockerUser=$2; shift 2;; - -p | --registry-password) dockerPassword=$2; shift 2;; - --resource-group) resourceGroupName=$2; shift 2;; - --storage-account) storageAccountName=$2; shift 2;; - --storage-container) storageContainerName=$2; shift 2;; - --rancher) installRancher="true"; shift 1;; - --cluster-name) clusterName=$2; shift 2;; - --s3-bucket) s3Bucket=$2; shift 2;; - --s3-key-path) s3KeyPath=$2; shift 2;; - --quality-pipeline-name) export qualityPipelineName=$2; shift 2;; - --test-pipeline-name) export testPipelineName=$2; shift 2;; - --dockerfile) dockerFile=$2; shift 2;; - --aws-access-key) awsAccessKey="$2"; shift 2;; - --aws-secret-access-key) awsSecretAccessKey="$2"; shift 2;; - --aws-region) awsRegion="$2"; shift 2;; - --package-pipeline-name) export packagePipelineName=$2; shift 2;; + -c | --config-file) configFile=$2; shift 2;; + -n | --pipeline-name) pipelineName=$2; shift 2;; + -d | --local-directory) localDirectory=$2; shift 2;; + -a | --artifact-path) artifactPath=$2; shift 2;; + -b | --target-branch) targetBranch=$2; shift 2;; + -l | --language) language=$2; shift 2;; + -t | --target-directory) targetDirectory=$2; shift 2;; + --build-pipeline-name) export buildPipelineName=$2; shift 2;; + --sonar-url) sonarUrl=$2; shift 2;; + --sonar-token) sonarToken=$2; shift 2;; + -i | --image-name) imageName=$2; shift 2;; + -u | --registry-user) dockerUser=$2; shift 2;; + -p | --registry-password) dockerPassword=$2; shift 2;; + --resource-group) resourceGroupName=$2; shift 2;; + --storage-account) storageAccountName=$2; shift 2;; + --storage-container) storageContainerName=$2; shift 2;; + --rancher) installRancher="true"; shift 1;; + --cluster-name) clusterName=$2; shift 2;; + --s3-bucket) s3Bucket=$2; shift 2;; + --s3-key-path) s3KeyPath=$2; shift 2;; + --quality-pipeline-name) export qualityPipelineName=$2; shift 2;; + --test-pipeline-name) export testPipelineName=$2; shift 2;; + --dockerfile) dockerFile=$2; shift 2;; + --aws-access-key) awsAccessKey="$2"; shift 2;; + --aws-secret-access-key) awsSecretAccessKey="$2"; shift 2;; + --aws-region) awsRegion="$2"; shift 2;; + --package-pipeline-name) export packagePipelineName=$2; shift 2;; --env-provision-pipeline-name) envProvisionPipelineName="$2"; shift 2;; - --k8s-provider) k8sProvider=$2; shift 2;; - --k8s-namespace) k8sNamespace="$2"; shift 2;; - --k8s-deploy-files-path) k8sDeployFiles=$2; shift 2;; - --k8s-image-pull-secret-name) k8sImagePullSecret=$2; shift 2;; - -h | --help) help="true"; shift 1;; - -w) webBrowser="true"; shift 1;; + --k8s-provider) k8sProvider=$2; shift 2;; + --k8s-namespace) k8sNamespace="$2"; shift 2;; + --k8s-deploy-files-path) k8sDeployFiles=$2; shift 2;; + --k8s-image-pull-secret-name) k8sImagePullSecret=$2; shift 2;; + -h | --help) help="true"; shift 1;; + -w) webBrowser="true"; shift 1;; --) shift; break;; esac done diff --git a/scripts/pipelines/azure-devops/templates/aks/aks-provisioning.yml.template b/scripts/pipelines/azure-devops/templates/aks/aks-provisioning.yml.template index 36b905911..e606cebef 100644 --- a/scripts/pipelines/azure-devops/templates/aks/aks-provisioning.yml.template +++ b/scripts/pipelines/azure-devops/templates/aks/aks-provisioning.yml.template @@ -93,4 +93,4 @@ steps: command: 'apply' commandOptions: '-destroy -var cluster_name=$(clusterName)' workingDirectory: '$(terraformWorkingDirectory)' - environmentServiceNameAzureRm: '$(serviceConnection)' + environmentServiceNameAzureRm: '$(serviceConnection)' \ No newline at end of file diff --git a/scripts/pipelines/azure-devops/templates/aks/obtain-dns.sh b/scripts/pipelines/azure-devops/templates/aks/obtain-dns.sh index 2acfae458..a45b6b205 100644 --- a/scripts/pipelines/azure-devops/templates/aks/obtain-dns.sh +++ b/scripts/pipelines/azure-devops/templates/aks/obtain-dns.sh @@ -18,4 +18,4 @@ az network public-ip update --resource-group "$iprg" --name "$ipname" --dns-name dns="$(az network public-ip show --resource-group "$iprg" --name "$ipname" --query "[dnsSettings.fqdn]" --output tsv)" -echo "##vso[task.setvariable variable=dns;]$dns" +echo "##vso[task.setvariable variable=dns;]$dns" \ No newline at end of file diff --git a/scripts/pipelines/common/pipeline_generator.lib b/scripts/pipelines/common/pipeline_generator.lib index 5ac0473f0..b7719ddf0 100644 --- a/scripts/pipelines/common/pipeline_generator.lib +++ b/scripts/pipelines/common/pipeline_generator.lib @@ -1,242 +1,242 @@ -#!/bin/bash -function help { - echo "" - echo "Generates a $pipeline_type on $provider based on the given definition." - echo "" - echo "Common flags:" - echo " -c, --config-file [Required] Configuration file containing $pipeline_type definition." - echo " -n, --pipeline-name [Required] Name that will be set to the $pipeline_type." - echo " -d, --local-directory [Required] Local directory of your project." - echo " -a, --artifact-path Path to be persisted as an artifact after $pipeline_type execution, e.g. where the application stores logs or any other blob on runtime." - echo " -b, --target-branch Name of the branch to which the Pull Request will target. PR is not created if the flag is not provided." - [[ "$provider" != "gcloud" ]] && echo " -w Open the Pull Request on the web browser if it cannot be automatically merged. Requires -b flag." - echo "" - echo "Build $pipeline_type flags:" - echo " -l, --language [Required] Language or framework of the project." - echo " -t, --target-directory Target directory of build process. Takes precedence over the language/framework default one." - echo "" - echo "Test $pipeline_type flags:" - echo " -l, --language [Required] Language or framework of the project." - [ "$provider" == "azure-devops" ] && echo " --build-pipeline-name [Required] Build $pipeline_type name." - echo "" - echo "Quality $pipeline_type flags:" - echo " -l, --language [Required] Language or framework of the project." - echo " --sonar-url [Required] Sonarqube URL." - echo " --sonar-token [Required] Sonarqube token." - [ "$provider" == "azure-devops" ] && echo " --build-pipeline-name [Required] Build $pipeline_type name." - [ "$provider" == "azure-devops" ] && echo " --test-pipeline-name [Required] Test $pipeline_type name." - echo "" - [ "$provider" == "github" ] && echo "" - [ "$provider" == "github" ] && echo "CI pipeline flags" - [ "$provider" == "github" ] && echo " --build-pipeline-name [Required] Name of the job calling the build $pipeline_type." - [ "$provider" == "github" ] && echo " --test-pipeline-name Name of the job calling the test $pipeline_type." - [ "$provider" == "github" ] && echo " --quality-pipeline-name Name of the job calling the quality $pipeline_type." - [ "$provider" == "github" ] && echo "" - echo "Package pipeline flags:" - echo " -l, --language [Required, if dockerfile not set] Language or framework of the project." - echo " --dockerfile [Required, if language not set] Path from the root of the project to its Dockerfile. Takes precedence over the language/framework default one." - [ "$provider" == "azure-devops" ] && echo " --build-pipeline-name [Required] Build $pipeline_type name." - [ "$provider" == "azure-devops" ] && echo " --quality-pipeline-name [Required] Quality $pipeline_type name." - [ "$provider" == "github" ] && echo " --ci-pipeline-name [Required] CI $pipeline_type name." - echo " -i, --image-name [Required] Name (excluding tag) for the generated container image." - echo " -u, --registry-user [Required, unless AWS] Container registry login user." - echo " -p, --registry-password [Required, unless AWS] Container registry login password." - echo " --aws-access-key [Required, if AWS] AWS account access key ID. Takes precedence over registry credentials." - echo " --aws-secret-access-key [Required, if AWS] AWS account secret access key." - echo " --aws-region [Required, if AWS] AWS region for ECR." - echo "" - echo "Library package $pipeline_type flags:" - echo " -l, --language [Required] Language or framework of the project." - echo "" - echo "Azure AKS provisioning $pipeline_type flags:" - echo " --resource-group [Required] Name of the resource group for the cluster." - echo " --storage-account [Required] Name of the storage account for the cluster." - echo " --storage-container [Required] Name of the storage container where the Terraform state of the cluster will be stored." - echo " --rancher Install Rancher to manage the cluster." - echo "" - echo "AWS EKS provisioning $pipeline_type flags:" - echo " --cluster-name [Required] Name for the cluster." - echo " --s3-bucket [Required] Name of the S3 bucket where the Terraform state of the cluster will be stored." - echo " --s3-key-path [Required] Path within the S3 bucket where the Terraform state of the cluster will be stored." - echo " --aws-access-key [Required, on first run] AWS account access key ID." - echo " --aws-secret-access-key [Required, on first run] AWS account secret access key." - echo " --aws-region [Required, on first run] AWS region for provisioning resources." - - echo " --rancher Install Rancher to manage the cluster." - echo "" - echo "Deploy $pipeline_type flags:" - echo "" - echo " --package-pipeline-name [Required] Package $pipeline_type name." - echo " --env-provision-pipeline-name [Required] Environment provisioning $pipeline_type name." - echo " --k8s-provider [Required] Kubernetes cluster provider name. Accepted values: EKS, AKS." - echo " --k8s-namespace [Required] Kubernetes namespace where the application will be deployed." - echo " --k8s-deploy-files-path [Required] Path from the root of the project to the YAML manifests directory." - echo " --k8s-image-pull-secret-name Name for the generated secret containing registry credentials. Required when using a private registry to host images." - echo "" - - exit -} - -function validateRegistryLoginCredentials { - # if the user chose to push to a registry and the user has not already given a password - # then prompt the user - if [ -v dockerUser ] && [ ! -v dockerPassword ] - then - read -rsp "Please enter Docker registry password..." dockerPassword - fi - - if [ -v awsRegion ] && [ -v awsAccessKey ] && [ ! -v awsSecretAccessKey ] - then - read -rsp "Please enter AWS secret access key..." awsSecretAccessKey - fi -} - -function ensurePathFormat { - currentDirectory=$(pwd) - - # When necessary, converts a relative path into an absolute path, and a Windows-style path (e.g. "C:\Users" or C:/Users) into a - # Unix-style path using forward slashes (e.g. "/c/Users"). - localDirectory=${localDirectory//'\'/"/"} - cd "${localDirectory}" || { echo -e "${red}Error: Local directory '${localDirectory}' does not exist. Check provided path (missing quotes?)."; exit 1; } - localDirectory=$(pwd) - - # Return to initial directory - cd "$currentDirectory" -} - -function importConfigFile { - # Import config file. - source $configFile - IFS=, read -ra flags <<< "$mandatoryFlags" - - # Check if the config file was supplied. - if test -z "$configFile" - then - echo -e "${red}Error: $pipeline_type definition configuration file not specified." >&2 - exit 2 - fi - - # Check if the required flags in the config file have been activated. - for flag in "${flags[@]}" - do - if test -z $flag - then - echo -e "${red}Error: Missing parameters, some flags are mandatory." >&2 - echo -e "${red}Use -h or --help flag to display help." >&2 - exit 2 - fi - done -} - -function checkInstallations { - # Check if Git is installed - if ! [ -x "$(command -v git)" ]; then - echo -e "${red}Error: Git is not installed." >&2 - exit 127 - fi - - # Check if the CLI tool for the provider is installed - if ([ "$provider" == "github" ] && ! [ -x "$(command -v gh)" ]); then - echo -e "${red}Error: Github CLI is not installed." >&2 - exit 127 - elif ([ "$provider" == "azure-devops" ] && ! [ -x "$(command -v az)" ]); then - echo -e "${red}Error: Azure CLI is not installed." >&2 - exit 127 - fi - - # Check if Python is installed - if ! [ -x "$(command -v python)" ]; then - echo -e "${red}Error: Python is not installed." >&2 - exit 127 - fi -} - -function createNewBranch { - echo -e "${green}Creating the new branch: ${sourceBranch}..." - echo -ne ${white} - - # Create the new branch. - cd "${localDirectory}" - git checkout -b ${sourceBranch} -} - -function copyYAMLFile { - echo -e "${green}Copying the corresponding files into your directory..." - echo -ne ${white} - - # Create .pipelines and scripts if they do not exist. - mkdir -p "${localDirectory}/$scriptFilePath" - - # Generate pipeline YAML from template and put it in the repository. - # We cannot use a variable in the definition of resource in the pipeline so we have to use a placeholder to replace it with the value we need - commonEnvSubstList='${buildPipelineName} ${testPipelineName} ${qualityPipelineName} ${pipelineName} ${ciPipelineName} ${packagePipelineName}' - envsubst "${commonEnvSubstList} ${specificEnvSubstList}" < "${hangarPath}/${templatesPath}/${yamlFile}.template" > "${localDirectory}/${pipelinePath}/${yamlFile}" - - # Check if an extra artifact to store is supplied. - if test -n "$artifactPath" - then - # Add the extra step to the YAML. - if [ "$provider" == "azure-devops" ] - then - cat "${hangarPath}/${commonTemplatesPath}/store-extra-path.yml" >> "${localDirectory}/${pipelinePath}/${yamlFile}" - elif [ "$provider" == "github" ] - then - # (in case of -a flag set) Here we replace a comment inside the template file by a step in Github Actions to upload an artifact - storeExtraPathContent="\n - name: Publish Additional Output Artifact\n uses: actions\/upload-artifact@v3\n with:\n name: additional-pipeline-output\n path: \"\${{ env.artifactPath }}\"" - sed -i "s/# mark to insert step for additonal artifact #/$storeExtraPathContent\n/" "${localDirectory}/${pipelinePath}/${yamlFile}" - elif [ "$provider" == "gcloud" ] - then - # (in case of -a flag set) We first check if an artifact is already uploaded in this template, - # to know if we need to add an artifact to the list of uploaded artifact because there is/are already artifact(s) uploaded, - # or if we need to add the artifact section because there is not artifact uploaded yet - grep -e "^artifacts:" "${localDirectory}/${pipelinePath}/${yamlFile}" > /dev/null && storeExtraPathContent=" - additional_output_\${BUILD_ID}.tar" || storeExtraPathContent="\nartifacts:\n objects:\n location: gs:\/\/\${PROJECT_ID}_cloudbuild\/pipelinesArtifacts\/${pipelineName}\/commit_\${SHORT_SHA}\n paths:\n - additional_output_\${BUILD_ID}.tar" - # We replace the comment used to know where to add the lines defined is the line above - sed -i "s/# mark to insert entry in artifact for additonal artifact #/$storeExtraPathContent\n/" "${localDirectory}/${pipelinePath}/${yamlFile}" - # Then we add a step to the pipeline to create a tar of the additional artiact, we create a tar for 2 reasons: - # - Keep the permissions of the files as they are - # - In Gcloud, if you try to upload a folder and not a file, it fails - stepCreateTarArtifact="- name: \${_DOCKER_IMAGE_STEPS}\n args: ['tar', '-cf', 'additional_output_\${BUILD_ID}.tar', '\${_ARTIFACT_PATH}']" - sed -i "s/# mark to insert step for additonal artifact #/$stepCreateTarArtifact\n/" "${localDirectory}/${pipelinePath}/${yamlFile}" - fi - else - if [ "$provider" == "github" ] || [ "$provider" == "gcloud" ] - then - # If no -a flag set, we just remove the mark used to place the additional entries and step in the yaml - sed -i '/# mark to insert step for additonal artifact #/d' "${localDirectory}/${pipelinePath}/${yamlFile}" - sed -i '/# mark to insert entry in artifact for additonal artifact #/d' "${localDirectory}/${pipelinePath}/${yamlFile}" - fi - fi -} - -function copyCommonScript { - echo -e "${green}Copying the script(s) common to any $pipeline_type files into your directory..." - echo -ne ${white} - - ! (ls "${hangarPath}/${commonTemplatesPath}"/*.sh) &> /dev/null || cp "${hangarPath}/${commonTemplatesPath}"/*.sh "${localDirectory}/${scriptFilePath}" -} - -function commitCommonFiles { - echo -e "${green}Commiting and pushing into Git remote..." - echo -ne ${white} - - # Move into the project's directory and pushing the template into the Azure DevOps repository. - cd "${localDirectory}" - - # Add the YAML files. - git add "$pipelinePath" -f - - # Git commit and push it into the repository. - # changing all files to be executable - find "$pipelinePath" -type f -name '*.sh' -exec git update-index --chmod=+x {} \; - - git commit -m "Adding the source YAML" - git push -u origin ${sourceBranch} -} - -function setTargetDirectory { - case $language in - node | angular) targetDirectory="./" ;; - quarkus*) targetDirectory="./target/" ;; - *) echo -e "${red}Error: Specified language '${language}' is not supported." >&2; exit 1 - esac -} +#!/bin/bash +function help { + echo "" + echo "Generates a $pipeline_type on $provider based on the given definition." + echo "" + echo "Common flags:" + echo " -c, --config-file [Required] Configuration file containing $pipeline_type definition." + echo " -n, --pipeline-name [Required] Name that will be set to the $pipeline_type." + echo " -d, --local-directory [Required] Local directory of your project." + echo " -a, --artifact-path Path to be persisted as an artifact after $pipeline_type execution, e.g. where the application stores logs or any other blob on runtime." + echo " -b, --target-branch Name of the branch to which the Pull Request will target. PR is not created if the flag is not provided." + [[ "$provider" != "gcloud" ]] && echo " -w Open the Pull Request on the web browser if it cannot be automatically merged. Requires -b flag." + echo "" + echo "Build $pipeline_type flags:" + echo " -l, --language [Required] Language or framework of the project." + echo " -t, --target-directory Target directory of build process. Takes precedence over the language/framework default one." + echo "" + echo "Test $pipeline_type flags:" + echo " -l, --language [Required] Language or framework of the project." + [ "$provider" == "azure-devops" ] && echo " --build-pipeline-name [Required] Build $pipeline_type name." + echo "" + echo "Quality $pipeline_type flags:" + echo " -l, --language [Required] Language or framework of the project." + echo " --sonar-url [Required] Sonarqube URL." + echo " --sonar-token [Required] Sonarqube token." + [ "$provider" == "azure-devops" ] && echo " --build-pipeline-name [Required] Build $pipeline_type name." + [ "$provider" == "azure-devops" ] && echo " --test-pipeline-name [Required] Test $pipeline_type name." + echo "" + [ "$provider" == "github" ] && echo "" + [ "$provider" == "github" ] && echo "CI pipeline flags" + [ "$provider" == "github" ] && echo " --build-pipeline-name [Required] Name of the job calling the build $pipeline_type." + [ "$provider" == "github" ] && echo " --test-pipeline-name Name of the job calling the test $pipeline_type." + [ "$provider" == "github" ] && echo " --quality-pipeline-name Name of the job calling the quality $pipeline_type." + [ "$provider" == "github" ] && echo "" + echo "Package pipeline flags:" + echo " -l, --language [Required, if dockerfile not set] Language or framework of the project." + echo " --dockerfile [Required, if language not set] Path from the root of the project to its Dockerfile. Takes precedence over the language/framework default one." + [ "$provider" == "azure-devops" ] && echo " --build-pipeline-name [Required] Build $pipeline_type name." + [ "$provider" == "azure-devops" ] && echo " --quality-pipeline-name [Required] Quality $pipeline_type name." + [ "$provider" == "github" ] && echo " --ci-pipeline-name [Required] CI $pipeline_type name." + echo " -i, --image-name [Required] Name (excluding tag) for the generated container image." + echo " -u, --registry-user [Required, unless AWS] Container registry login user." + echo " -p, --registry-password [Required, unless AWS] Container registry login password." + echo " --aws-access-key [Required, if AWS] AWS account access key ID. Takes precedence over registry credentials." + echo " --aws-secret-access-key [Required, if AWS] AWS account secret access key." + echo " --aws-region [Required, if AWS] AWS region for ECR." + echo "" + echo "Library package $pipeline_type flags:" + echo " -l, --language [Required] Language or framework of the project." + echo "" + echo "Azure AKS provisioning $pipeline_type flags:" + echo " --resource-group [Required] Name of the resource group for the cluster." + echo " --storage-account [Required] Name of the storage account for the cluster." + echo " --storage-container [Required] Name of the storage container where the Terraform state of the cluster will be stored." + echo " --rancher Install Rancher to manage the cluster." + echo "" + echo "AWS EKS provisioning $pipeline_type flags:" + echo " --cluster-name [Required] Name for the cluster." + echo " --s3-bucket [Required] Name of the S3 bucket where the Terraform state of the cluster will be stored." + echo " --s3-key-path [Required] Path within the S3 bucket where the Terraform state of the cluster will be stored." + echo " --aws-access-key [Required, on first run] AWS account access key ID." + echo " --aws-secret-access-key [Required, on first run] AWS account secret access key." + echo " --aws-region [Required, on first run] AWS region for provisioning resources." + + echo " --rancher Install Rancher to manage the cluster." + echo "" + echo "Deploy $pipeline_type flags:" + echo "" + echo " --package-pipeline-name [Required] Package $pipeline_type name." + echo " --env-provision-pipeline-name [Required] Environment provisioning $pipeline_type name." + echo " --k8s-provider [Required] Kubernetes cluster provider name. Accepted values: EKS, AKS." + echo " --k8s-namespace [Required] Kubernetes namespace where the application will be deployed." + echo " --k8s-deploy-files-path [Required] Path from the root of the project to the YAML manifests directory." + echo " --k8s-image-pull-secret-name Name for the generated secret containing registry credentials. Required when using a private registry to host images." + echo "" + + exit +} + +function validateRegistryLoginCredentials { + # if the user chose to push to a registry and the user has not already given a password + # then prompt the user + if [ -v dockerUser ] && [ ! -v dockerPassword ] + then + read -rsp "Please enter Docker registry password..." dockerPassword + fi + + if [ -v awsRegion ] && [ -v awsAccessKey ] && [ ! -v awsSecretAccessKey ] + then + read -rsp "Please enter AWS secret access key..." awsSecretAccessKey + fi +} + +function ensurePathFormat { + currentDirectory=$(pwd) + + # When necessary, converts a relative path into an absolute path, and a Windows-style path (e.g. "C:\Users" or C:/Users) into a + # Unix-style path using forward slashes (e.g. "/c/Users"). + localDirectory=${localDirectory//'\'/"/"} + cd "${localDirectory}" || { echo -e "${red}Error: Local directory '${localDirectory}' does not exist. Check provided path (missing quotes?)."; exit 1; } + localDirectory=$(pwd) + + # Return to initial directory + cd "$currentDirectory" +} + +function importConfigFile { + # Import config file. + source $configFile + IFS=, read -ra flags <<< "$mandatoryFlags" + + # Check if the config file was supplied. + if test -z "$configFile" + then + echo -e "${red}Error: $pipeline_type definition configuration file not specified." >&2 + exit 2 + fi + + # Check if the required flags in the config file have been activated. + for flag in "${flags[@]}" + do + if test -z $flag + then + echo -e "${red}Error: Missing parameters, some flags are mandatory." >&2 + echo -e "${red}Use -h or --help flag to display help." >&2 + exit 2 + fi + done +} + +function checkInstallations { + # Check if Git is installed + if ! [ -x "$(command -v git)" ]; then + echo -e "${red}Error: Git is not installed." >&2 + exit 127 + fi + + # Check if the CLI tool for the provider is installed + if ([ "$provider" == "github" ] && ! [ -x "$(command -v gh)" ]); then + echo -e "${red}Error: Github CLI is not installed." >&2 + exit 127 + elif ([ "$provider" == "azure-devops" ] && ! [ -x "$(command -v az)" ]); then + echo -e "${red}Error: Azure CLI is not installed." >&2 + exit 127 + fi + + # Check if Python is installed + if ! [ -x "$(command -v python)" ]; then + echo -e "${red}Error: Python is not installed." >&2 + exit 127 + fi +} + +function createNewBranch { + echo -e "${green}Creating the new branch: ${sourceBranch}..." + echo -ne ${white} + + # Create the new branch. + cd "${localDirectory}" + git checkout -b ${sourceBranch} +} + +function copyYAMLFile { + echo -e "${green}Copying the corresponding files into your directory..." + echo -ne ${white} + + # Create .pipelines and scripts if they do not exist. + mkdir -p "${localDirectory}/$scriptFilePath" + + # Generate pipeline YAML from template and put it in the repository. + # We cannot use a variable in the definition of resource in the pipeline so we have to use a placeholder to replace it with the value we need + commonEnvSubstList='${buildPipelineName} ${testPipelineName} ${qualityPipelineName} ${pipelineName} ${ciPipelineName} ${packagePipelineName}' + envsubst "${commonEnvSubstList} ${specificEnvSubstList}" < "${hangarPath}/${templatesPath}/${yamlFile}.template" > "${localDirectory}/${pipelinePath}/${yamlFile}" + + # Check if an extra artifact to store is supplied. + if test -n "$artifactPath" + then + # Add the extra step to the YAML. + if [ "$provider" == "azure-devops" ] + then + cat "${hangarPath}/${commonTemplatesPath}/store-extra-path.yml" >> "${localDirectory}/${pipelinePath}/${yamlFile}" + elif [ "$provider" == "github" ] + then + # (in case of -a flag set) Here we replace a comment inside the template file by a step in Github Actions to upload an artifact + storeExtraPathContent="\n - name: Publish Additional Output Artifact\n uses: actions\/upload-artifact@v3\n with:\n name: additional-pipeline-output\n path: \"\${{ env.artifactPath }}\"" + sed -i "s/# mark to insert step for additonal artifact #/$storeExtraPathContent\n/" "${localDirectory}/${pipelinePath}/${yamlFile}" + elif [ "$provider" == "gcloud" ] + then + # (in case of -a flag set) We first check if an artifact is already uploaded in this template, + # to know if we need to add an artifact to the list of uploaded artifact because there is/are already artifact(s) uploaded, + # or if we need to add the artifact section because there is not artifact uploaded yet + grep -e "^artifacts:" "${localDirectory}/${pipelinePath}/${yamlFile}" > /dev/null && storeExtraPathContent=" - additional_output_\${BUILD_ID}.tar" || storeExtraPathContent="\nartifacts:\n objects:\n location: gs:\/\/\${PROJECT_ID}_cloudbuild\/pipelinesArtifacts\/${pipelineName}\/commit_\${SHORT_SHA}\n paths:\n - additional_output_\${BUILD_ID}.tar" + # We replace the comment used to know where to add the lines defined is the line above + sed -i "s/# mark to insert entry in artifact for additonal artifact #/$storeExtraPathContent\n/" "${localDirectory}/${pipelinePath}/${yamlFile}" + # Then we add a step to the pipeline to create a tar of the additional artiact, we create a tar for 2 reasons: + # - Keep the permissions of the files as they are + # - In Gcloud, if you try to upload a folder and not a file, it fails + stepCreateTarArtifact="- name: \${_DOCKER_IMAGE_STEPS}\n args: ['tar', '-cf', 'additional_output_\${BUILD_ID}.tar', '\${_ARTIFACT_PATH}']" + sed -i "s/# mark to insert step for additonal artifact #/$stepCreateTarArtifact\n/" "${localDirectory}/${pipelinePath}/${yamlFile}" + fi + else + if [ "$provider" == "github" ] || [ "$provider" == "gcloud" ] + then + # If no -a flag set, we just remove the mark used to place the additional entries and step in the yaml + sed -i '/# mark to insert step for additonal artifact #/d' "${localDirectory}/${pipelinePath}/${yamlFile}" + sed -i '/# mark to insert entry in artifact for additonal artifact #/d' "${localDirectory}/${pipelinePath}/${yamlFile}" + fi + fi +} + +function copyCommonScript { + echo -e "${green}Copying the script(s) common to any $pipeline_type files into your directory..." + echo -ne ${white} + + ! (ls "${hangarPath}/${commonTemplatesPath}"/*.sh) &> /dev/null || cp "${hangarPath}/${commonTemplatesPath}"/*.sh "${localDirectory}/${scriptFilePath}" +} + +function commitCommonFiles { + echo -e "${green}Commiting and pushing into Git remote..." + echo -ne ${white} + + # Move into the project's directory and pushing the template into the Azure DevOps repository. + cd "${localDirectory}" + + # Add the YAML files. + git add "$pipelinePath" -f + + # Git commit and push it into the repository. + # changing all files to be executable + find "$pipelinePath" -type f -name '*.sh' -exec git update-index --chmod=+x {} \; + + git commit -m "Adding the source YAML" + git push -u origin ${sourceBranch} +} + +function setTargetDirectory { + case $language in + node | angular) targetDirectory="./" ;; + quarkus*) targetDirectory="./target/" ;; + *) echo -e "${red}Error: Specified language '${language}' is not supported." >&2; exit 1 + esac +} diff --git a/scripts/pipelines/github/pipeline_generator.sh b/scripts/pipelines/github/pipeline_generator.sh index 556f6f248..569e5d6d1 100644 --- a/scripts/pipelines/github/pipeline_generator.sh +++ b/scripts/pipelines/github/pipeline_generator.sh @@ -1,7 +1,6 @@ #!/bin/bash set -e -FLAGS=$(getopt -a --options c:n:d:a:b:l:i:u:p:hw --long "config-file:,pipeline-name:,local-directory:,artifact-path:,target-branch:,language:,build-pipeline-name:,sonar-url:,sonar-token:,image-name:,registry-user:,registry-password:,resource-group:,storage-account:,storage-container:,cluster-name:,s3-bucket:,s3-key-path:,quality-pipeline-name:,dockerfile:,test-pipeline-name:,aws-access-key:,aws-secret-access-key:,aws-region:,ci-pipeline-name:,help" -- "$@") - +FLAGS=$(getopt -a --options c:n:d:a:b:l:i:u:p:hw --long "config-file:,pipeline-name:,local-directory:,artifact-path:,target-branch:,language:,build-pipeline-name:,sonar-url:,sonar-token:,image-name:,registry-user:,registry-password:,resource-group:,storage-account:,storage-container:,cluster-name:,s3-bucket:,s3-key-path:,quality-pipeline-name:,dockerfile:,test-pipeline-name:,aws-access-key:,aws-secret-access-key:,aws-region:,ci-pipeline-name:,help,rancher" -- "$@") eval set -- "$FLAGS" while true; do case "$1" in @@ -20,6 +19,7 @@ while true; do --resource-group) resourceGroupName=$2; shift 2;; --storage-account) storageAccountName=$2; shift 2;; --storage-container) storageContainerName=$2; shift 2;; + --rancher) installRancher="true"; shift 1;; --cluster-name) clusterName=$2; shift 2;; --s3-bucket) s3Bucket=$2; shift 2;; --s3-key-path) s3KeyPath=$2; shift 2;; diff --git a/scripts/pipelines/github/templates/aks/aks-pipeline.cfg b/scripts/pipelines/github/templates/aks/aks-pipeline.cfg new file mode 100644 index 000000000..e466c2dd5 --- /dev/null +++ b/scripts/pipelines/github/templates/aks/aks-pipeline.cfg @@ -0,0 +1,60 @@ +# Mandatory flags. +mandatoryFalgs="$pipelineName,$localDirectory,$clusterName,$resourceGroupName,$storageAccountName,$storageContainerName," +# Path to the templates. +templatesPath="scripts/pipelines/github/templates/aks" +#Path to common kubernetes templates. +commonKubernetesPath="scripts/pipelines/github/templates/common/kubernetes" +# YAML file name. +yamlFile="aks-provisioning.yml" +# Source branch. +sourceBranch="feature/aks-provisioning" +# Path to terraform templates. +terraformTemplatesPath="scripts/environment-provisioning/azure/aks" +# Path to terraform scripts. +terraformPath=".terraform/aks" +# Default cluster operation. +operation="create" +# Install Rancher on AKS cluster. +if test -z $installRancher +then + installRancher=false +fi + +# Function that copies the necessary scripts into the directory. +function copyScript { + # Create .terraform/aks folder if it does not exist. + mkdir -p "${localDirectory}/${terraformPath}" + + # Copy the terraform files. + cd "${hangarPath}/${terraformTemplatesPath}" + cp * "${localDirectory}/${terraformPath}" + + # Copy the script for the DNS name into the directory. + cp "${hangarPath}/${templatesPath}/obtain-dns.sh" "${localDirectory}/${scriptFilePath}/obtain-dns.sh" + + # Copy the common files for kubernetes + cp "${hangarPath}/${commonKubernetesPath}"/*.sh "${localDirectory}/${scriptFilePath}" +} + +function commitFiles { + # Add the terraform files. + git add .terraform -f + + # Changing all files to be executable. + find .terraform -type f -name '*.sh' -exec git update-index --chmod=+x {} \; + + # Git commit and push it into the repository. + git commit -m "Adding the terraform files" + git push -u origin ${sourceBranch} +} + +# Function that adds the variables to be used in the pipeline. +function addPipelineVariables { + export clusterName + export resourceGroupName + export storageAccountName + export storageContainerName + export installRancher + export operation + specificEnvSubstList='${clusterName} ${resourceGroupName} ${storageAccountName} ${storageContainerName} ${installRancher} ${operation}' +} diff --git a/scripts/pipelines/github/templates/aks/aks-provisioning.yml.template b/scripts/pipelines/github/templates/aks/aks-provisioning.yml.template new file mode 100644 index 000000000..43b97e230 --- /dev/null +++ b/scripts/pipelines/github/templates/aks/aks-provisioning.yml.template @@ -0,0 +1,105 @@ +name: $pipelineName + +on: + workflow_dispatch: + inputs: + cluster_name: + description: 'Name for the cluster.' + required: true + type: string + default: $clusterName + install_rancher: + description: 'Installs Rancher on AKS when set to true.' + required: false + type: string + default: $installRancher + cluster_operation: + description: 'Operation to be performed on AKS Cluster. create or destroy' + required: false + type: string + default: $operation + +env: + terraformWorkingDirectory: "${{ github.workspace }}/.terraform/aks" + +jobs: + terraform-actions: + name: Provisioning AKS + runs-on: ubuntu-latest + steps: + - name: login cli + run: | + az login -u "${{ secrets.AZURE_USERNAME }}" -p "${{ secrets.AZURE_PASSWORD }}" + + - name: Checkout Repository + uses: actions/checkout@master + + - name: Terraform - Init + id: init + run: terraform init -backend-config=resource_group_name=$resourceGroupName -backend-config=storage_account_name=$storageAccountName -backend-config=container_name=$storageContainerName -backend-config=key=terraform.tfstate + working-directory: ${{ env.terraformWorkingDirectory }} + continue-on-error: false + + - name: Terraform - apply + if: ${{ github.event.inputs.cluster_operation == 'create' }} + id: apply + working-directory: ${{ env.terraformWorkingDirectory }} + run: terraform apply -var 'cluster_name=${{ github.event.inputs.cluster_name }}' --auto-approve + continue-on-error: false + + - name: Terraform Destroy + if: ${{ github.event.inputs.cluster_operation == 'destroy' }} + id: destroy + working-directory: ${{ env.terraformWorkingDirectory }} + run: terraform apply -destroy -var 'cluster_name=${{ github.event.inputs.cluster_name }}' --auto-approve + continue-on-error: false + + - name: Install nginx-ingress controller + if: ${{ github.event.inputs.cluster_operation == 'create' }} + id: installnginx + env: + KUBECONFIG: "${{ env.terraformWorkingDirectory }}/kubeconfig" + run: .github/workflows/scripts/install-nginx-ingress.sh + + - name: Obtain DNS name + if: ${{ github.event.inputs.cluster_operation == 'create' }} + id: obtaindns + env: + KUBECONFIG: "${{ env.terraformWorkingDirectory }}/kubeconfig" + run: | + .github/workflows/scripts/obtain-dns.sh '${{ github.event.inputs.cluster_name }}' + dnsname="${{ github.event.inputs.cluster_name }}.westeurope.cloudapp.azure.com" + echo "::set-output name=dnsname::$dnsname" + + - name: Publish DNS name + if: ${{ github.event.inputs.cluster_operation == 'create' }} + id: publishdns + run: | + cd .github + mkdir -p vars + cd vars + touch "deployment-env-variables.env" + echo "aks_dns_name=${{ steps.obtaindns.outputs.dnsname }}" >> deployment-env-variables.env + + - name: Publish kubeconfig + uses: actions/upload-artifact@v3.1.0 + if: ${{ github.event.inputs.cluster_operation == 'create' }} + with: + name: kubeconfig + path: ${{ env.terraformWorkingDirectory }}/kubeconfig + if-no-files-found: error + + - name: Push AKS variables + uses: EndBug/add-and-commit@v7 + if: ${{ github.event.inputs.cluster_operation == 'create' }} + with: + default_author: github_actions + cwd: '.github/vars/' + + - name: Install Rancher + if: ${{ github.event.inputs.install_rancher == 'true' && github.event.inputs.cluster_operation == 'create'}} + id: installrancher + env: + KUBECONFIG: "${{ env.terraformWorkingDirectory }}/kubeconfig" + run: .github/workflows/scripts/install-rancher.sh ${{ steps.obtaindns.outputs.dnsname }} + diff --git a/scripts/pipelines/github/templates/aks/obtain-dns.sh b/scripts/pipelines/github/templates/aks/obtain-dns.sh new file mode 100644 index 000000000..c6f431573 --- /dev/null +++ b/scripts/pipelines/github/templates/aks/obtain-dns.sh @@ -0,0 +1,21 @@ +#!/bin/bash +ip="$(kubectl get svc nginx-ingress-nginx-ingress-controller --namespace nginx-ingress -o jsonpath='{.status.loadBalancer.ingress[0].ip}')" + +while test -z "$ip" +do + sleep 5s + ip="$(kubectl get svc nginx-ingress-nginx-ingress-controller --namespace nginx-ingress -o jsonpath='{.status.loadBalancer.ingress[0].ip}')" +done + +# Obtain the AKS cluster name +dnsname=$1 + +ipname=$(az network public-ip list --query "[?ipAddress!=null]|[?contains(ipAddress, '$ip')].[name]" --output tsv) + +iprg=$(az network public-ip list --query "[?ipAddress!=null]|[?contains(ipAddress, '$ip')].[resourceGroup]" --output tsv) + +az network public-ip update --resource-group "$iprg" --name "$ipname" --dns-name "$dnsname" + +dns="$(az network public-ip show --resource-group "$iprg" --name "$ipname" --query "[dnsSettings.fqdn]" --output tsv)" + +echo "$dns" diff --git a/scripts/pipelines/github/templates/common/kubernetes/install-nginx-ingress.sh b/scripts/pipelines/github/templates/common/kubernetes/install-nginx-ingress.sh new file mode 100644 index 000000000..0914cf122 --- /dev/null +++ b/scripts/pipelines/github/templates/common/kubernetes/install-nginx-ingress.sh @@ -0,0 +1,4 @@ +#!/bin/bash +helm repo add bitnami https://charts.bitnami.com/bitnami +helm repo update +helm install nginx-ingress bitnami/nginx-ingress-controller --set ingressClassResource.default=true --set containerSecurityContext.allowPrivilegeEscalation=false --namespace nginx-ingress --create-namespace \ No newline at end of file diff --git a/scripts/pipelines/github/templates/common/kubernetes/install-rancher.sh b/scripts/pipelines/github/templates/common/kubernetes/install-rancher.sh new file mode 100644 index 000000000..66c61ac78 --- /dev/null +++ b/scripts/pipelines/github/templates/common/kubernetes/install-rancher.sh @@ -0,0 +1,7 @@ +#!/bin/bash +helm repo add rancher-latest https://releases.rancher.com/server-charts/latest +helm repo add jetstack https://charts.jetstack.io +helm repo add +kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.5.1/cert-manager.crds.yaml +helm install cert-manager "jetstack/cert-manager" --namespace cert-manager --create-namespace --version v1.5.1 +helm install rancher "rancher-latest/rancher" --namespace cattle-system --create-namespace --set hostname="$1" \ No newline at end of file