diff --git a/labs/day-2/05-modernize-data/README.md b/labs/day-2/05-modernize-data/README.md index 8693582..f4aa670 100644 --- a/labs/day-2/05-modernize-data/README.md +++ b/labs/day-2/05-modernize-data/README.md @@ -12,32 +12,27 @@ Learning objectives Students will learn to: -* Gather information about a SQL server database -* Prepare and assess a SQL Server database for migration to Azure. -* Learn about diffenent authentication mechanisms in SQL server -* Learn about different types of backups and migration techniques -* Run an assessment of the source SQL Server database, analyze the report -* Use the managed database target selected during Lab 04. -* Configure private connectivity to the Azure SQL Database from both the Azure VM and also target PaaS services -* Create Azure Data Migration Service (DMS) and configure to run the "Self hosted Integration Runtime" on the Azure VM. -* Run DMS online migration to SQL MI and verify migration. -* Configure the application with the database FQDN exported by Lab 04. -* Document differences between Azre SQLDB and Azire SQL MI and lessons learned. +* Gather information about a SQL server database. +* Learn about diffenent authentication mechanisms in SQL server. +* Learn about different types of backups and migration techniques. +* Run an assessment of the source SQL Server database, analyze the report. +* Create Azure Data Migration Service (DMS) +* Run DMS online migration to SQL MI and verify migration. ## Challenge 1 — Validate the source database ### Student tasks 1. Using SQL Server Management Studio, connect to the source database using the instructor-approved method. -2. Confirm that SQL Server services are running on the source database VM - windows service named "SQL Server (MSSQLSERVER)" -3. Connect to the local SQL Server with SQL Server Management Studio (SSMS) using "sa" SQL login given to you -4. Record the SQL Server version,edition - right click on the server and type "new query". Execute the following SQL +2. Confirm that SQL Server services are running on the source database VM - windows service named "SQL Server (MSSQLSERVER)". +3. Connect to the local SQL Server with SQL Server Management Studio (SSMS) using "sa" SQL login given to you. +4. Record the SQL Server version,edition - right click on the server and type "new query". Execute the following SQL. ```sql Use master; select @@version ; ``` -5. Right click on database eshop and click on peroperties to determine database size, collation, disk file name and size of database files and "recovery mode", like +5. Right click on database eshop and click on peroperties to determine database size, collation, disk file name and size of database files and "recovery mode", as shown. [![this](./images/Challenge_1_db_properties.png)](./images/Challenge_1_db_properties.png) @@ -46,7 +41,7 @@ select @@version ; 8. You connected to the SQL server using SQL authentication. What are other ways to authenticate to a SQL server? 9. (Research on this) What is a logical and physical backup of SQL database ? 10. How is recovery mode and logical and/or physical backup related ? -11. Put the database into full recovery mode in SSMS running this query +11. Put the database into full recovery mode in SSMS running this query. ```sql ALTER DATABASE @@ -54,7 +49,7 @@ ALTER DATABASE SET RECOVERY full ; ``` -12. Verify that the eshop database was placed in full recovery mode +12. Verify that the eshop database was placed in full recovery mode. ```sql SELECT @@ -75,7 +70,7 @@ DBCC CHECKDB (N'eShop') ; * SSMS connects to the source instance. * The student records the source version, edition, size etc. -* You can explain at least four different authentication mechanisms and show at least two ways to connnect +* You can explain at least four different authentication mechanisms and show at least two ways to connnect. * You can explain recovery model and different types of backups. ## Challenge 2 — Pre-migration assessment of the source database @@ -84,36 +79,50 @@ SSMS 22 is the latest version of Microsoft’s SQL Server Management Studio, a 6 ![SSMS](./images/ssms_22.png) -The Azure SQL Managed Instance configured in this lab is configured to authenticate using Entra only. Notice that there are several ways you can authenticate to SQL server using Entra. In this lab, you must select authentication method Entra with Password. The public endpoint is used to connect from the virtual lab environment therefore when connecting using SSMS, port 3342 needs to be used. +### Student tasks + + +1. Using SSMS, connect to the SQL Server 2016 that resides on a VM in this lab environment. The connection to "on-premises" SQL server is preconfigured in the lab. Right click on the server and choose "Migrate SQL Server". + + ![SSMS](./images/Challenge_2_assessment_launch_1.png) + +2. #### Do not Migrate or Upgrade the database. + +Run a "Migration rediness assessment". An html file will open in your browser once the assessment completes. This is the report. + + ![SSMS](./images/Challenge_2_assessment_launch_2.png) -### Validate firewall access to SQL MI before begining. +3. Investigate the report. Find out compatibility issues with different SQL targets. - Validate from the portal, that the NSG for the VNet that Azure SQL Managed Instance is using is allowing inbount access to port 3342. If not, add an inbound rule for port 3342. From the overview page of SQL MI, click on virtual network/subnet. On the subnet page, find the Network Security Group name. Pull up that NSG by name and add an imbound port rule like this: +![Assessment](./images/Challenge_2_assessment_full_report.png) + +4. Prepare to connect to the target SQL Managed Instance + +The Azure SQL Managed Instance configured in this lab is configured to authenticate using Entra only. + +Notice that there are several ways you can authenticate to SQL server using Entra. In this lab, you must select authentication method Entra with Password. The public endpoint is used to connect from the virtual lab environment therefore when connecting using SSMS, port 3342 needs to be used. + +#### Validate firewall access to SQL MI before begining. + + Validate from the portal, that the NSG for the VNet that Azure SQL Managed Instance is allowing inbount access to port 3342. If not, add an inbound rule for port 3342. From the overview page of SQL MI, click on virtual network/subnet. On the subnet page, find the Network Security Group name. Pull up that NSG by name and add an imbound port rule like this: ![NSG1](./images/NSG1.png) -2- Select the right option in SSMS when loging in. To find out the Entra ID to use for login, navigate from the portal to the deployed Azure SQL Managed Instance and go to Microsoft Entra ID on the left under Security. +5. Select the right user login to connect to SQL MI. To find out the Entra ID to use for login, navigate from the portal to the deployed Azure SQL Managed Instance and go to Microsoft Entra ID on the left under Security. +*Note* This will be the same user as your azure login . ![SQLMI_ENTRA](./images/sqlmi_entra.png) -To retrieve the public endpoint FQDN and port number to use as part of the connection information, navigate to the Azure SQL Managed Instance in the portal. Go to *Connection strings* an lookup the public endpoint examples. +6. To retrieve the public endpoint FQDN and port number to use as part of the connection information, navigate to the Azure SQL Managed Instance in the portal. Go to *Connection strings* an lookup the public endpoint examples. ![AZSQLMIPE](./images/AzSQLMIPE.png) -Use that Entra ID to login using SSMS. +Use that Entra ID and password to login using SSMS. Under "Databases" there is no database listed now. ![SSMS_LOGIN](./images/ssms_login.png) -## Student tasks -1. Using SSMS, connect to the SQL Server 2016 that is is resides on a VM in this lab environment. The connection to "on-premises" SQL server is preconfigured in the lab. Right click on the server and choose "Migrate SQL Server" - ![SSMS](./images/Challenge_2_assessment_launch_1.png) -2. Do not Migrate or Upgrade the database. Run a "Migration rediness assessment". An html file will open in your browser once the assessment completes. This is the report. - ![SSMS](./images/Challenge_2_assessment_launch_2.png) -3. Investigate the report. Find out compatibility issues with different SQL targets ![Assessment](./images/Challenge_2_assessment_full_report.png) -4. Connect to the target SQL MI using SSMS. Select the right option in SSMS when loging in. To find out the Entra ID to use for login, navigate from the portal to the deployed Azure SQL Managed Instance and go to Microsoft Entra ID on the left under settings. In this lab, this Entra ID is same as your Azure login - also shared under "Resources" section on the top. - -![SQLMI_ENTRA](./images/sqlmi_entra.png) +7. Connect to the target SQL MI using SSMS. Notice that there are no databaes there. ## Success criteria @@ -133,28 +142,28 @@ In this part of the lab you will create the necesary Azure resources to perform - Deploy an Azure Database Migration Service (DMS) to migrate the database. -## Student tasks +### Student tasks -### 1. Resource provider +#### 1. Resource provider From the Azure portal, go to *Subscriptions*. Expand on settings. Click on *Ressource Providers* and confirm that *Microsoft.DataMigration* is registered. If it is not, then register it. ![ResourceProvider](./images/dms_resource_provider.png) -### 2. Azure SQL MI System Assigned Managed Identity +#### 2. Azure SQL MI System Assigned Managed Identity Locate the pre-deployed Azure SQL Managed instance in your lab subscription. Go to the *Identity* blade under Security and confirm that *System Assigned Managed Identity* is enabled. If it is not then enable it. ![SQLMI_SAMI](./images/SQLMI_SAMI.png) -### 3. Storage Account +#### 3. Storage Account Provision an Azure Storage Account in the same region as your SQL MI and create a Blob container in it to store source database backup. ![Storage1](./images/Storage_1.png) ![Storage3](./images/Storage_3.png) -Once the storage account is created, you will need to grant to your current Azure user the permission to manage Blob containers. Do this via IAM. Assign the *Storage Blob Data Owner* role to your own Entra ID. +Once the storage account is created, you will need to grant to your current Azure user the permission to manage Blob containers. Do this via IAM. Assign the *Storage Blob Data Owner* (or Contributor) role to your own Entra ID. ![Storage5](./images/Storage_5.png) ![Storage6](./images/Storage_6.png) @@ -170,7 +179,7 @@ Create a Blob container in the storage account and a folder within the container ![Storage11](./images/Storage_11.png) ![Storage12](./images/Storage_12.png) -### 4. Database Migration Service (DMS) +#### 4. Database Migration Service (DMS) In the same region where Azure SQL MI is deployed in the lab subscription, deploy Azure Database Migration Services (DMS). @@ -178,19 +187,19 @@ Create a Blob container in the storage account and a folder within the container ## Success criteria -- Resource provider is registered for data migrations -- SQL MI configured for System Assigned Managed Identity -- Storage account is created in the same region as MI, with a Blob container in it -- DMS is deployed in the same region as SQL MI +- Resource provider is registered for data migrations. +- SQL MI configured for System Assigned Managed Identity. +- Storage account is created in the same region as MI, with a Blob container in it. +- DMS is deployed in the same region as SQL MI. ## Challenge 4 — Online Migraton to SQL Managed Instance In this challenge you will perform database backups to the Azure Storage account provisioned earlier. You will then use DMS to perform an *Online* migration. To perform and online migration, DMS restores backups then takes advantage of the *Log Replay Service* (LRS) to replay transaction logs and complete the migration. -## Student tasks +### Student tasks -### 1. Backup the database to Azure Storage using SSMS 22 +#### 1. Backup the database to Azure Storage using SSMS 22 Launch SSMS from the desktop: @@ -204,7 +213,7 @@ Select *Full* for backup type and *URL* for *Back up to*. Click on *Add* to sele ![SSMS22_2](./images/SSMS22_2.png) -Click on *New Container* +Click on *New Container*. ![SSMS22_3](./images/SSMS22_3.png) @@ -212,11 +221,15 @@ Login to the lab's subscription and select the storage account and container cre ![SSMS22_4](./images/SSMS22_4.png) -Note: DMS can restore a backup stored either in the root level of a container or inside a container - not any level further below it. Complete the backup - it should take less than a minute. +Note: + + DMS can restore a backup stored either in the root level of a container or inside a container - not any level further below it. + + Complete the backup, prefix the file with "full_" so you can identify the file easily in the azure portal for the storage container. The backup should take less than a minute. ![SSMS22_5](./images/SSMS22_5.png) -Repeat the task this time taking a differential backup instead of a full backup. +Repeat the task this time taking a differential backup - with a name prefix line "diff_" . ![SSMS22_6](./images/SSMS22_6.png) @@ -229,21 +242,22 @@ Do you think it is ever possible to have a differential backup larger than full ### 2. Migrate the database online using DMS -Navigate to the Azure portal to the DMS service created earlier. Select "New Migration" +Navigate to the Azure portal to the DMS service created earlier. Select "New Migration". ![DMS_6](./images/DMS_1.png) Next, choose the migration scenario - from Sql Server to Azure SQL MI. -Choose Blob as backup location and online as backup mode. +Choose Blob as backup location and online as backup mode. ![DMS_7](./images/DMS_7.png) -Select *Blob Storage* as the location of the backup files and *Online* as the migration mode. +On the next page, configure details as shown. +Note: -Configure details as shown. - -Note: that DMS needs to configure a SQL datanase to track progress of migration for restartability. What you are entering here is information for that tracking database - not your source or target database. You need not manage this instance for this lab. + DMS needs to configure a SQL datanase to track progress of migration for restartability. + + What you are entering here is information for that tracking database - not your source or target database. You need not manage this instance for this lab. **Also note** @@ -273,125 +287,68 @@ If all is well, the full backups would have been restored. ![DMS_13](./images/DMS_13.png) -The work is not done yet. Since this is an online migration, transaction logs need to be replayed. The LRS can only be triggered via Azure CLI or PowerShell. The *datamigration* extension needs to be installed on the VM. - -*az extension add --name datamigration --upgrade* - -Once that is done, as a test to prove that transactions logs completed successfully and no data loss occured, go to the source database and add a new row to a table. Use SSMS 22 to launch a query window and run the command. - -![DMS_14](./images/DMS_14.png) +The migration is not complete yet. -Next we need to backup the transaction logs to the storage account, in the same folder/location as the database backups. Use SSMS to do this as well. Opt for *Transactions Logs* as the back up type. - -![DMS_15](./images/DMS_15.png) - -You can follow the progress of the log replay from the portal, same place as where the progress of the migration was being followed. +Since this is an online migration with minimal outage, we need to apply any incremental data that was added sunce the migration started. This means backup of transaction logs need to be restored. -![DMS_122](./images/DMS_12.png) -The transaction logs have all been played when there are no files left to restore. +In this lab, we will add some data to simulate that. -![DMS_17](./images/DMS_17.png) +Go to the source database and add a new row to a table. Use SSMS 22 to launch a query window and run the command. -Perform the cutover. Wait for it to complete. +You can enter a few more rows if you want. -![DMS_18](./images/DMS_18.png) - -Navigate to the Azure SQL Managed instance and confirm the database is there and in good state. Click on *Databases* in the left to list all databases on this managed instance. - -![DMS_19](./images/DMS_19.png) - -Go back to SSMS 22. Connect to the Azure SQL Managed Instance and eShop database using Entra. Explore the tables and make sure they are all there. Confirm the row you added above is also present. - -#### Congratulations - you have migrated to Azure SQL - -## Challenge 5 — Enable private endpoint for SQL Managed Instance - -## Student tasks - -1. Create a private endpoint for the Azure SQL logical server. Go to Azure portal - -> Security --> Networking --> Private Access -2. Configure the private endpoint in the same region where you want to connect from. -3. One the azure portal, search for your private DNS zone just created for SQL Database. Under DNS Management ---> Virtual Network Links, -add links to the Vnet where you want to connect to this SQL database from azure. -4. Ensure net connectibity using private endpoint from your source something like -```text -test-NetConnection .database.windows.net -Port 1434 +```sql +INSERT INTO + dbo.store +VALUES + ( 'Test', 'Test', 'Test', 'Test' ) ; ``` -privatelink.database.windows.net "privatelink.database.windows.net" - -1. Link the private DNS zone to the VNet. -2. Associate the private endpoint with a DNS zone group. -3. Confirm that the private endpoint connection is approved. +Next we need to backup the transaction logs to the storage account, in the same folder/location as the database backups. Use SSMS to do this as well. Opt for *Transactions Logs* as the back up type. -## Success criteria +Mote: -* The Azure SQL server name resolves to a private IP from the VM. -* TCP 1433 is reachable from the VM. -* Public network access remains disabled. +Prefix the file name with "log_" so you can identify the file easily. -## Challenge 6 — Application readiness - Entra-id authentication for app +![DMS_15](./images/DMS_15.png) -## Student tasks +You can follow the progress of the log replay from the portal, same place as where the progress of the migration was being followed. -1. Read the dedicated Lab 04 runtime identity resource ID: +Note: - ```powershell - $runtimeIdentityResourceId = gh variable get LAB06_RUNTIME_IDENTITY_RESOURCE_ID - $runtimeIdentityPrincipalId = az identity show ` - --ids $runtimeIdentityResourceId ` - --query principalId ` - --output tsv - ``` +- When you take a backup of the source database, DMS may take a minute or two to see the backup. -2. Connect to the Azure SQL `eShop` database as the configured Microsoft Entra administrator. -3. Create a container user for the Container App identity. Use a unique alias and its object ID so the database principal does not depend on Entra display-name uniqueness: +- Notice the order of restore operaion of DMS - first full, then differential, then logs - in the same order that the backup was taken. - ```sql - CREATE USER [caldova_retail_app] FROM EXTERNAL PROVIDER - WITH OBJECT_ID = ''; +![DMS_17](./images/DMS_17.png) - ALTER ROLE db_datareader ADD MEMBER [caldova_retail_app]; - ALTER ROLE db_datawriter ADD MEMBER [caldova_retail_app]; - ``` +Wait till all transaction logs have all been restored and no files left ( as seen in azure storage ) to restore. -4. Do not grant `db_owner` or `db_ddladmin`; the runtime application reads and writes data but does not own schema deployment. -5. Prepare a passwordless application connection string using the Azure SQL FQDN: +Perform the cutover. Wait for it to complete. - ```text - Server=tcp:.database.windows.net,1433;Initial Catalog=eShop;Authentication=Active Directory Managed Identity;Encrypt=True;TrustServerCertificate=False;Connection Timeout=30; - ``` +![DMS_122](./images/DMS_12.png) -6. Confirm TLS encryption and managed-identity authentication. -7. Test representative application operations. -8. Identify features that require redesign after migration. -9. Document rollback criteria and a cutover plan. -## Success criteria -The Container App identity has only the required database roles, the connection string contains no password, and the student demonstrates that application readiness requires more than successful data copy. +![DMS_18](./images/DMS_18.png) +Navigate to the Azure SQL Managed instance and confirm the database is there and in good state. Click on *Databases* in the left to list all databases on this managed instance. -Suggested class schedule +![DMS_19](./images/DMS_19.png) -| | | | -| --- | --- | --- | -| **Phase** | **Challenges** | **Suggested time** | -| Source preparation | 1–3 | 45–60 minutes | -| Target preparation | 4–7 | 60 minutes | -| DMS configuration | 8–9 | 45 minutes | -| Troubleshooting | 10–11 | 60 minutes | -| Data migration | 12 | 30–60 minutes | -| Validation and closeout | 13–15 | 60 minutes | +Go back to SSMS 22. Connect to the Azure SQL Managed Instance and eShop database using Entra. Explore the tables and make sure they are all there. Confirm the row you added above is also present. +Run this following query to confirm the new row(s) you added -### Instructor debrief questions +```sql +SELECT + * +FROM + dbo.store +WHERE + name like 'Test%' ; +``` -1. Why must compatibility assessment occur before migration? -2. What roles do Private Link and private DNS play? -3. Why should clients use the Azure SQL FQDN instead of its private IP? -4. What evidence is required before declaring the migration successful? -5. What would change for a production migration with minimal downtime? -6. Which steps should be automated for repeatable delivery? +#### Congratulations - you have migrated to Azure SQL -**Lab principle:** A migration is complete only after compatibility, connectivity, schema, data, application behavior, security, and operational readiness have all been validated with evidence.