Utilize the DeploymentPipelines-DeployAll PowerShell script and Azure DevOps to update Microsoft Fabric Deployment Pipeline stages

Reading Time: 7 minutes

In this post I want to cover how you can utilize the DeploymentPipelines-DeployAll PowerShell script and Azure DevOps to update Microsoft Fabric Deployment Pipeline stages.

By the end of this post, you will know what the changes are required to update Microsoft Fabric Deployment Pipeline stages with the DeploymentPipelines-DeployAll PowerShell script and the classic pipelines in Azure DevOps.

In addition, you will understand the concept of working with approvals in Azure DevOps better. Along the way I share plenty of links.

To see how to do this with YAML pipelines instead, please refer to my other post titled “Update Fabric Deployment Pipeline stages with DeploymentPipelines-DeployAll and YAML Pipelines“.

In addition, you can find a sample YAML pipeline in my AzureDevOps-AutomateFabricDeploymentPipeline GitHub repository. Which you can import into Azure DevOps and use as a template.

About DeploymentPipelines-DeployAll.ps1

For those who are not aware, “DeploymentPipelines-DeployAll.ps1” is one of two sample PowerShell scripts that you can download from Microsoft on their page that covers automating deployment pipelines via APIs.

You can implement this script to deploy all the items in a workspace to stages that you create in a Microsoft Fabric deployment pipeline.

I made some amendments to the script to show one way that you can get it to work within Azure DevOps. Including how to customize it to deploy to multiple deployment pipeline stages with parameters.

To manage expectations, Microsoft states in their considerations and limitations that only Power BI items are currently supported by service principals. I have tested the below method with a Data Warehouse, and it does appear to be the case.

Which means that for now you can deploy items like reports and semantic models until everything is supported by service principals. Alternatively, look to use another way to authenticate short-term.

Why use DeploymentPipelines-DeployAll PowerShell script and Azure DevOps to update Microsoft Fabric Deployment Pipeline stages?

Working with deployment pipelines within Microsoft Fabric can be a great experience. Because it provides a nice GUI-based way for you to deploy updates to multiple stages within the Microsoft Fabric environment.

However, there are scenarios that require you to manage the deployments to the different stages you created your deployment pipeline outside of the Microsoft Fabric environment.

For example, when an approval is required for a deployment to a Production workspace.

To answer this scenario, Microsoft recently introduced deployment pipelines Fabric REST APIs. Which allows you to orchestrate deployment pipelines using services such as Azure Pipelines in Azure DevOps.

Which are great if you want to work with the APIs for custom scenarios relating to your deployment pipelines outside of the Microsoft Fabric environment.

However, if you just want a simple way to deploy to multiple stages outside of Microsoft Fabric without developing too deeply with the APIs you can look to use the DeploymentPipelines-DeployAll PowerShell script from Microsoft instead.

With a couple of tweaks, you can make it work better with Azure Pipelines within Azure DevOps. By allowing you to pass through parameters. Plus, it is a quick way to introduce an approvals process.

Customizing DeploymentPipelines-DeployAll to work with Azure DevOps

Full disclosure, this particular section is the same as the “customizing DeploymentPipelines-DeployAll to work with YAML pipelines” section in my previous post on how to do the same thing with YAML pipelines.

To customize the DeploymentPipelines-DeployAll PowerShell script I first created a new repository in Azure Repos in Azure DevOps. Deselecting the option to create a README file.

Once done, I created a new folder locally to store the local clone(copy) of the Git repository and opened it in Visual Studio Code.

I then downloaded the PowerShell script from the Microsoft page that covers automating deployment pipelines via APIs into a subfolder called “scripts”.

I then made the following code changes. You can choose alternative modifications if you wish.

First of all, I commented out the static parameters, and added them as parameter values that can be entered externally. As you can see below.

# Old static parameters
# $deploymentPipelineName = "MyDP"      # The name of the deployment pipeline
# $sourceStageName = "Development"                    # The name of the source stage
# $targetStageName = "Test"                    # The name of the target stage
# $deploymentNote = "Deployment completed"                       # The deployment note (Optional)

# New ones
param(
[string]$deploymentPipelineName,
[string]$sourceStageName,
[string]$targetStageName,
[string]$deploymentNote
)

I did this so that the script would be flexible enough to handle dynamic parameters provided by the YAML pipeline.

Second change I made was to comment out the Connect-AzAccount command as below.

    # Login to Azure
    #Connect-AzAccount | Out-Null

I did this so that the authentication can be handled by the YAML Pipeline instead.

After I had done this, I saved the file and initialized the folder as a Git Repository in Visual Studio Code.

I then performed an initial commit for the Git repository and synchronized it with the new repository in Azure DevOps. I covered how to do both of these in a previous post where I covered my initial Microsoft Fabric Git integration tests for Power BI Reports.

Checking the variable groups for the classic pipelines

Once I verified that the synchronization worked, I went into Azure Pipelines in Azure DevOps and checked that the below two variable groups I had created previously were in place.

• GIDemoNS to store non-sensitive values such as stage names
• GIDemoS to store sensitive values

Even though you can enter sensitive values manually I highly recommend you connect the variable group to Azure Key Vault instead and select secrets from there.

Preparing the Release Pipeline in Azure DevOps to update Microsoft Fabric Deployment Pipeline stages

Because I am not creating any build artifacts I opted to work only with a Release pipeline.

Within the Releases feature I clicked the down arrow next to “New” and selected “New release pipeline”. On the screen to select a template I chose to start with an empty job.

I then entered an initial stage name of Test and then closed the window to go to the canvas.

After that I went to add an artifact. Because I am working directly with a Git repository instead of a build artifact I selected the “Azure Repos Git” source type.

From there I selected the relevant Git repository and the dev branch since that is the branch that the Dev workspace specified in the Fabric deployment pipeline is connected with.

Once done I clicked on the “Variables” tab and linked the two variable groups that I checked earlier. For the benefit of this post, I kept the scope of the groups to the release. However, individual stages can be selected.

I then went back into the Pipeline canvas and clicked underneath the Test stage name. From there, I selected my self-hosted agent pool before adding two new PowerShell tasks.

First task runs the Connect-AzAccount command to authenticate with the service principal credentials. It does this with the below code:

$SecureStringPwd = ConvertTo-SecureString $(servicePrincipalKey) -AsPlainText -Force
$pscredential = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $(servicePrincipalId), $SecureStringPwd
                  
Connect-AzAccount -ServicePrincipal -Credential $pscredential -Tenant $(tenantId)

Second task runs the DeploymentPipelines-DeployAll PowerShell script with the below parameters:

-deploymentPipelineName $(deploymentPipelineName) -sourceStageName $(DevStageName) -targetStageName $(TestStageName) -deploymentNote $(deploymentNote)'

Below is a description of each parameter:

• deploymentPipelineName – Name of the deployment pipeline in Microsoft Fabric
• DevStageName – The Dev stage name which is the source for the deployment in the deployment pipeline.
• TestStageName – The Test stage name, which is the target stage for the deployment in the deployment pipeline.
• deploymentNote – Any note I may wish to add.

Adding a Production stage in Azure DevOps to update Microsoft Fabric Deployment Pipeline stages

I then added a production stage. Because the new stage had similar logic as my Test stage, I navigated underneath my Test stage and selected “Clone”. I then selected the new stage and renamed it to “Production”.

From there, I renamed the second PowerShell task and changed the arguments for the task to the below. To reflect the fact that this time I was deploying from the Test stage to the Production stage in the Fabric deployment pipeline.

-deploymentPipelineName $(deploymentPipelineName) -sourceStageName $(TestStageName) -targetStageName $(ProdStageName) -deploymentNote $(deploymentNote)

Once I had changed the second task I went back into my pipeline canvas.

I then hovered over the left-hand side of my Production stage and selected “Pre-deployment conditions”. Which allowed me to add a pre-deployment approval to my production stage as below:

Which means that an approval must be done before the production stage is updated.

Once I added the approval and renamed my release pipeline my release pipeline looked like the below.

I then saved my pipeline and created a release to start the pipeline. As you can see below, when I went into the release I was asked to approve the deployment to Production.

After approving the stage, the release pipeline completed successfully.

After the pipeline had completed, I went back into the deployment pipeline in Microsoft Fabric. I then checked the deployment history to make sure the latest deployments had taken place.

As you can see, it states that the Service Principal initiated the deployments. Since we authenticated with that account.

Final words about utilizing the DeploymentPipelines-DeployAll PowerShell script and Azure DevOps

I hope me sharing how you can utilize the DeploymentPipelines-DeployAll PowerShell script and Azure DevOps to update Microsoft Fabric Deployment Pipeline stages helps some of you.

Especially if you want a way to perform approvals for deployment stages outside of Microsoft Fabric.

Of course, if you have any comments or queries about this post feel free to reach out to me.