Operationalize fabric-cicd to work with Microsoft Fabric and GitHub Actions

Reading Time: 6 minutes

In this post I want to show how you can operationalize fabric-cicd to work with Microsoft Fabric and GitHub Actions. Since I got asked if this post was available whilst I was helping at the ask the experts panel during the Microsoft Fabric Community Conference.

Just so that everybody is aware, fabric-cicd is a Python library that allows you to perform CI/CD of various Microsoft Fabric items into Microsoft Fabric workspaces. At this moment in time there is a limited number of supported item types. However, that list is increasing.

Below is a diagram to help visualize this. Based on a development branch in a Git repository. Preferably updated via the recommended development process by Microsoft.

Previously, I shared the results of my initial tests of fabric-cicd. Now I want to show how you can implement fabric-cicd in an efficient manner. In order to deploy to multiple Microsoft Fabric workspaces with GitHub Actions for a cleaner and more modular approach.

To manage expectations, I do not cover implementing approvals with GitHub environments in this post. However, I do share plenty of links in this post.

If you need help with any jargon, then I recommend that you read one my other posts. Which is a Microsoft Fabric Git integration jargon guide for Fabricators.

Sample Git repository

I made a template for the contents of this post publicly available in GitHub. Which you are free to download and work with. It is called GitHub-fabric-cicd-sample.

However, I do have one request. If you do find the repository useful, please click on the star in the right-hand corner of the repository GitHub.

GitHub secrets and variables

Before working on the below example, I first created GitHub secrets and repository variables.

For the benefit of simplicity, I created these in the repository by going to Settings->Secrets and variables->Actions.

I added the below GitHub secrets:

• AZURE_CLIENT_ID – My service principal client ID.
• AZURE_CLIENT_SECRET – My service principal secret. Note this is the secret value.
• AZURE_TENANT_ID – My Microsoft Entra tenant ID.

I then added the below repository variables:

• ItemsInScope – List of all items you want deployed. For this example, I opted for the below items in the below format:
  “Notebook,Environment,Report,SemanticModel”
• ProdWorkspace – GUID value of the production workspace.
• TestWorkspace – GUID value of the test workspace.

I decided to add the resourceUrl as an environment variable in the GitHub workflow. Since the chances of that been updated are slim.

Python script to operationalize fabric-cicd to work with Microsoft Fabric and GitHub Actions

I opted to work with exactly the same Python script that I created in my previous post on how to operationalize fabric-cicd to work with Microsoft Fabric and YAML Pipelines in Azure DevOps. As you can see below.

I decided to keep the same script since it will still work by passing the relevant arguments.

from fabric_cicd import FabricWorkspace, publish_all_items, unpublish_all_orphan_items
import argparse

parser = argparse.ArgumentParser(description='Process some variables.')
parser.add_argument('--WorkspaceId', type=str)
parser.add_argument('--Environment', type=str)
parser.add_argument('--RepositoryDirectory', type=str)
parser.add_argument('--ItemsInScope', type=str)
args = parser.parse_args()

# Convert item_type_in_scope into a list
allitems = args.ItemsInScope
item_type_in_scope=allitems.split(",")
print(item_type_in_scope)

# Initialize the FabricWorkspace object with the required parameters
target_workspace = FabricWorkspace(
    workspace_id= args.WorkspaceId,
    environment=args.Environment,
    repository_directory=args.RepositoryDirectory,
    item_type_in_scope=item_type_in_scope,    
)

# Publish all items defined in item_type_in_scope
publish_all_items(target_workspace)

# Unpublish all items defined in item_type_in_scope not found in repository
unpublish_all_orphan_items(target_workspace)

Operationalize fabric-cicd to work with Microsoft Fabric and GitHub Actions

To make fabric-cicd operate efficiently with GitHub Actions first copied over the sample items that I had worked with in a previous post. Where I covered how to operationalize fabric-cicd to work with Microsoft Fabric and YAML Pipelines in Azure DevOps.

I then created a starter workflow. I provided a name for the workflow, set to trigger to manual and specified the resourceUrl environment variable with the below code.

name: Deploy via fabric-cicd

#Sets the trigger to manual
on:
  workflow_dispatch:

env:
  resourceUrl: https://api.fabric.microsoft.com

Afterwards, I configure two jobs in my GitHub workflow. One to deploy to a test workspace and another to deploy to the production workspace.

Job to deploy to a test workspace in Microsoft Fabric with fabric-cicd

I first created a job to deploy to the test workspace. Which contained multiple steps.

jobs:

  # Job to deploy to Test via fabric-cicd
  'Deploy_to_Test':
    # I use a GitHub-Hosted runner here, you can opt to use a self-hosted one instead
    runs-on: windows-latest

First, I checked out the repository, so that the subsequent Actions could access it.

# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v4.2.2

Afterwards, I added the Setup Python Action to specify Python 3.11.

# Use specific version of Python
- name: Setup Python
 uses: actions/setup-python@v5.5.0
 with:
   # Version range or exact version of Python or PyPy to use, using SemVer's version range syntax. Reads from .python-version if unset.
   python-version: 3.11

Once done, I added the fabric-cicd library.

# Install fabric-cicd library
- name: Install fabric-cicd library
  run: |
      python -m pip install --upgrade pip
      pip install fabric-cicd

From there, I added the PowerShell to authenticate as the service principal.

# Authenticate as Service Principal
- name: Authenticate as Service Principal
  run: |
      Install-Module -Name Az.Accounts -AllowClobber -Force

      $SecureStringPwd = ConvertTo-SecureString ${{secrets.AZURE_CLIENT_SECRET}} -AsPlainText -Force
      $pscredential = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList ${{secrets.AZURE_CLIENT_ID}}, $SecureStringPwd
                              
    Connect-AzAccount -ServicePrincipal -Credential $pscredential -Tenant ${{secrets.AZURE_TENANT_ID}}    

    $fabricToken = (Get-AzAccessToken -ResourceUrl ${{env.resourceUrl}}).Token

Finally, I added the below command to run the Python script to deploy with fabric-cicd to the test workspace,

# Run script to deploy with fabric-cicd to Test
- name: Run script to deploy with fabric-cicd to Test
  run: python auth_spn_secret_AzDo.py --WorkspaceId ${{vars.TestWorkspace}} --Environment "Test" --RepositoryDirectory ".\workspace" --ItemsInScope  ${{vars.ItemsInScope}}

One key point I want to highlight about the above command is the syntax to specify a repository variable. Because you need to specify vars in the brackets instead of secrets.

Job to deploy to a Production workspace in Microsoft Fabric with fabric-cicd

To deploy to the production workspace I added a new job and added the same Actions as I did to deploy to my test workspace for test deployments. Only difference was that I changed the arguments in my final Python command.

# Run script to deploy with fabric-cicd to Test
  - name: Run script to deploy with fabric-cicd to Prod
  run: python auth_spn_secret_AzDo.py --WorkspaceId ${{vars.ProdWorkspace}} --Environment "Prod" --RepositoryDirectory ".\workspace" --ItemsInScope  ${{vars.ItemsInScope}}

Test results to operationalize fabric-cicd to work with Microsoft Fabric

When I ran the pipeline the test deployment completed.

When I checked my production workspace I could see all the deployed items. Including the mirrored database.

Plus, just like in my previous post my “parameter.yml” file included the below statements. In order to utilize the find_replace functionality.

find_replace:
    # Replace Power BI Text
    - find_value: "This is a Report in Dev!!!"
      replace_value:
        Test: "This is a Report in Test!!!"
        Prod: "This is a Report in Prod!!!"

I checked, and both the test and production reports had been updated accordingly.

Conclusion of test results to operationalize fabric-cicd with GitHub Actions

By following the above, you can operationalize fabric-cicd to work with Microsoft Fabric and GitHub Actions. Like I mentioned earlier in this post, fabric-cicd is ideal for the supported item types specified on the fabric-cicd page. Plus, the list is growing.

At this moment in time you can experience errors when attempting to unpublish mirrored databases. You can publish them fine though.

Of course, everything I shared both in this post and the Git repository can be customized to suit your needs.

Other things you can consider doing are as follows:

• Keep an eye on fabric-cicd updates.
• Implement an approvals process with environments.
• Test with your own development items configured with Microsoft Fabric Git integration.
• Work with alternative secrets and variables to suit your own strategy. Plus, alternative methods to pass through the parameters.

Final words

I hope that showing this way to operationalize fabric-cicd to work with Microsoft Fabric and GitHub Actions helps some of you get started. Plus, inspire some of you to experiment in your own environments.

Because I do believe that fabric-cicd has a lot of potential.

Of course, if you have any comments or queries about this post feel free to reach out to me.