Automating Feature Workspace maintainance in Microsoft Fabric

At the Microsoft Fabric Community Conference in Las Vegas just a few weekends ago, Microsoft announced the public preview of the Fabric CLI - a powerful, developer-first command line interface that brings a file-system-inspired way to explore and manage your Fabric environment. As someone who's been deep in the weeds with the Fabric REST APIs for quite some time (and have blogged about it before), I'm excited to see how the Fabric CLI builds on the APIs to make automation more intuitive and accessible than ever.

In this blog post, I’ll walk you through how to use the Fabric CLI from within Python to support a best-practice approach for auto-generating and auto-configuring feature development workspaces in Microsoft Fabric.

For this initial example, we’ll use GitHub Actions along with a service principal for authentication, which, as of April 21st, 2025, is only supported when GitHub is used as the Git provider in Fabric. I plan to build on this foundation and expand the repo with support for full solution setup, orchestration, and deployment automation in the near future.

This post ties closely to my session at FabCon earlier this month — "From Setup to CI/CD: Automating Microsoft Fabric for Scalable Data Solutions" - where I showcased an end-to-end automation approach. If you’re interested, you can find the session materials and sample code here:

• 🔗 Session code & presentation from FabCon 2025: github.com/gronnerup/Fabric

• 🛠 This article's code repo (ongoing work): github.com/gronnerup/FabricAutomation

Prerequisites and Requirements

Before automating the creation of isolated feature development workspaces in Microsoft Fabric using the Fabric CLI and GitHub Actions, make sure you have the following in place:

1. Service Principal Authentication

This solution uses service principal authentication with a client secret, allowing secure, automated access to your Fabric environment. You’ll need to create an App Registration in Microsoft Entra ID and ensure the service principal is properly configured for Fabric API access.

In your GitHub repository, define these repository secrets:

• SPN_TENANT_ID – The Tenant ID of your Microsoft Fabric environment.

• SPN_CLIENT_ID – The Client ID (Application ID) of your app registration.

• SPN_CLIENT_SECRET – The Client Secret of the app registration.

Make sure the service principal is enabled for the Fabric REST APIs by following the official guidance:
Enable service principal for Fabric REST APIs

2. GitHub Personal Access Token (PAT)

You’ll also need to create a GitHub Personal Access Token (PAT) to enable Fabric’s Git integration. This token is used to authenticate Fabric when connecting to your GitHub repository.

Follow this guide to create a PAT and connect your workspace to Git:
Connect to a Git repo (Microsoft Learn)

3. Fork the Repository

To get started, fork the repository to your own GitHub account so you can safely configure secrets and CI/CD pipelines: https://github.com/gronnerup/FabricAutomation

This provides a clean slate to experiment and build on top of the existing automation approach.

My Approach to Continuous Integration with Git

When implementing Continuous Integration (CI) in Microsoft Fabric, it's essential to have a clear structure for both your workspaces and your Git repository. This helps ensure that your development process supports scalability, collaboration, and automation from day one.

Workspace Structure: Layer-Separated for Clarity and Control

The question of how to best structure workspaces in Fabric has been the subject of many discussions across blog posts, LinkedIn and Reddit threads. While there's no single “right” answer, my recommendation, based on practical experience and architectural clarity, is to follow a layer-separated workspace pattern.

This architecture separates your Fabric solution into logical layers, such as:

• Store: Lakehouse etc.

• Ingest: Notebooks, Data Pipelines, etc.

• Prepare: Notebooks focused on shaping and cleansing data

• Serve: Semantic Models and related artifacts

• Orchestrate: Data Pipelines or Notebooks driving execution logic

• Core: Components such as Variable Libraries, Environments, and Fabric Databases used for metadata

Each layer gets its own dedicated workspace, allowing for:

• Transparent organization of items and responsibilities

• Improved access control at the workspace level

• Capacity separation, which is especially useful in large-scale environments

This structure does introduce one consideration: isolated feature development workspaces may need to mirror more than one layer, depending on the scope of the feature being implemented. In other words, a single feature branch may touch multiple workspaces — and that’s okay, as long as it’s organized.

Git Repository Structure: One Repo to Rule Them All

To support this workspace setup effectively, I recommend keeping all your Fabric resources in a single Git repository. Within this repo, each solution layer is represented by a subfolder, and each layer-specific workspace connects to its respective folder via Fabric’s Git integration.

A typical structure might look like this:

/.azure-pipelines    # Azure DevOps pipelines
/.github             # GitHub Actions workflows
/automation          # Scripts, deployment helpers etc.
/documentation       # Solution documentation. Can be used for Azure DevOps project Wiki
/solution            # Solution folders for the different layers
  /Core
  /Ingest
  /Orchestrate
  /Prepare
  /Serve
  /Store

This structure offers a few key benefits:

• End-to-end feature branches – You can implement a business requirement across all relevant layers (and include documentation!) in a single branch.

• CI/CD alignment – Makes it easier to automate build/test/deploy processes using GitHub Actions or Azure Pipelines.

• Organizational clarity – Developers always know where to find and contribute to specific parts of the solution.

With this setup, isolated feature development workspaces are created dynamically and point to the relevant subfolders. This aligns perfectly with the approach demonstrated in this blog post, and it’s designed to scale with the complexity of your data platform.

Automating the Feature Development Process

As highlighted in Microsoft’s official documentation on deployment and development processes, it’s considered best practice to isolate development work outside of your main collaboration branch. This ensures cleaner version control, better collaboration, and minimizes disruption to ongoing work.

Following Git standards, development should happen in feature branches, each representing a specific unit of work. This allows for focused development, easier reviews, and safer integration into the mainline once complete.

When working in Microsoft Fabric, isolated development also means creating separate workspaces to support and validate your changes. There are two primary ways to do this:

1. Manual setup via the Fabric UI

2. Programmatic setup via the Fabric REST APIs or the Fabric CLI

But why stop at manual or semi-automated processes?

Taking It to the Next Level: Automating Workspace Creation

By leveraging GitHub Actions or Azure DevOps pipelines, we can automate the entire process of setting up — and later tearing down — feature development workspaces. This not only saves time but ensures consistency across environments.

In my approach, I use a recipe file that defines exactly how feature workspaces should be configured. This file, feature.json, lives in the repository at: automation/resources/environments/

{
    "feature_name" : "*{feature_name}-{layer_name}",
    "capacity_name": "MyCapacity",
    "git_settings": {
        "gitProviderDetails": {
            "gitProviderType": "GitHub",
            "ownerName": "MyGitHubProfile",
            "repositoryName": "MyGitHubRepo"
        },
        "myGitCredentials": {
            "source": "ConfiguredConnection",
            "connectionId": "00000000-0000-0000-0000-000000000000"
        }
    },
    "permissions": {
        "admin": [
            {"type": "Group", "id": "00000000-0000-0000-0000-000000000000"}
        ],
        "contributor": [
            {"type": "User", "id": "00000000-0000-0000-0000-000000000000"}
        ]
    },
    "layers": {
        "Prepare": {
            "spark_settings": {
                "pool": {
                    "starterPool": {
                        "maxExecutors": 1,
                        "maxNodeCount": 1
                    }
                }
            },
            "git_directoryName": "solution/prepare"
        },
        "Ingest": { "git_directoryName": "solution/prepare" },
        "Orchestrate": { "git_directoryName": "solution/orchestrate" }
    }
}

The key elements include:

• A naming convention for feature workspaces (prefixed with an asterisk for easy visibility)

• The target capacity for deployment

• Git integration settings and authentication

• Permissions configuration for users and/or groups

• Layer-specific settings such as Spark pool resource limits can be particularly useful - for example, by configuring a single-node Spark pool, you can reduce vCore consumption and minimize the risk of hitting concurrency limits.

GitHub Workflows: Creation and Cleanup

Inside the .github/workflows folder of the repository, you’ll find two workflows:

1. Create Fabric feature workspaces on feature branch creation
   Triggered when a new feature branch is created.

2. Cleanup Fabric feature workspaces on merge to main
   Triggered when the feature is merged into main.

Both workflows call the Python script fabric_feature_maintainance.py (found in automation/scripts), which handles the actual creation or deletion logic. Under the hood, the script uses the Fabric CLI, calling commands via a utility function defined in: automation/scripts/modules/fabric_cli_functions.py

CLI commands are executed using a simple run_command() function:

def run_command(command: str) -> str:
    try:
        result = subprocess.run(
            ["fab", "-c", command],
            capture_output=True,
            text=True,
            check=EXIT_ON_ERROR
        )
        return result.stdout.strip()
    ...

And for functionality not yet covered by Fabric CLI commands, I use the powerful fab api command to interact directly with the Fabric REST API - for example, when connecting and synchronizing Git repositories.

Quickstart Walkthrough

Curious how this works in practice? Here’s a simple walkthrough to get you up and running with automated feature workspace creation in Microsoft Fabric.

1. Fork the Repository

Head over to:
👉 https://github.com/gronnerup/FabricAutomation
Fork it to your own GitHub account.

2. Set Up Your Secrets and Service Principal

Make sure you’ve followed the prerequisites:

• Create a service principal and assign necessary API permissions

• Configure your repository secrets:

  • SPN_TENANT_ID

  • SPN_CLIENT_ID

  • SPN_CLIENT_SECRET

• Set up Git integration with a GitHub Personal Access Token (PAT) and create a new cloud connection in Fabric go generate the required connection id. Choose Github - Source control as the connection type.

3. Customize the feature.json recipe file

Edit the file automation/resources/environments/feature.json
Define how your feature workspaces should be created:

• Workspace naming pattern

• Fabric capacity

• Git repo connection settings

• Layers to include and optional Spark pool settings

4. Create a Feature Branch

Create a new branch in your GitHub repository by using the naming convention feature/\***.

This will automatically trigger the GitHub Action responsible for creating your feature workspaces.

5. Watch the Workspaces Come to Life

Within seconds, your configured feature workspaces will appear in Microsoft Fabric - connected to Git and syncronized, with permissions and Spark settings applied.

6. Merge and Clean Up Automatically

When the feature is complete and you merge your branch into main, a separate GitHub Action will trigger and clean up the feature workspaces - keeping your Fabric environment tidy and focused.

Wrapping Up

Automating the creation of feature workspaces in Microsoft Fabric is a key step toward a scalable, repeatable, and developer-friendly data platform. By combining the power of the Fabric CLI, GitHub Actions, and a simple recipe-based configuration, we can streamline the entire development process - from branch creation to workspace provisioning and eventual cleanup.

This is just the beginning.

In the coming weeks and months, I’ll continue to enhance the FabricAutomation repository to reflect my latest work, including:

• Automated solution setup for new projects and environments

• Solution automation using a metadata-driven framework

• CI/CD pipelines using Fabric CLI and the fabric-cicd Python library

• Support for Azure DevOps Pipelines

• Branching and merging strategies for structured, enterprise-grade development

• Enhanced support for user specific recipe files and much more…

Stay tuned - and feel free to star the repo or follow along if you're as excited about Fabric automation as I am. 🚀