Service Principal For Azure

In an earlier article on Managed Identity in Azure, I demonstrated how managed identity can significantly enhance the security and efficiency of your cloud applications. By using managed identities one can eliminate the need to manage credentials manually reducing the risk of credentials being exposed or misused. Managed identities allow Azure services to authenticate without needing to store credentials in the code or configuration files. This approach simplifies authentication and increases overall security of the Azure systems.

Why use Service Principal ?

There are some nuances that one should be aware of when it comes to service principal. Passwords or secrets associated with Service Principals have expiration periods and require regular rotation. This is especially handy when you would want to provide expiry based authentication for the applications. For example, you would want to provide access to certain systems or applications only for a limited period of time without worrying about manually tracking the expiration or revoking credentials, and without needing to embed credentials within the applications themselves. Though the management of Service Principal credentials still poses challenges the biggest advantage being that the credentials can be centrally managed and has a certain expiry.

What is Service Principal ?

A Service Principal in Azure can be defined as an identity created for applications and services that can be used to securely authenticate and access Azure resources. It acts as an Azure AD entity with assigned permissions, allowing services to interact with resources like databases or storage. Service Principals typically use client ID and secret credentials, which need to be securely managed and rotated regularly.

The Setup

To get started we need to register the application at entra.microsoft.com

Once logged in, navigate to Applications >>App registrations >> New registration

I registered the application under the name SP_ADF_POC. We would required ClientId and TenantId values to reference in the code from the registered app.

Clicking the New client secret allows you to set the expiry for the Service Principal credentials.

Now that the feature is enabled and all set, lets create a ADF package that imports data from a text file on a Blob container into a Azure SQL database. We will create a empty Azure Data Factory and then add components to import data from a text file on a Azure container into a Azure SQL Database.

We can do it through Azure portal or PowerShell.

To get started with PowerShell, we have to install the following PS modules.

Install-Module -Name Az -AllowClobber -Force
Install-Module -Name Az.Sql -AllowClobber -Force
Install-Module -Name Az.DataFactory -AllowClobber  -Force
Install-Module -Name Az.Resources -AllowClobber -Force
Install-Module -Name Az.ManagedServiceIdentity -AllowClobber -Force
Install-Module -Name Az.Storage -AllowClobber -Force

Incase if you need to upgrade your PowerShell environment you can do it through the following command

winget install --id Microsoft.PowerShell --source winget

Code

Connect to Azure PowerShell. You can use PowerShell ISE.

Connect-AzAccount

In PowerShell

$resourceGroupName = "Your Resource Group" 
$sqlServerName="adf-sqlserverdb"
$aadAdmin="User to be assigned as Azure SQL server admin"
Set-AzSqlServerActiveDirectoryAdministrator `
  -ResourceGroupName $resourceGroupName `
  -ServerName $sqlServerName `
  -DisplayName $aadAdmin `
  -ObjectId (Get-AzADUser -UserPrincipalName $aadAdmin).Id

In Azure Portal

Set the admin for Azure SQL instance. The Sql instance in this case is adf-sqlserverdb

You also might want to set up Network access to the Azure Sql Server instance.

Lets connect to the Azure SQL Instance adf-sqlserverdb through SSMS and create a database named SP_ADF_POC and create a table that maps the source data schema. I named the table as tbl and has 2 columns date and values.

The SSMS connection to the Azure Sql works ok as I have logged in through the Entra authentication which was assigned as a admin to the instance.

Next we create a ADF pipeline that uses Service Principal.

In PowerShell

$TenantId = "TenantId of the registeted app in Entra" 
$ClientId = "ClientId of the registeted app in Entra"
$ClientSecret = "ClientSecret of the registeted app in Entra"
$resourceGroupName = "Your Resource Group"
$DataFactoryName="ADF-User-Service-Principal"
$Location = "Central India"
$SecureClientSecret = ConvertTo-SecureString $ClientSecret -AsPlainText -Force
$Credential = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $ClientId, $SecureClientSecret
Connect-AzAccount -ServicePrincipal -Tenant $TenantId -Credential $Credential
Set-AzDataFactoryV2 -ResourceGroupName $ResourceGroupName -Name $DataFactoryName -Location $Location

The above code should create a ADF named ADF-User-Service-Principal.

You can alternatively create the ADF through Azure Portal. Point to be noted, is that we wont have to set any managed identities to this resource, instead we have to grant contributor access to the service principal through IAM » Add role assignment

Once done the service principal should be visible under the Role assignments tab

Now create a linked service to a blob storage(location: temporaryontainer/destination) where the text file resides using the service principal details.

As expected the connection failed. This is because the service principal has no access to the blob container. To fix this issue, grant Storage Blob Container Contributor access to the service principal.

Once done, retest the linked service connection and this time it should succeed

Next create a linked service to connect to the Azure SQL database.The connection to the SQL database failed.

This is because the service principal SP_ADF_POC is not a user in the database SP_ADF_POC. So we would have to add this service principal as a user to the database.

CREATE USER [SP_ADF_POC] FROm EXTERNAL PROVIDER;
GO
ALTER ROLE db_owner ADD MEMBER[SP_ADF_POC]

I hope the names that I used do not cause confusions, as I have used the name SP_ADF_POC both for the service principal and SQL Database.

Retest the connection and this time the database connection should succeed.

Now that all the settings are in place, lets now create a simple data pipeline that imports data from a text file to the SQL database in table tbl. Set the Source and Sink of the pipeline to the linked services created earlier.

Execute the package and it should succeed.

Conclusion

In conclusion implementing service principals in Azure streamlines the authentication process for certain use cases where you would want to maintain credentials across Azure services and applications without the overhead of needing to store credentials in the code or configuration files.

Thanks for reading !!!