Azure blob storage index tags- search blob by tags using REST API & bearer token authorization

Overview:

In the previous blog, we understood what are index tags and their use case. Also, we saw how to upload files with tags using REST API. If you have not checked it then please go through it before proceeding ahead.

In this blog, we will go through steps to query blobs by tags using REST API. Below are steps to create a request and test using Postman.

Important Notes:

1. Finding data using blob index tags can be performed by the Storage Blob Data Owner and by anyone with a Shared Access Signature that has permission to find blobs by tags (the f SAS permission).

2. Storage Blob Data Owner role can be assigned at the storage account level or specific container level. Depending on the scope of the role assignment, we can use specific APIs as below-

   • For storage account scope- Find Blobs by Tags (REST API) - Azure Storage

   • For container-level scope- Find Blobs by Tags in Container (REST API) - Azure Storage

3. The secondary index that Find Blobs by Tags uses is eventually consistent. Updates to blob tags might not be immediately visible to Find Blobs by Tags operations.

For this blog, we will use bearer token for authorization and role assignment with scope as storage account.

Pre-requisite:

1. Azure storage account with one or more containers should be created.

2. Storage account container (one or more) should have some sample files with index tags as - fileType='demo', env='dev'.

   example-

   

3. Register one app in Azure AD and get the client id, client secret for the same. This app will be used to assign RBAC and get bearer token. For this demo, I have already registered an app with the name PostmanUser.

Demo Steps:

1. To query blobs by tags, we need to create new Postman REST API request in below format-

   • URL- https://{{azure_storage_account}}.blob.core.windows.net?comp=blobs&where=<filter_condition>

   • Header parameter: x-ms-version = 2021-04-10 (you can use any latest version as well).

Here, I am using Postman environment variable named as azure_storage_account for storage account name but you can directly replace it with your storage account name.

FYI, Environment variable definition as below-

1. The value of the where URI parameter must be properly URI encoded. All tag values are sting and must be enclosed in single quotes. Below are the operators supported in where clause with examples-

   Operator	Description	Example
   =	Equal	&where=Status = 'In Progress'
   >	Greater than	&where=LastModified > '2018-06-18 20:51:26Z'
   >=	Greater than or equal	&where=Priority >= '05'
   <	Less than	&where=Age < '032'
   <=	Less than or equal	&where=Reviewer <= 'Smith'
   AND	Logical and	&where=Name > 'C' AND Name < 'D'
   @container	Specify a container	&where=@container='mycontainer' AND Name = 'C'

   For our scenario, we will use below filter expression-

   fileType='demo' AND env='dev'

   As we need to provide this filter expression in URI encoded format, we can go to URL: https://www.urlencoder.org/, enter the filter condition and get encoded string, as shows below.

   

   Replace <filter condition> with this encoded string in the request URL. Your request will look like below-

   

2. To get the bearer token, first, we need to assign Storage Blob Data Owner to our client app on the storage account. Go to Storage account > Access Control (IAM) > Add role assignment. Complete the role assignment and it should look like below.

   

3. Then, follow this blog and get access token for storage account i.e. use resource parameter value as- https://storage.azure.com in get token request. Make sure you are using the same client app to generate token which we have assigned a role on the storage account.

   Copy the generated token and paste it into the Authorization tab of the request as below.

   

4. Hit Send button and you will get an XML response in the below format. It will list all the blobs matching as per filter criteria. Each blob in the result will contain a container name and tags matching with the filter expression (where clause). Below is a sample output where you can see that two files are listed from different containers.

   

Conclusion:

In this blog, we understood how to query blobs by index tags across containers in a storage account.

Thanks for reading.

Keep Learning!