Supercharge Your Azure API Calls: Master Azure Resource Manager batching with PowerShell

---

Introduction & Problem Statement

Picture this: You're tasked with auditing 200 virtual machines across multiple Azure subscriptions. Your PowerShell script needs to check VM status, retrieve configurations, and gather diagnostic information. Using the traditional approach, this means making 600+ individual API calls to Azure Resource Manager (ARM) - three calls per VM.

The result? Your script crawls along for 30+ minutes, constantly hits throttling limits, and occasionally fails with timeout errors. Meanwhile, your manager is asking why the monthly audit report is always late, and you're spending more time babysitting scripts than actually analyzing the data.

What if I told you the same audit could be completed in under 3 minutes with just 30 API calls?

That's the power of Azure Resource Manager API batching. A lesser-known but incredibly powerful Azure capability that can reduce your API calls by up to 80%. In this guide, we'll explore two PowerShell functions that make this possible: New-AzureBatchRequest and Invoke-AzureBatchRequest hosted in the AzureCommonStuff module.

---

Understanding the Challenge

Why Individual API Calls Don't Scale

Azure Resource Manager implements rate limiting to protect the service infrastructure. The current limits are:

• Read operations: 12,000 requests per hour per subscription

• Write operations: 1,200 requests per hour per subscription

While these numbers might seem generous, they disappear quickly in automation scenarios. Consider a simple VM inventory script:

• 100 VMs × 3 API calls each = 300 requests

• That's 25% of your hourly read quota in a single script run

The Performance Tax

Every individual API call carries overhead:

• Network latency: 50-200ms per request, depending on your location

• Authentication processing: Token validation for each request

• Connection overhead: TCP handshake and SSL negotiation

• JSON processing: Multiple small payloads instead of one optimized batch

Common Workarounds and Their Limitations

Most PowerShell developers try these approaches:

1. Sequential processing with delays - Slow and still hits limits

2. Parallel processing with ForEach-Object -Parallel - Actually makes throttling worse

3. Custom retry logic - Adds complexity but doesn't solve the root problem

These approaches treat the symptoms, not the cause. We need a fundamentally different approach.

The Hidden Solution

Azure Resource Manager supports an undocumented batch API endpoint that combines up to 20 individual requests into a single HTTP call. This isn't in the official documentation, but it's the same mechanism the Azure portal uses behind the scenes (as shown in the image below 👇). It's stable, reliable, and incredibly powerful once you know how to use it.

---

Solution Overview

How Batching Works

Instead of making individual API calls, we package multiple requests into a single batch operation. Azure processes these requests concurrently on the server side and returns all results in one response.

# Traditional approach (slow)
$vmData = @()
foreach ($vmName in $vmNames) {
    $vm = Get-AzVM -ResourceGroupName $rgName -Name $vmName
    $vmData += $vm
}
# Result: 100 VMs = 100 API calls

# Batching approach (fast)
$batchRequests = New-AzureBatchRequest -url "/subscriptions/$subscriptionId/resourceGroups/$rgName/providers/Microsoft.Compute/virtualMachines/<placeholder>?api-version=2023-07-01" -placeholder $vmNames
$vmData = Invoke-AzureBatchRequest -batchRequest $batchRequests
# Result: 100 VMs = 5 API calls (batches of 20)

Key Concepts

Batch Request Objects: Each request contains a method (GET, POST, etc.), URL, and optional headers. The batch system processes these concurrently and maintains the order of results.

Placeholder Pattern: When you need similar requests with different IDs, the placeholder system generates requests automatically without manual loops.

Error Isolation: If one request in a batch fails, the others continue processing. Each response includes its own status code and error details.

Integration Points

The batching functions integrate seamlessly with:

• Existing Azure authentication (uses your current PowerShell session)

• Standard PowerShell patterns (pipeline-friendly, verbose logging)

• Azure throttling limits (built-in retry logic with exponential backoff)

---

How It Works

New-AzureBatchRequest

Function New-AzureBatchRequest creates standardized request objects that can be batched together:

• URL Flexibility: You can use absolute URLs (https://management.azure.com/...) or relative URLs (/subscriptions/...). The function handles both seamlessly.

• Placeholder Magic: The real power comes from the placeholder parameter. Instead of writing loops, you specify a URL template with <placeholder> and provide an array of values:

# Instead of this manual loop:
$requests = @()
foreach ($subscriptionId in $subscriptionIds) {
    $requests += @{
        Name = "sub_$subscriptionId"
        HttpMethod = "GET"
        URL = "/subscriptions/$subscriptionId/providers/Microsoft.Compute/virtualMachines?api-version=2023-07-01"
    }
}

# Use this elegant approach:
$requests = New-AzureBatchRequest -url "/subscriptions/<placeholder>/providers/Microsoft.Compute/virtualMachines?api-version=2023-07-01" -placeholder $subscriptionIds

Invoke-AzureBatchRequest

Function Invoke-AzureBatchRequest handles the complex orchestration of sending batches and processing responses:

• Automatic Chunking: The function automatically splits your requests into chunks of 20 (Azure's limit) and processes them sequentially.

• Response Beautification: By default, the function extracts just the data you need from Azure's verbose response format, making results much easier to work with.

Step-by-Step Walkthrough

Let's trace through a complete batching operation:

Step 1: Create Batch Requests

# Get VM status across multiple resource groups
$resourceGroups = @('rg-web-servers', 'rg-database-servers', 'rg-cache-servers')
$subscriptionId = (Get-AzContext).Subscription.Id

$requests = New-AzureBatchRequest -url "/subscriptions/$subscriptionId/resourceGroups/<placeholder>/providers/Microsoft.Compute/virtualMachines?api-version=2023-07-01" -placeholder $resourceGroups

Behind the scenes, this creates three request objects:

# Generated request objects (simplified)
@(
    @{ Name = "123456"; HttpMethod = "GET"; URL = "/subscriptions/abc.../resourceGroups/rg-web-servers/providers/Microsoft.Compute/virtualMachines?api-version=2023-07-01" },
    @{ Name = "789012"; HttpMethod = "GET"; URL = "/subscriptions/abc.../resourceGroups/rg-database-servers/providers/Microsoft.Compute/virtualMachines?api-version=2023-07-01" },
    @{ Name = "345678"; HttpMethod = "GET"; URL = "/subscriptions/abc.../resourceGroups/rg-cache-servers/providers/Microsoft.Compute/virtualMachines?api-version=2023-07-01" }
)

Step 2: Execute the Batch

$allVMs = Invoke-AzureBatchRequest -batchRequest $requests

The function:

1. Validates all URLs and request objects

2. Chunks requests into groups of 20

3. Sends each chunk as a single HTTP POST to Azure's batch endpoint

4. Processes responses and handles errors

5. Beautifies results by extracting just the VM data

Step 3: Work with Results

# Results are ready to use immediately
$windowsVMs = $allVMs | Where-Object { $_.storageProfile.osDisk.osType -eq 'Windows' }
$runningVMs = $allVMs | Where-Object { $_.powerState -eq 'VM running' }

Write-Host "Found $($allVMs.Count) total VMs, $($windowsVMs.Count) Windows VMs, $($runningVMs.Count) running"

---

Practical Examples

Real-World Scenario: Security Compliance Audit

Here's a more complex example that demonstrates the power of batching for compliance scenarios:

# Scenario: Audit network security groups, storage accounts, and key vaults across subscriptions
$subscriptionIds = (Get-AzSubscription | Where-Object State -eq 'Enabled').Id

# Build audit requests for multiple resource types
$auditRequests = @()

# Network Security Groups
$auditRequests += New-AzureBatchRequest -url "/subscriptions/<placeholder>/providers/Microsoft.Network/networkSecurityGroups?api-version=2024-03-01" -placeholder $subscriptionIds -name "NSGs"

# Storage Accounts  
$auditRequests += New-AzureBatchRequest -url "/subscriptions/<placeholder>/providers/Microsoft.Storage/storageAccounts?api-version=2023-05-01" -placeholder $subscriptionIds -name "Storage"

# Key Vaults
$auditRequests += New-AzureBatchRequest -url "/subscriptions/<placeholder>/providers/Microsoft.KeyVault/vaults?api-version=2024-11-01" -placeholder $subscriptionIds -name "KeyVaults"

# Execute all requests efficiently
Write-Host "Starting audit of $($subscriptionIds.Count) subscriptions..."
$auditResults = Invoke-AzureBatchRequest -batchRequest $auditRequests

# Process results by resource type
$nsgs = $auditResults | Where-Object { $_.RequestName -like "NSGs_*" }
$storageAccounts = $auditResults | Where-Object { $_.RequestName -like "Storage_*" }
$keyVaults = $auditResults | Where-Object { $_.RequestName -like "KeyVaults_*" }

# Generate compliance report
Write-Host "Audit Summary:"
Write-Host "- Network Security Groups: $($nsgs.Count)"
Write-Host "- Storage Accounts: $($storageAccounts.Count)"  
Write-Host "- Key Vaults: $($keyVaults.Count)"

# Check for common security issues
$publicStorageAccounts = $storageAccounts | Where-Object { $_.properties.allowBlobPublicAccess -eq $true }
if ($publicStorageAccounts) {
    Write-Warning "Found $($publicStorageAccounts.Count) storage accounts with public blob access enabled"
}

Performance comparison for this scenario:

• Traditional approach: 600+ API calls, 15-20 minutes

• Batching approach: 30 API calls, 2-3 minutes

Common Variations: Different HTTP Methods

While most scenarios use GET requests, you can batch other operations too:

# Batch different types of operations
$mixedRequests = @()

# GET requests for current state
$mixedRequests += New-AzureBatchRequest -url "/subscriptions/$subscriptionId/resourceGroups/<placeholder>/providers/Microsoft.Compute/virtualMachines?api-version=2023-07-01" -placeholder $resourceGroups -name "GetVMs"

# POST requests for operations (example: restart VMs)
$vmIds = @('vm1', 'vm2', 'vm3')
$mixedRequests += New-AzureBatchRequest -url "/subscriptions/$subscriptionId/resourceGroups/$rgName/providers/Microsoft.Compute/virtualMachines/<placeholder>/restart?api-version=2024-11-01" -placeholder $vmIds -method "POST" -name "RestartVMs"

# Execute mixed batch
$results = Invoke-AzureBatchRequest -batchRequest $mixedRequests

# Check operation results
$getResults = $results | Where-Object { $_.RequestName -like "GetVMs_*" }
$restartResults = $results | Where-Object { $_.RequestName -like "RestartVMs_*" }

Troubleshooting Tips

When things don't work as expected, try these approaches:

# Enable verbose output to see what's happening
$VerbosePreference = 'Continue'
$results = Invoke-AzureBatchRequest -batchRequest $requests -Verbose

# Check for failed requests by examining raw responses
Invoke-AzureBatchRequest -batchRequest $requests -dontBeautifyResult

# Test individual URLs before batching
$testUrl = "/subscriptions/$subscriptionId/resourceGroups/$rgName/providers/Microsoft.Compute/virtualMachines?api-version=2023-07-01"
try {
    $testResult = Invoke-AzRestMethod -Uri "https://management.azure.com$testUrl" -Method GET
    Write-Host "Test successful: $($testResult.StatusCode)"
} catch {
    Write-Error "URL test failed: $_"
}

---

Tips and Considerations

Performance Optimization

Use Resource Graph Query: If possible, gather your data from the Resource Graph Table, which will always be faster than api calls. Nice example of getting IAM assignments.

Monitor Your Quotas: Even with batching, be mindful of overall API limits across your entire environment. Use Get-AzConsumptionUsageDetail to monitor usage patterns.

Common Mistakes to Avoid

URL Template Errors: Make sure your placeholder strings don't conflict with actual URL parameters:

# Wrong - conflicts with OData filter syntax
$badUrl = "/virtualMachines?$filter=name eq '<placeholder>'"

# Right - placeholder in path segment
$goodUrl = "/virtualMachines/<placeholder>"

Authentication Scope Issues: Ensure your PowerShell session has permissions for all resources you're querying. Cross-subscription batching requires appropriate access.

Api version errors: Ensure you append a valid api version (api-version=2023-07-01 for example) to each URL request!

".../resourceGroups/$rgName/providers/Microsoft.Compute/virtualMachines?api-version=2023-07-01"

If you are unsure what api to use:

• Check official documentation.

• Use a random one, and when the request fails with a 400 error, check the error message for the list of correct api versions.

• Use the official corresponding Az cmdlet with -debug parameter (Get-AzStorageAccount -debug) and check the 'Absolute uri' output.

• Use developer tools (F12) in your browser when using the Azure Portal and check the request URL there.

---

Getting Started

These functions are part of the AzureCommonStuff module on PowerShell Gallery. Here's how to install it:

# Install the module (one-time setup)
Install-Module AzureCommonStuff

# Import the module in your session
Import-Module AzureCommonStuff

# Verify the functions are available
Get-Command New-AzureBatchRequest, Invoke-AzureBatchRequest

# Get help
Get-Help New-AzureBatchRequest -Examples
Get-Help Invoke-AzureBatchRequest -Examples

Prerequisites

Before using these functions, make sure you have:

• PowerShell 5.1 or PowerShell 7+

• Az PowerShell module installed (Install-Module Az)

• Authenticated Azure session (Connect-AzAccount)

• Appropriate permissions for the resources you want to access

---

Next Steps & Resources

Congratulations! You now know how to dramatically improve the performance of your Azure automation scripts using ARM API batching. Here's what you can do next:

Immediate Actions

1. Identify bottlenecks in your existing scripts where you're making multiple similar API calls

2. Start small by converting one script to use batching and measure the performance improvement

3. Expand gradually to other automation scenarios once you're comfortable with the pattern

Advanced Techniques

• Azure Resource Graph: For read-only scenarios across multiple subscriptions, consider combining batching with Azure Resource Graph queries

• Parallel batching: Use ForEach-Object -Parallel to run multiple independent batches simultaneously

• Automated reporting: Build PowerShell-based reporting solutions that leverage batching for regular compliance and inventory reports

Related Functions

The AzurePIMStuff module includes functions that use batching under the hood; feel free to take inspiration there.

Summary

Azure Resource Manager API batching transforms the way you interact with Azure at scale. By reducing API calls and execution times, these PowerShell functions unlock new possibilities for automation, reporting, and compliance scenarios.

The functions New-AzureBatchRequest and Invoke-AzureBatchRequest leverage an undocumented but stable Azure API that's used by the Azure portal itself. While it's not officially documented, it's proven reliable across multiple Azure updates and represents a significant competitive advantage for anyone doing serious Azure automation work.

Start experimenting with small batches, measure the performance improvements, and gradually adopt batching across your Azure automation portfolio. Your scripts will run faster, your users will be happier, and you'll spend less time waiting for operations to complete and more time solving interesting problems.