Currently Azure AI Search includes functionality to autonomously index documents and generate embeddings for use within Azure Open AI queries.
However, there is not currently built-in functionality to autonomously ‘enrich‘ document data within Azure AI Search where API Management is governing access to Azure Open AI.

Many enterprises are looking for patterns to govern and manage access to their Azure Open AI resources, one such pattern is to place Azure API Management in front of Azure Open AI providing a governance gateway.
Solutions requiring access to Azure Open AI for embedding and completion operations must therefore communicate via Azure API Management, including Azure AI Search.

This blog covers a pattern that allows Azure AI Search Integrated Vectorisation to undertake the generation of document embedding data using Azure Open AI where Azure API Management is present as middleware.

The Pattern

The goal of this pattern is to enable the Azure Search Indexer to generate embeddings using a skillset definition that points to Azure API Management.
The Azure API Management method in-turn relays the embeddings requests back and forth to Azure Open AI. The indexer then places the embedding vector data into the AI Search Index.
The steps within the process are listed in more detail below:

An Azure AI Search Indexer run is initiated, reading a document from the configured Data Source.
The Skillset attached to the indexer has a ‘CustomWebAPISkill’ defined to generate embeddings, with a URI and credentials pointing to the API Management ‘Embeddings for Azure Search‘ endpoint.
JSON data is sent to the API Management method by the Indexer.
The API Management endpoint ‘Embeddings for Azure Search‘ receives the strings to have embeddings generated for, records the ‘recordId’ values, transforms the JSON body to that expected by Azure Open AI and forwards on the request to Azure Open AI.
Azure Open AI receives the embedding request from API Management, generates the embedding data and sends the response back to the API Management resource.
The API Management resource receives and transforms the embedding data from Azure Open AI, placing the ‘recordId’s’ from the original input back onto the collection elements, ensuring the JSON schema is of a format expected by the Azure Search indexer.
The response is then forwarded back to the indexer which stores the embedding data, along with the document metadata into the AI Search Index.

Challenges and Solutions

Azure AI Search indexers have no built-in support for generating embedded data where API Management is governing access to Azure Open AI.
An Azure AI Search Skillset can be pointed to an API Management method, however a ‘CustomWebAPISkill’ is required.
An Azure AI Search Skillset using a ‘CustomWebAPISkill’ for enrichment has a fixed input and output JSON schema that significantly differs from that used by Azure Open AI embeddings. Manipulation of the body data within API Management is required inbound and outbound.
The Azure AI Search Indexer passes ‘recordId’ values for each item within the JSON body which should be relayed back onto the same data blocks after Azure Open AI has generated the embedding values. This ensures existing enriched data can be updated. API Management processing must handle this ID mapping.
Azure API Management requires unique method names for it’s inputs and replays the method name to the output. Adding a new method e.g. ‘embeddingsForSearch’ would result in the request being sent to Azure Open AI at the same address which Azure Open AI does not support.
The API Management resource must therefore be configured to ‘Rewrite’ the URL to simply be ‘embeddings‘ for the backend Azure Open AI call.

Implementation

Note: The examples shown below are provided ‘as-is’ as a proof-of-concept and should not be considered for production use.

Assumptions

Azure AI Search, Azure APIM and Azure Open AI Services have been deployed.
Azure APIM has been configured with the built-in Azure Open AI configuration.
Azure Open AI has the ‘text-embedding-ada-002‘ model deployed.

Azure AI Search Configuration

Skillset

{
    "name": "test-skillset",
    "description": "Skillset to chunk documents and generate embeddings",
    "skills": [
    {
        "@odata.type": "#Microsoft.Skills.Text.SplitSkill",
        "name": "#1",
        "description": "Split skill to chunk documents",
        "context": "/document",
        "inputs": [
         {
            "name": "text",
            "source": "/document/content",
            "inputs": []
         }
        ],
        "outputs": [
         {
            "name": "textItems",
            "targetName": "pages"
         }
        ],
        "defaultLanguageCode": "en",
        "textSplitMode": "pages",
        "maximumPageLength": 200,
        "pageOverlapLength": 5,
        "unit": "characters"
    },
    {
        "@odata.type": "#Microsoft.Skills.Custom.WebApiSkill",
        "name": "#2",
        "description": "APIM Embedding",
        "context": "/document/pages/*",
        "inputs": [
         {
            "name": "text",
            "source": "/document/pages/*",
            "inputs": []
         },
        "outputs": [
          {
            "name": "vector",
            "targetName": "vector"
           }
        ],
        "uri": "https://{APIMSERVICENAME}.azure-api.net/openai/deployments/text-embedding-ada-002/embeddings-ai-search?api-version=2023-05-15",
        "httpHeaders": {
            "{APIMHEADERKEYNAME}": "{APIMHEADERKEYVALUE}"
        },
        "httpMethod": "POST",
        "timeout": "PT30S",
        "batchSize": 5
     }
    ],
    "@odata.etag": "\"0x8DCFCEB6F53B10F\"",
    "indexProjections": {
     "selectors": [
      {
        "targetIndexName": "index1",
        "parentKeyFieldName": "parent_id",
        "sourceContext": "/document/pages/*",
        "mappings": [
         {
            "name": "vector",
            "source": "/document/pages/*/vector"
         },
         {
            "name": "chunk",
            "source": "/document/pages/*"
         },
         {
            "name": "metadata_storage_path",
            "source": "/document/metadata_storage_path"
         },
         {
            "name": "title",
            "source": "/document/title"
         }
        ]
        }
    ],
    "parameters": {
        "projectionMode": "skipIndexingParentDocuments"
    }
  }
}

Parameters

{APIMSERVICENAME} - The name of the APIM service that forms part of the URL to access it.
{APIMHEADERKEYNAME} - The APIM header key name used for authentication.
{APIMHEADERKEYVALUE} - The APIM header key value used for authentication.

Index

Indexer

{
    "@odata.context":"https://{CONTEXTNAME}/$metadata#indexers/$entity",
    "@odata.etag":"\"0x8DCFCEBAA8B8DFC\"",
    "name":"indexer1",
    "description":null,
    "dataSourceName":"{DATASOURCENAME}",
    "skillsetName":"{SKILLSETNAME}",
    "targetIndexName":"{INDEXNAME}",
    "disabled":null
    "schedule":null,
    "parameters":{
     "batchSize":1,
     "maxFailedItems":null,
     "maxFailedItemsPerBatch":null,
     "base64EncodeKeys":null,
     "configuration":{
        "dataToExtract":"contentAndMetadata",
        "parsingMode":"default"
     }
    },
    "fieldMappings":[
     {
        "sourceFieldName":"metadata_storage_name",
        "targetFieldName":"title",
        "mappingFunction":null
     }
    ],
    "outputFieldMappings":[
     {
        "sourceFieldName":"/docemnt/pages/*/vector",
        "targetFieldName":"vector",
        "mappingFunction":null
     }
    ],
    "cache":null,
    "encryptionKey":null
}

References

Custom Web API skill in an Azure AI Search enrichment pipeline

Azure Open AI REST API - Embeddings

Unlocking the Power of Azure: Azure AI Search Integrated Vectorisation with API Management and Azure Open AI