ODC with Graph - Ingesting Application Data into Microsoft Graph

Stefan WeberStefan Weber
13 min read

In most cases, when we create an integration with Microsoft 365, we use data and actions from the Microsoft 365 ecosystem in OutSystems applications. This can include reading user profile information, retrieving files from a OneDrive folder, sending an email on behalf of a user or resource mailbox, and much more.

However, the Graph API also lets us add our application data to the Graph, making it directly accessible to Microsoft 365 applications and services.

  • Context IQ in Outlook Web - Allows users to find in-content suggestions from ingested application data while composing an email.
💡
At the time of writing, Context IQ in Outlook Web is in preview.
  • Microsoft 365 apps - Allows application data to be discoverable from microsoft365.com under Quick Access and My Content.

  • Microsoft 365 Copilot - Allows users to easily find, summarize, and learn important details about all content relevant to a user's natural language prompts, including ingested application data.

  • Microsoft Search - Allows data to be searchable for users in Microsoft Search, including Office.com, Bing at Work, and SharePoint.

Each of the above integrations has specific requirements on how external data must be injected into the Graph to become available. A full description of the individual requirements is available here.

In this tutorial, we will learn how to add application data to Microsoft Graph for use with Microsoft Search. In future tutorials, we will cover the other integrations as well.

Demo Application

This article series includes a demo application called "ODC with Graph Demo," available on ODC Forge. Be sure to download the version of the application that matches this article.

For this article, you need to install Version 0.1 from ODC Forge.

  • In the ODC Portal, go to Forge - All Assets.

  • Search for "ODC with Graph Demo".

  • Click on the Asset (Do not click on Install on the tile!).

  • Switch to the Version tab and click on Install next to Version 0.1.

Version 0.1 depends on other Forge components:

  • OAuthTokenExchange - An external logic library that helps retrieve access tokens easily. In this tutorial however we use an action to retrieve the discovery document from Microsoft Entra only.

  • Graph External Data Connections API - A connector library that integrates with Microsoft Graph API for external data connections.

What we build

The demo application lets users create and manage simple customer testimonials. These testimonials will be available in Microsoft Search, enabling users to search and filter them.

The screenshot above shows a sample search result for a testimonial. Notice the selected Testimonials tab, known as a vertical, which makes it easy to filter search results to show only testimonials. The URL redirects the user to the demo application to view the full details of the testimonial.

To achieve this, we need to follow several steps:

  • Register a connection and data schema - The connection serves as a container within the Graph API, storing content items (data). The schema defines the property names and data types of these content items.

  • Customize M365 Search & Intelligence - We need to specify how search results for our content items are displayed in M365 search and set up a vertical for easy filtering of search results.

  • Sync application data - Whenever a user adds, modifies, or deletes a testimonial in the demo application, we need to create, update, or delete the content item in Microsoft Graph.

Let's start with the prerequisites.

Prerequisites

Interacting with Microsoft Graph requires credentials from your Microsoft Entra tenant, with permissions to manage a data connection and content items.

Register Entra Application

In the Entra admin center, go to Applications - App Registrations and click on New registration.

  • Name - Graph External Data Connection ODC

  • Supported account types - Accounts in this organizational directory only

  • Click Register

From the Overview page, copy the values for:

  • Application (client) ID

  • Directory (tenant) ID

Next, select the Certificates & secrets menu. In the Client secrets tab, click on New client secret.

  • Description - Demo Application

  • Expires - 90 days (3 months)

  • Click Add

Copy the Value, which is the Client Secret. It will only be shown once.

Finally, we need to assign the permission to create and manage a Graph data connection.

In the API permissions menu, click Add permission.

  • Select Microsoft Graph and Application permission.

  • Search for and select ExternalConnection.ReadWrite.OwnedBy permission.

  • Search for and select ExternalItem.ReadWrite.OwnedBy permission.

  • Click Add permission.

  • Remove the default User.Read permission.

This permission requires an administrator to grant admin consent.

  • Click Grant admin consent <your domain> and confirm the dialog.

With your TenantId, ClientId, and ClientSecret, you can now set up the demo application's settings.

Demo Application Settings

In ODC Portal - Apps, choose the ODC with Graph Demo application.

In the configuration tab, enter the following values:

  • EntraTenantId - Directory (tenant) ID.

  • EntraClientID - Application (client) ID.

  • EntraClientSecret - Secret value copied after creating a new secret.

Leave the fourth setting, ConnectionId, with its default value. We'll explore this one shortly.

Run the Demo Application

With the settings defined, you can now run the demo application (make sure to grant your user the ODCwithGraphDemo role first).

On the start screen, you will see an alert box notifying you that your Microsoft Graph tenant is not yet configured.

Click the Configure Data Connection button to register a data connection and a data schema. This process can take up to 15 minutes to complete. We will explore the details of connection and schema registration shortly. For now, we just want to be able to add some testimonials to the demo application.

Once the connection and data schema is ready you can add your first testimonal.

When you Save, the testimonial is stored in the Testimonial entity and then added to Microsoft Graph. Similarly, if you update or delete an existing testimonial, it is updated or deleted in Microsoft Graph.

💡
If you encounter error messages, check your Entra application API permissions and the application settings for EntraClientId, EntraClientSecret, and EntraTenantId.

After you create some testimonials, try searching for one of the company names or testimonial texts in Microsoft 365. You will notice that none of your testimonials appear yet. This is because, although we can already add data to Microsoft Graph, we still need to complete some final configuration steps in Microsoft 365 Search & Intelligence and enable our connection.

Microsoft 365 Search & intelligence Configuration

Microsoft 365 Search & Intelligence can be accessed through the Microsoft 365 admin center under Settings.

Here, we configure the vertical—the tab in the search bar that lets you filter testimonials only—how testimonials are displayed in the search results, and finally, we activate our connection.

Add a Vertical

A vertical is a tab on the search results page that shows results from a specific content source, like our injected testimonial data. It helps users to filter their search on a particular category of information, making the results more relevant. Some of the default verticals are

  • All - shows results from all sources

  • Files - shows documents and files

  • People - shows user profiles

  • Sites - shows Sharepoint sites

In Search & intelligence settings switch to the Customizations tab and select Verticals from the menu options.

  • Click Add

  • Name - Testimonials

  • Click Next

  • In the Select a content source screen select Connectors and then the configured Testimonials data source.

  • Click Next

  • Skip the Add a query screen by clicking Next

  • Skip the Filters screen by clicking Next

  • Review the vertical and click Add Vertical

Back in the Verticals list select the checkbox next to Testimonials and click the Enable button in the top menu.

Add a Result type

Result types are a customization feature that controls how search results are displayed based on specific conditions, such as the content source (like our injected testimonial data), but not limited to that. With a result type, you configure an Adaptive Card, a user interface widget that is bound to result data, which defines how a search result is shown to the user in Microsoft Search. It is a powerful way to enhance the user experience for your custom data.

In Search & intelligence settings switch to the Customizations tab and select Result types from the menu options.

  • Click Add

  • Name - Testimonial

  • Click Next

  • Content Source - Testimonials (the name of our External Data Connections container)

  • Click Next

  • Skip the Set rules for the result type screen by clicking Next.

  • In the Design your layout screen paste the following Adaptive Card template to the text area.

{
    "type": "AdaptiveCard",
    "version": "1.3",
    "body": [
        {
            "type": "ColumnSet",
            "columns": [
                {
                    "type": "Column",
                    "width": "auto",
                    "items": [
                        {
                            "type": "Image",
                            "url": "https://res.cdn.office.net/midgard/versionless/defaultmrticon.png",
                            "horizontalAlignment": "center",
                            "size": "small"
                        }
                    ],
                    "horizontalAlignment": "center"
                },
                {
                    "type": "Column",
                    "width": "stretch",
                    "items": [
                        {
                            "type": "ColumnSet",
                            "columns": [
                                {
                                    "type": "Column",
                                    "width": "auto",
                                    "items": [
                                        {
                                            "type": "TextBlock",
                                            "text": "[${company}](${url})",
                                            "weight": "bolder",
                                            "size": "medium",
                                            "maxLines": 3,
                                            "color": "accent"
                                        }
                                    ],
                                    "spacing": "none"
                                }
                            ],
                            "spacing": "small"
                        },
                        {
                            "type": "TextBlock",
                            "text": "[${url}](${url})",
                            "spacing": "small",
                            "weight": "bolder",
                            "color": "dark"
                        },
                        {
                            "type": "Container",
                            "items": [
                                {
                                    "type": "TextBlock",
                                    "text": "**${lastUpdatedByName}** modified {{DATE(${lastUpdatedOn})}}",
                                    "spacing": "small",
                                    "$when": "${lastUpdatedByName!='' && lastUpdatedOn!=''}"
                                },
                                {
                                    "type": "TextBlock",
                                    "text": "Modified on {{DATE(${lastUpdatedOn})}}",
                                    "spacing": "small",
                                    "$when": "${lastUpdatedByName=='' && lastUpdatedOn!=''}"
                                },
                                {
                                    "type": "TextBlock",
                                    "text": "Modified by __${lastUpdatedByName}__",
                                    "spacing": "small",
                                    "$when": "${lastUpdatedByName!='' && lastUpdatedOn==''}"
                                }
                            ],
                            "spacing": "small"
                        },
                        {
                            "type": "TextBlock",
                            "text": "${ResultSnippet}",
                            "maxLines": 2,
                            "wrap": true,
                            "spacing": "small"
                        }
                    ],
                    "spacing": "medium"
                }
            ]
        }
    ],
    "$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
    "$data": {
        "lastUpdatedOn": "2019-09-25T06:08:39Z,SHORT",
        "ResultSnippet": "Wonderful work. Very helpful.",
        "lastUpdatedByName": "Stefan Weber",
        "url": "https://without.systems",
        "company": "without.systems"
    }
}

Notice the ${property} elements in the Adaptive Card that connect individual properties of the defined data schema of a testimonial to the card. The ${ResultSnippet} is a special system property that links the inserted content value and also provides search highlighting.

  • Click Next

  • Review the Result type and click Add result type

💡
It may take several hours for a new result type to become available.

Activate Connection

The final step is to activate our connection.

In the Search & Intelligence settings, switch to the Data Sources tab.

  • Click on Include Connector Results in the Required Actions column of the Testimonials connection and confirm the dialog.
💡
After activating the connection, it may take a few minutes before you can search for testimonials and filter using the Testimonial vertical.

By the end of this step, you should be able to search for testimonials in Microsoft Search.

Lets walk through the demo implementation details.

Demo Application Walkthrough

In this walkthrough, we will explore the key implementation details. Be sure to check the comments in the various server actions as well.

In ODC Studio, open the ODC with Graph Demo application.

Data Connection and Schema Registration

In the Logic tab, open the RegisterGraphDataConnection action under Server Actions - Graph. This action was executed when you clicked the Configure Data Connection button on the demo application's start screen.

  • GetEntraAccessToken - Retrieves an access token from Entra via OAuth client-credentials flow.

The demo application requests a new access token every time it interacts with Microsoft Graph. In a production environment, you would set up a token cache to request a new token only when the previous one has expired.

  • Graph_CreateExternalConnection - This action is provided by the Graph External Data Connections API connector library and creates a new data connection with

    • id - the short name of the data connection

    • name - the display name of the data connection

    • description - additional description of the data connection.

  • Graph_CreateOrUpdateConnectionSchema - This action registers a data schema for the data connection.

A data schema is a set of properties with its data type that represent the content item’s (testimonal) metadata. A single property definition in JSON looks like this.

{
  "name": "company",
  "type": "string",
  "isSearchable": true,
  "isRetrievable": true,
  "isQueryable": true,
  "labels": [
     "title"
  ],
  "aliases": []
}

Besides the property name, the definition specifies if you can search for the value (isSearchable), if the property can be used in KQL queries (isQueryable), or if the property can be used in result types (isRetrievable). Additionally, you can map properties to well-known labels of the Microsoft Search Index, like title in the example above. More information can be found in the documentation.

Ingest Content Item

In the Logic tab, open the IngestContent action under Server Actions - Graph. This action is triggered by the SaveTestimonial action after a testimonial is created or updated. It creates or updates the content item in Graph.

The Graph_CreateOrUpdateItem action at the bottom of the action flow expects a serialized JSON structure as Request parameter. The reason is that the object you have to pass to the Graph endpoint is dynamic based on the schema you defined.

Inspect the Item structure in Data - Structures - Content. The structure consists of the following parts

  • Id - the Id is the unique identifier of the content item, the primary key value of our testimonal.

  • Properties - Attributes that correspond to the schema we registered for the connection

  • AccessControlList - Permissions that specify who can view/read the content item. ACLs are not covered in this tutorial though.

  • Content - In our case the content object will hold the value of the testimonal text.

Lets break the server action flow down

  • Testimonal_Get - This server action retrieves the full details of the requested testimonial.

  • Id - We assign the testimonal identifier to the Id property of the Item structure.

  • Properties - Assign the metadata values of the testimonials to the Item Properties objects.

  • Content - Assign the text of the testimonial as content object of type text to the Item Content object.

  • AccessControlListEntry - Access Control List that define who has access to the content item are not covered in this tutorial, but you must add at least one access control list entry. We take the easy route here and grant permission to everyone.

  • SerializeItem - Serialize the Item structure and request parameter for the Graph_CreateOrUpdateItem action.

This concludes the demo application walkthrough. Explore the rest of the demo application and the configuration options in Microsoft 365 Search & intelligence. The final step is to cleanup everything.

Cleanup

Once you have finished exploring data ingestion to Microsoft Graph, you should perform the following cleanup steps in Microsoft 365 Search & Intelligence:

  • Remove the Result type

  • Remove the Vertical

  • Delete the Data Connection

Notes and Recommendations

The demo application performs all actions synchronously, triggered from the frontend. In a real use case, you should offload data ingestion to Microsoft Graph to a workflow.

In addition, I recommend implementing a queue that stores an entry for each data item (testimonial) to create, update, and delete, and removes the entry from the queue once the data ingestion operation succeeds. This way, you minimize the risk of application data and graph data becoming inconsistent.

The Graph External Data Connections API library integrates with the endpoints to create a connection and its schema. However, since this is a one-time operation, you can also use Postman or a curl command instead of doing it in your application.

Be patient when changing Search & Intelligence settings like verticals or result types in the M365 admin center. In some cases, it may take several hours to see and use a new configuration due to caching.

Summary

In this tutorial, we built a simple integration with Microsoft Graph and ingested OutSystems application data into the Search index. Having a central way to search for both Microsoft 365 data and OutSystems application data is a great benefit for users who would otherwise have to switch between applications. Adding application data to Microsoft Search also provides additional benefits that I will explore in later tutorials.

I hope you enjoyed it and would appreciate your feedback.

0
Subscribe to my newsletter

Read articles from Stefan Weber directly inside your inbox. Subscribe to the newsletter, and don't miss out.

Written by

Stefan Weber
Stefan Weber

As a seasoned Senior Director at Telelink Business Services EAD, a leading IT full-service provider headquartered in Sofia, Bulgaria, I lead the charge in our Application Services Practice. In this role, I spearhead the development of tailored software solutions using no-code/low-code platforms and cutting-edge cloud-ready/cloud-native solutions based on the Microsoft .NET stack. Throughout my diverse career, I've accumulated a wealth of experience in various capacities, both technically and personally. The constant desire to create innovative software solutions led me to the world of Low-Code and the OutSystems platform. I remain captivated by how closely OutSystems aligns with traditional software development, offering a seamless experience devoid of limitations. While my managerial responsibilities primarily revolve around leading and inspiring my teams, my passion for solution development with OutSystems remains unwavering. My personal focus extends to integrating our solutions with leading technologies such as Amazon Web Services, Microsoft 365, Azure, and more. In 2023, I earned recognition as an OutSystems Most Valuable Professional, one of only 80 worldwide, and concurrently became an AWS Community Builder.