From CLU to Semantic Kernel: Building a Secure, Intelligent Teams Bot

In the AI and natural language processing world, building a solid chat interface can be tricky. When we started with Azure's Conversational Language Understanding (CLU), we wanted to make a system that could really get what users were saying and respond well. We did this by linking the predicted "TopIntent" directly to certain handler methods. This way, we made it easier to figure out what users wanted and do the right thing, making the interaction smooth.

public async Task ProcessIntentAsync(ITurnContext turnContext, CancellationToken cancellationToken)
{
    var query = turnContext.Activity.Text; // "What's the busiest queue today?"
    var userId = turnContext.Activity.From.Id;
    var tenantId = await _tenantResolver.GetTenantIdForUser(userId);

    // Get intent from Azure CLU
    var cluResponse = await _conversationsClient.AnalyzeConversationAsync(query);

    // Check if user has permission for this intent
    if (!await _permissionChecker.HasPermissionForIntent(userId, tenantId, cluResponse.TopIntent))
    {
        await turnContext.SendActivityAsync($"You don't have permission to access {prediction.TopIntent}.");
        return;
    }

    // Process the intent using a factory pattern
    var intentProcessor = _intentFactory.GetProcessor(cluResponse.TopIntent);
    if (intentProcessor != null)
    {
        await intentProcessor.ProcessAsync(turnContext, cluResponse.Entities);
    }
    else
    {
        await turnContext.SendActivityAsync("I'm not sure how to help with that.");
    }
}

Azure CLU Limitations: Why We Needed More

While this implementation worked, we quickly encountered several limitations:

Rigid Parameter Extraction

We had to define explicit entities for each parameter and handle missing entities manually:

private async Task HandleAgentPerformanceIntent(ITurnContext turnContext, List<Entity> entities)
{
    // Required entity extraction
    var agentEntity = entities.FirstOrDefault(e => e.Category == "AgentName");
    if (agentEntity == null)
    {
        await turnContext.SendActivityAsync("Please specify an agent name.");
        return;
    }

    var dateRangeEntity = entities.FirstOrDefault(e => e.Category == "DateRange");
    if (dateRangeEntity == null)
    {
        await turnContext.SendActivityAsync("Please specify a time period.");
        return;
    }

    // Continue with handler...
}

Maintenance Burden

Each new capability required extensive changes across multiple systems:

1. Creating and training a new CLU intent with dozens of examples

2. Adding entity definitions for any parameters

3. Adding a new case to our switch statement

4. Implementing a new handler method with parameter extraction

5. Adding permission checks specific to that intent

Contextual Amnesia

The bot couldn't maintain conversation context or understand follow-up questions:

First query: "What was my busiest queue last month?"
    Bot processes GetBusiestQueue intent successfully

Follow-up query: "And what about December?"
    CLU has no context from previous query, so this likely fails or matches a different intent entirely

Semantic Kernel Approach: Dynamic Function Selection

With Semantic Kernel, we eliminated hard-coded intent matching in favor of AI-powered function selection:

public class QueueAnalyticsPlugin
{
    [KernelFunction, Description("Gets statistics for the busiest queue within a specified time period.")]
    [Parameter("dateRange", "The date range to analyze (e.g., 'last month', 'yesterday', etc.)")]
    public async Task<string> GetBusiestQueue(string dateRange, KernelArguments arguments)
    {
        // Security and implementation...
    }

    [KernelFunction, Description("Compares call volumes between two queues over a time period.")]
    [Parameter("queue1", "The first queue to compare")]
    [Parameter("queue2", "The second queue to compare")]
    [Parameter("dateRange", "The date range to analyze")]
    public async Task<string> CompareQueues(string queue1, string queue2, string dateRange, KernelArguments arguments)
    {
        // Security and implementation...
    }
}

This approach allows the LLM to understand the user's intent and select the most appropriate function, even with varied phrasing and follow-up questions, while maintaining the same security guarantees.

The Semantic Kernel Plugin Architecture

Semantic Kernel allowed us to create a more powerful bot by organizing functionality into plugins that the LLM can intelligently select based on user intent.

public class SemanticKernelBot
{
    private readonly Kernel _kernel;
    private readonly IPermissionService _permissionService;

    public SemanticKernelBot(Kernel kernel, IPermissionService permissionService)
    {
        _kernel = kernel;
        _permissionService = permissionService;

        // Register all plugins
        RegisterPlugins();
    }

    private void RegisterPlugins()
    {
        // Register our domain-specific plugins
        _kernel.Plugins.AddFromObject(new QueueAnalyticsPlugin(_permissionService), "QueueAnalytics");
        _kernel.Plugins.AddFromObject(new AgentPerformancePlugin(_permissionService), "AgentPerformance");
        _kernel.Plugins.AddFromObject(new CallStatisticsPlugin(_permissionService), "CallStatistics");
    }
}

Plugin Implementation with Security Checks

Each plugin function automatically enforces security checks before accessing any data:

public class QueueAnalyticsPlugin
{
    private readonly IMediator _mediator;
    private readonly IPermissionService _permissionService;

    public QueueAnalyticsPlugin(IMediator mediator, IPermissionService permissionService)
    {
        _mediator = mediator;
        _permissionService = permissionService;
    }

    [KernelFunction, Description("Gets statistics for the busiest queue within a specified time period.")]
    [Parameter("dateRange", "The date range to analyze (e.g., 'last month', 'yesterday', etc.)")]
    public async Task<string> GetBusiestQueue(string dateRange, KernelArguments arguments)
    {
        // Extract tenant ID and user ID from context
        var tenantId = arguments["tenantId"] as string;
        var userId = arguments["userId"] as string;

        // Check permissions - this is the security boundary
        if (!await _permissionService.HasPermission(userId, tenantId, Permission.ViewQueueAnalytics))
        {
            return "You don't have permission to view queue analytics. Please contact your administrator.";
        }

        // Convert natural language date range to DateTime values
        var (startDate, endDate) = ParseDateRange(dateRange);

        // Reuse existing mediator query
        var result = await _mediator.Send(new GetBusiestQueueQuery(tenantId, startDate, endDate));

        return $"The busiest queue between {startDate:d} and {endDate:d} was {result.QueueName} with {result.CallCount} calls.";
    }
}

Reusing Existing Business Logic

A key advantage is that each plugin simply calls our existing mediator-based business logic:

// Inside our plugin
var result = await _mediator.Send(new GetBusiestQueueQuery(tenantId, startDate, endDate));

This means:

1. No duplication of business logic

2. Existing security checks remain in place

3. All tenant isolation guarantees are maintained

Natural Language Understanding with Context

Semantic Kernel enables the bot to maintain context across the conversation:

public async Task HandleMessageAsync(ITurnContext turnContext)
{
    var userQuery = turnContext.Activity.Text;
    var userId = turnContext.Activity.From.Id;

    // Get chat history for context
    var chatHistory = await _chatHistoryRepository.GetForUser(userId);

    // Execute query through Semantic Kernel with context
    var kernelArguments = new KernelArguments
    {
        ["userId"] = userId,
        ["tenantId"] = await _tenantResolver.GetTenantIdForUser(userId),
        ["history"] = chatHistory.ToString()
    };

    var result = await _kernel.InvokePromptAsync(userQuery, kernelArguments);
    await turnContext.SendActivityAsync(result.ToString());
}

This allows natural follow-up questions:

User: "What was my busiest queue last month?"
Bot: "The busiest queue in January was Support with 1,245 calls."
User: "What about December?"
Bot: "In December, the busiest queue was Sales with 982 calls."

The Best of Both Worlds: Hybrid Approach

While Semantic Kernel excels at conversation, CLU is still better for specific structured outputs like Adaptive Cards:

public async Task HandleMessageAsync(ITurnContext turnContext)
{
    var userQuery = turnContext.Activity.Text;

    // First check if this is a request for an Adaptive Card
    var cluResult = await _languageClient.PredictAsync(userQuery);

    if (cluResult.TopIntent == "MostBusyAgent" && cluResult.TopScore > 0.7)
    {
        // Use deterministic card generator
        var card = await _cardGenerator.CreateMostBusyAgentAdaptiveCard(cluResult.Entities);
        await turnContext.SendActivityAsync(MessageFactory.Attachment(card));
        return;
    }

    // Otherwise, use Semantic Kernel for natural language handling
    await HandleWithSemanticKernel(turnContext);
}

Multi-tenancy and Security: Never Compromised

Every function implements permission checks before accessing any data:

[KernelFunction]
public async Task<string> GetAgentPerformance(string agentName, string dateRange, KernelArguments arguments)
{
    var tenantId = arguments["tenantId"] as string;
    var userId = arguments["userId"] as string;

    // Security check #1: User permission
    if (!await _permissionService.HasPermission(userId, tenantId, Permission.ViewAgentData))
    {
        return "You don't have permission to view agent performance data.";
    }

    // Security check #2: Data boundary - ensure agent belongs to tenant
    if (!await _agentRepository.BelongsToTenant(agentName, tenantId))
    {
        return $"No agent named '{agentName}' was found in your organization.";
    }

    // Proceed with data retrieval...
}

These checks ensure:

• Users can only access data they're authorized to view

• No cross-tenant data leakage can occur

• All requests are properly scoped to the user's tenant

Conclusion

Transitioning to Semantic Kernel transformed the Clobba Teams Bot into a more powerful, context-aware assistant while maintaining strict security guarantees.

The key advantages:

1. Powerful natural language understanding that handles variations in phrasing

2. Contextual awareness that maintains conversation state

3. Security-first approach with permission checks in every function

4. Tenant isolation guarantees to prevent data leakage

5. Simplified development through plugin architecture

By combining the strengths of Semantic Kernel and Azure CLU, we've created a bot that's both more powerful and more secure—giving users a modern AI experience without compromising on security.