Building Your Own Enterprise AI Assistant: Architecture Patterns with RAG and Tool Calling

Moving From Passive Chatbots to Actionable Assistants

In our previous guide, we demonstrated how Retrieval-Augmented Generation (RAG) grounds LLMs with private company policies using vector search. However, a pure RAG pipeline has a fundamental limitation: it is strictly read-only.

If an employee asks: "What is our annual peripheral reimbursement policy?", RAG retrieves the handbook and provides the correct answer ($500/year). But if the employee asks: "How much have I spent on hardware this year, and can you file an expense report for my new $300 monitor?", RAG alone fails.

To solve real enterprise problems, an AI assistant requires both Memory (Knowledge Retrieval via RAG) and Hands (Execution Capabilities via Tool/Function Calling).

In this deep-dive guide, we explore the core architectural patterns for building a production-grade Enterprise AI Assistant in modern C# (.NET 10) using Microsoft Semantic Kernel, unifying vector retrieval with secure API function calling.

The 3 Fundamental Architectural Patterns

When engineering an enterprise AI assistant, teams typically evolve across three distinct architectural patterns:

Step-by-Step Implementation in C# (.NET 10)

Let's build an enterprise assistant that can answer policy questions from RAG, check employee balances via API, and submit expense claims.

1. Defining Enterprise Plugins as Tools

In Semantic Kernel, tools are standard C# methods decorated with [KernelFunction] and explicit descriptions that tell the LLM when and how to invoke them:

using System.ComponentModel;
using Microsoft.SemanticKernel;

public class ExpenseManagementPlugin
{
    // Simulating transactional enterprise database
    private static readonly Dictionary<string, decimal> UserBalances = new()
    {
        ["EMP-1042"] = 150.00m // $150 already claimed out of $500 cap
    };

    [KernelFunction, Description("Retrieves the total expenses already reimbursed to an employee for the current calendar year.")]
    public string GetReimbursedAmount(
        [Description("The unique employee identification code, e.g., EMP-1042")] string employeeId)
    {
        if (UserBalances.TryGetValue(employeeId, out var spent))
        {
            return $"Employee {employeeId} has been reimbursed ${spent:F2} this year.";
        }
        return $"Employee {employeeId} has no recorded expense claims for the current year.";
    }

    [KernelFunction, Description("Submits an official employee peripheral reimbursement claim for manager approval.")]
    public string SubmitExpenseClaim(
        [Description("The employee ID")] string employeeId,
        [Description("The item description (e.g., UltraWide Monitor)")] string itemDescription,
        [Description("Total amount to claim in USD")] decimal amount)
    {
        // Enterprise guardrail: validate before booking
        if (amount > 500)
        {
            return $"Error: Single peripheral claims cannot exceed the $500 annual limit.";
        }

        // Book the transaction
        if (UserBalances.ContainsKey(employeeId))
            UserBalances[employeeId] += amount;
        else
            UserBalances[employeeId] = amount;

        return $"SUCCESS: Expense claim of ${amount:F2} for '{itemDescription}' submitted for {employeeId}. Reference ID: EXP-{Guid.NewGuid().ToString()[..8].ToUpper()}.";
    }
}

2. Packaging RAG Search as a Callable Kernel Plugin

Instead of forcing RAG retrieval on every prompt, we wrap our vector collection from our previous post into a tool:

using System.ComponentModel;
using Microsoft.Extensions.VectorData;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Embeddings;

public class CompanyPolicyPlugin
{
    private readonly IVectorStoreRecordCollection<ulong, DocumentChunk> _vectorCollection;
    private readonly ITextEmbeddingGenerationService _embeddingService;

    public CompanyPolicyPlugin(
        IVectorStoreRecordCollection<ulong, DocumentChunk> vectorCollection,
        ITextEmbeddingGenerationService embeddingService)
    {
        _vectorCollection = vectorCollection;
        _embeddingService = embeddingService;
    }

    [KernelFunction, Description("Searches official company policy handbooks and HR documentation for rules, allowances, and deadlines.")]
    public async Task<string> SearchCompanyPoliciesAsync(
        [Description("The semantic search query regarding policies, perks, or deadlines")] string query)
    {
        var queryVector = await _embeddingService.GenerateEmbeddingAsync(query);
        var searchResults = _vectorCollection.SearchAsync(queryVector, top: 1);

        string matchedContext = string.Empty;
        await foreach (var match in searchResults)
        {
            matchedContext += $"[Excerpt from {match.Record.Title}]: {match.Record.Content}\n";
        }

        return string.IsNullOrWhiteSpace(matchedContext) 
            ? "No matching policy documents found in the knowledge base." 
            : matchedContext;
    }
}

3. Orchestrating the Assistant with Auto Tool Invocation

Now, we configure the Kernel with automatic function calling enabled. The LLM automatically inspects tool schemas and decides when to search policies, check balances, or submit forms:

using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
using Microsoft.SemanticKernel.Connectors.Google;

// 1. Build Kernel and register plugins
var kernelBuilder = Kernel.CreateBuilder();
kernelBuilder.AddGoogleAIGeminiChatCompletion("gemini-2.5-flash", apiKey);

// Register both Knowledge (RAG) and Action (Tools)
kernelBuilder.Plugins.AddFromType<ExpenseManagementPlugin>("ExpensePlugin");
kernelBuilder.Plugins.AddFromObject(new CompanyPolicyPlugin(vectorCollection, embeddingService), "PolicyPlugin");

var kernel = kernelBuilder.Build();
var chatService = kernel.GetRequiredService<IChatCompletionService>();

// 2. Enable Automatic Tool Invocation
var promptSettings = new GeminiPromptExecutionSettings
{
    FunctionChoiceBehavior = FunctionChoiceBehavior.Auto() // SK handles the multi-turn tool loop automatically
};

// 3. Initialize Conversation History
var chatHistory = new ChatHistory();
chatHistory.AddSystemMessage("""
    You are an intelligent enterprise HR & Finance assistant.
    Current authenticated user: Employee ID: EMP-1042.
    
    Guidelines:
    1. If the user asks about policy details, use 'SearchCompanyPoliciesAsync'.
    2. If the user asks about personal expenses or balances, use 'GetReimbursedAmount'.
    3. Always verify policy allowances and remaining budget BEFORE filing an expense claim.
    4. Provide clear, professional confirmations with reference IDs.
    """);

// User requests a multi-step task requiring BOTH RAG and Tools
string userPrompt = "Can I claim a $280 4K monitor, and if so, how much budget do I have left? If eligible, please submit the claim for me.";
chatHistory.AddUserMessage(userPrompt);

Console.WriteLine($"User: {userPrompt}\n");

// Execute the assistant loop
var response = await chatService.GetChatMessageContentAsync(chatHistory, promptSettings, kernel);
Console.WriteLine($"Assistant: {response.Content}");

Execution Trace: How the Assistant Solved the Multi-Step Request

Here is what happened under the hood when the single prompt above was submitted:

Critical Enterprise Security & Safety Guardrails

Giving an LLM access to write APIs and company data without safeguards is a major liability. Implement these three essential security patterns:

Architecture Comparison: Chatbot vs. RAG vs. Agentic Assistant