Beyond Agents: Compound AI Systems Done

Overview

In the previous journey, you helped Jordan Miller build a powerful, stateful AI agent. It can reason, use tools, and remember conversations. However, as Jordan began deploying this assistant to handle critical production incidents, they noticed a challenge: Reliability at Scale.

When an agent has complete autonomy, its path is “probabilistic.” It might choose to call a tool, or it might just explain why it thinks it doesn’t need to. In high-stakes environments, you often need Determinism—guaranteed steps that happen every time, wrapped around the AI’s “Smart Brain.”

In this module, we transition from building Agents to building Compound AI Systems.

System Anatomy

We are shifting from single-agent patterns to Compound AI Systems. In this path, we will build out the core concepts of the Agent Framework Workflows.

The Shift: From Models to Systems

Recent research (notably from Berkeley BAIR) highlights a critical shift in the industry: state-of-the-art results for complex tasks are no longer achieved by just “better models,” but by better architectures.

The “Safe Shell” Pattern

Jordan’s new strategy is the Hybrid AI Pattern:

1. The Shell (C#): Deterministic code that handles high-stakes data fetching, security checks, and final routing.
2. The Brain (AI): Probabilistic reasoning used only for the parts of the task that are too complex for hard-coded rules.

graph TD
    subgraph System [The Compound System]
        Step1[Fetch Telemetry - C#]
        Step2[Analyze - AI Agent]
        Step3[Escalate - C#]
        
        Step1 --> Step2
        Step2 --> Step3
    end
    style System fill:#fff,stroke:#6366f1,stroke-width:2px,stroke-dasharray: 5 5

Building the Compound System

Jordan knew that while a single AI agent is powerful, it isn’t enough to handle a chaotic production incident alone. When Jordan proposed wrapping the agent in deterministic code to Taylor Vance, Taylor asked the critical question: “If the agent isn’t controlling the flow anymore, what is? How do we orchestrate these steps without writing a massive, fragile state machine?”

The answer is Microsoft Agent Framework Workflows. This orchestration layer allows Jordan to build an intelligent, multi-step assembly line that perfectly balances the rigid rules of incident management with the flexible reasoning of an LLM.

Taylor and Jordan realized a fundamental distinction:

By leveraging the framework, Jordan can take advantage of several enterprise-grade features:

Core Concepts

Here is how Jordan translates these concepts into grounded code to build a reliable incident triage pipeline:

// 1. Define specialized executors (custom C# logic and AI Agents)
Executor fetchTelemetry = new FetchTelemetryExecutor();
AIAgent triageAgent = ...; // The "Smart Brain"
Executor escalationEngine = new EscalationEngineExecutor();

// 2. Tie executors and edges together into a directed graph
WorkflowBuilder builder = new(fetchTelemetry);
builder.AddEdge(fetchTelemetry, triageAgent);
builder.AddEdge(triageAgent, escalationEngine);
Workflow workflow = builder.Build();

// 3. Execute the workflow with streaming observability
await using StreamingRun run = await InProcessExecution.RunStreamingAsync(workflow, "checkout failures");

await foreach (WorkflowEvent evt in run.WatchStreamAsync())
{
    if (evt is AgentResponseUpdateEvent update) Console.Write(update.Update.Text);
}

Multi-Agent Patterns

With the WorkflowBuilder, you can easily codify common patterns for coordinating multiple components. Instead of one agent doing everything, you can orchestrate specialized units:

1. Sequential (The Assembly Line)

Agents or code executors pass data linearly. Good for rigid processes like “Translate → Summarize → Email.”

2. Concurrent (The Panel of Experts)

Run multiple agents at once to get diverse perspectives or process data faster.

3. Handoffs (The Support Desk)

A “Router” agent determines which specialized agent is best suited for the current task.

Summary & Next Steps

You’ve moved from thinking about who the agent is to how the system flows. By adopting the Compound AI mindset, you’ve given Jordan the tools to build systems that are as reliable as they are intelligent.

In the next tutorial, we will build Jordan’s first professional assembly line: a multi-step Incident Triage Workflow.