The Definitional Chaos
Everyone in the AI industry is talking about agents. Nobody agrees on what the word means.
OpenAI CEO Sam Altman describes agents as systems that do “meaningful work with minimal input.” Microsoft CEO Satya Nadella frames agents as “the new application form factor” — replacements for traditional apps. Amazon’s Jeff Bezos talks about agents as “connected action,” systems that bridge AI reasoning with real-world execution. Anthropic describes agents as systems where the model operates in a loop, making decisions about tool use at each step. Google, Meta, and every startup with a landing page have their own variations.
The ambiguity is not just semantic. It creates real confusion for engineering teams trying to make architectural decisions. When a product manager asks “should we build an agent?” — what exactly are they asking for? When a conference talk claims to demonstrate an “agentic system,” what should you expect? When Microsoft releases an “Agent Framework,” what distinguishes it from Semantic Kernel’s existing orchestration capabilities?
Without a shared definition, these conversations go in circles. So here is an engineer’s attempt at clarity.
A Practical Engineering Definition
Strip away the marketing language, and an AI agent has four essential components:
1. A language model — The reasoning engine that interprets goals, understands context, and decides what to do next.
2. Tools — Functions the model can invoke to interact with the outside world. A database query, an API call, a file operation, a web search. Without tools, the model can only generate text. With tools, it can take action.
3. An autonomous decision loop — The model operates in a cycle: observe the current state, decide on an action, execute it via a tool, observe the result, and decide the next action. This loop runs without human intervention for each step. The model is not waiting for a new prompt after each response — it is driving the process.
4. Memory — The ability to maintain context across the decision loop. At minimum, this means the conversation history that tracks what has been tried and what results were observed. More sophisticated agents have persistent memory across sessions, knowledge retrieval, and state management.
The definition: An AI agent is a system that combines an LLM with tools, an autonomous decision loop, and memory to accomplish tasks with minimal human intervention per step.
This definition is intentionally functional. It does not require any specific architecture, framework, or model provider. It describes what an agent does, not how it is implemented.
The Spectrum: From Chatbot to Autonomous Agent
Agents are not a binary category. They exist on a spectrum, and understanding where your system falls on that spectrum directly informs your architecture.
Level 0: Simple Chatbot
A user sends a message. The model responds. No tools, no autonomy, no persistent state beyond the conversation window. The application is a wrapper around a completion API.
User → LLM → Response → UserThis is where most .NET AI integrations start, and it is perfectly valid for many use cases. Customer support FAQ, content generation, summarization — these tasks do not need tools or autonomy.
Level 1: Tool-Using Assistant
The model has access to functions it can call. When the user asks a question, the model can decide to call a tool — query a database, check an API, search a knowledge base — and incorporate the result into its response. But the user drives each interaction. One request, one response (possibly with tool calls in between).
User → LLM → [Tool Call → Result]* → Response → UserThis is where Semantic Kernel’s function calling shines. You register plugins, enable AutoInvokeKernelFunctions, and the model decides when tools are needed:
// Level 1: Tool-using assistant
var kernel = Kernel.CreateBuilder()
.AddAzureOpenAIChatCompletion(deploymentName, endpoint, credential)
.Build();
kernel.Plugins.AddFromType<InventoryPlugin>();
kernel.Plugins.AddFromType<OrderPlugin>();
var settings = new OpenAIPromptExecutionSettings
{
ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions
};
// The model will call tools as needed to answer the user's question
var result = await kernel.InvokePromptAsync(
"What is the current stock level for SKU-4892?", new(settings));Level 2: Semi-Autonomous Agent
The model receives a higher-level goal rather than a specific question. It plans a sequence of steps, executes them using tools, evaluates intermediate results, and adapts its approach. However, it pauses at predefined checkpoints for human approval — before making changes to production data, before executing financial transactions, before sending external communications.
User (goal) → LLM → [Plan → Tool Call → Observe → Decide]* → [Human Checkpoint]* → ResultThis is the practical sweet spot for most production systems today. The agent has autonomy over routine steps but defers to humans for high-stakes decisions. OpenAI’s Operator operates primarily at this level, pausing for user confirmation when facing payment screens or ambiguous choices.
Level 3: Fully Autonomous Agent
The model receives a goal and executes it end-to-end without human intervention. It handles errors, adapts to unexpected results, and makes all decisions independently. This level requires high model reliability, comprehensive guardrails, and thorough testing — because there is no human in the loop to catch mistakes.
User (goal) → LLM → [Plan → Tool Call → Observe → Decide → Adapt]* → ResultFully autonomous agents are technically possible today but rare in production. The trust and safety requirements are high, and the consequences of model errors compound with each autonomous step. Most production deployments that claim Level 3 actually have hidden human oversight or are operating in low-risk domains.
How This Maps to .NET Tooling
The .NET ecosystem provides building blocks for every level of this spectrum.
Semantic Kernel plugins = Tools. Every [KernelFunction] you register is a tool the agent can use. The quality of your agent depends directly on the quality of your tools — clear descriptions, typed parameters, informative error messages, and predictable behavior.
Semantic Kernel planner = Decision loop. The planning infrastructure in SK — whether using the function-calling planner or the Handlebars planner — is the mechanism by which the model decides what to do next. Combined with AutoInvokeKernelFunctions, this creates the autonomous loop.
Microsoft Agent Framework = Full agent orchestration. The Agent Framework builds on SK to provide ChatCompletionAgent, AgentGroupChat for multi-agent coordination, agent handoff protocols, and structured orchestration patterns. This is where you build Level 2 and Level 3 systems.
AutoGen = Multi-agent conversations. Microsoft Research’s AutoGen framework provides a different abstraction for multi-agent systems, focused on agent-to-agent dialogue and collaborative task solving. Worth evaluating for scenarios where multiple specialized agents need to coordinate.
A basic Semantic Kernel agent — the starting point for Levels 1 and 2:
// A minimal agent with tools and an autonomous loop
var agent = new ChatCompletionAgent
{
Name = "ResearchAgent",
Instructions = """
You are a research assistant. Use the available tools to find information
and synthesize it into clear answers. If a tool call fails, try an
alternative approach before asking the user for help.
""",
Kernel = kernel,
Arguments = new KernelArguments(
new OpenAIPromptExecutionSettings
{
ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions
})
};
// The agent loop — iterate until the task is complete
var chat = new AgentGroupChat(agent);
chat.AddChatMessage(new ChatMessageContent(AuthorRole.User,
"Find the latest .NET 9 release notes and summarize the AI-related changes."));
await foreach (var message in chat.InvokeAsync())
{
Console.WriteLine($"{message.AuthorName}: {message.Content}");
}Why the Definition Matters for Architecture
The reason to care about definitions is not pedantic. It is architectural. You design a different system for a chatbot than you do for an agent, and confusion between the two leads to either over-engineering or under-engineering.
State management changes. A chatbot can be stateless between requests. An agent needs durable state — what tools have been called, what results were observed, what the current plan looks like, where in the decision loop execution currently sits. If the process crashes mid-execution, can it resume? For a chatbot, that question is irrelevant. For an agent, it is a core architectural requirement.
Error handling changes. A chatbot error produces a bad response. An agent error produces a bad action — potentially against a real system. Error handling in agents means tool-level retry logic, plan-level adaptation, and system-level circuit breakers. The orchestration patterns guide covers these patterns in detail.
Testing changes. You can test a chatbot by evaluating output quality. Testing an agent requires evaluating behavior — the sequence of tool calls, the decisions made at each step, the handling of error conditions, the respect for guardrails. Agent testing is closer to integration testing than unit testing.
Cost changes. A chatbot makes one completion call per user interaction. An agent might make ten, twenty, or fifty calls to complete a single task — each one consuming tokens. Cost modeling for agents requires understanding the typical loop depth, the token cost of tool call descriptions, and the multiplication effect of reasoning across multiple steps.
Security changes. A chatbot has a limited attack surface: prompt injection might produce bad text. An agent that can call tools — especially tools that modify data, send emails, or interact with external systems — has a dramatically larger attack surface. Every tool is a potential vector for prompt injection to cause real-world harm.
What .NET Teams Should Build Today vs Wait For
The practical guidance depends on where your current AI integration sits on the spectrum.
If you are at Level 0 (chatbot), move to Level 1 (tool-using assistant). This is the highest-value step. Adding even one tool — a database lookup, an API call, a knowledge search — transforms a chatbot from a text generator into a genuinely useful assistant. Semantic Kernel makes this straightforward with plugin registration and automatic function invocation.
If you are at Level 1, evaluate whether Level 2 would solve real problems. Not every application benefits from autonomy. Ask: are your users performing repetitive multi-step workflows that the model could handle? Are there processes where the model could plan and execute a sequence of tool calls more efficiently than a human clicking through UI screens? If yes, design a semi-autonomous agent with clear human-in-the-loop checkpoints.
Do not jump to Level 3 without compelling justification. Fully autonomous agents require significantly more engineering investment in reliability, safety, observability, and testing. The payoff is real for the right scenarios, but the cost is high. Start with Level 2 and expand autonomy gradually as you build confidence in your guardrails.
Invest in the foundation regardless of level. Good tool design, clear system prompts, structured logging, cost tracking, and evaluation frameworks serve you at every level of the spectrum. These are not agent-specific concerns — they are AI engineering fundamentals that become increasingly important as autonomy increases.
The industry will eventually converge on a shared definition. Until then, a practical engineering taxonomy — chatbot, tool-using assistant, semi-autonomous agent, fully autonomous agent — gives you the vocabulary to make clear architectural decisions and set accurate expectations with stakeholders.