A Developer’s Guide to Azure AI Agents

The Azure AI Agent Service is Microsoft’s enterprise-grade implementation of AI agents. It empowers developers to build, deploy, and scale sophisticated AI agents without managing the underlying infrastructure. Initially announced at the Microsoft Ignite conference in November 2024, this service is now available in a public preview.

Built on the same wire protocol as OpenAI’s Assistants API, developers can use OpenAI SDKs or Azure AI Foundry SDKs while adding enterprise features like enhanced security, compliance, and scalability. Let’s explore each service component and how they work together to create powerful AI applications.

Core Components of Azure AI Agents

Project

A project is your workspace within Azure AI Foundry that contains all your agent-related resources. It is the top-level container where you manage authentication, configure resources, and organize your agents. All communication with Azure OpenAI models, tools, and other Azure services is handled through project-level configurations. It is a logical boundary to deploy and provision all the resources related to your agent. You can think of a project as your development environment where all agent-related activities occur, from managing API connections to monitoring agent performance.

Agent

An agent is an autonomous AI entity that combines a language model with specific instructions and tools to perform specialized tasks. Each agent is defined by its model selection (GPT-4, Llama, or Mistral), customized instructions that shape its behavior, and tools that extend its capabilities. For example, you might create an agent specialized in data analysis by combining GPT-4 with Python coding capabilities and access to data visualization tools. Agents maintain consistent behavior across conversations and can work independently or collaborate with other agents to achieve complex goals. Azure AI Agents can utilize OpenAI or open-weight models such as Llama or Mistral as the LLMs. Every agent created within the Azure AI Foundry has a unique identifier.

Thread

A thread serves as the conversation container in Azure AI Agents, managing the flow of information between users and agents. It automatically handles context management, token windows and conversation history, ensuring that agents access relevant prior interactions while staying within model constraints. Threads can persist across multiple interactions, allowing for long-running tasks and complex workflows. When a user starts a conversation, the thread maintains the context and state, enabling coherent and contextual responses even in lengthy interactions. Threads are identified through a unique GUID, which can be used to refer to them during the execution of a workflow.

Message

Messages are basic communication units within threads, representing user inputs and agent responses. Each message can contain rich content, including text, file attachments, citations, and references to external resources. Messages are chronologically organized within a thread, building upon each other to create coherent conversations. When an agent processes a message, it can access the entire conversation history within the thread, enabling contextually appropriate responses that consider previous interactions.

Tools

Tools in Azure AI Agents extend an agent’s capabilities beyond basic conversation, enabling it to perform specific actions and access external resources. The service provides built-in tools like Code Interpreter for executing Python code, File Search for document analysis, and Bing Search for real-time web access. Additionally, you can integrate custom tools through Azure Functions or OpenAPI specifications. These tools are configurable at the agent level. They are executed within the context of specific runs, allowing agents to perform complex tasks like data analysis, content generation, or system integration.

Run

A run represents the execution lifecycle of an agent’s task within a thread. It receives a user message, processes it through the model, executes necessary tool calls, and generates responses. Runs can handle parallel function execution and maintain detailed step tracking for monitoring and debugging. Each run captures the complete interaction flow, including tool usage, making it valuable for understanding how agents make decisions and handle tasks.

The code snippet below brings everything together through a simple workflow:

# Create an agent with specific capabilities
agent = project_client.agents.create_agent(
 model="gpt-4o",
 name="data-analyst",
 instructions="You are a data analysis expert who helps users understand complex datasets",
 tools=[code_interpreter.definitions]
)

# Create a thread for a new conversation
thread = project_client.agents.create_thread()

# Add a user message to the thread
message = project_client.agents.create_message(
 thread_id=thread.id,
 role="user",
 content="Can you analyze this sales dataset and create a visualization?"
)

# Execute the agent on the thread
run = project_client.agents.create_run(
 thread_id=thread.id,
 agent_id=agent.id
)

The diagram below explains the relationship between the core components of Azure AI Agents:

👁 Image

Azure AI Agents enhances these core components with enterprise-grade capabilities. The service provides comprehensive security through Microsoft Entra ID integration, role-based access control, and network isolation. Compliance features include data residency controls, audit logging, and customer-managed keys. The service automatically handles scaling and availability, allowing you to focus on building your application logic rather than managing infrastructure.

Mapping to Azure AI Agents to the Agent Anatomy

Azure AI Agents framework maps closely to the principles outlined in the above illustration, which breaks down an AI agent’s anatomy into key components: Persona, Instruction, Task, Planning, Memory, Tools, and Delegation.

👁 Image
Each of these elements is fundamental to the design of Azure AI agents, enabling the creation of intelligent, role-specific, and collaborative AI systems.

Persona

Azure AI Agents implement personas by combining flexible model selection with detailed system instructions and role-specific configurations. The service allows you to choose from various models, including GPT-4, Llama, and Mistral, and then shape the agent’s personality and expertise through detailed system messages.

Instruction

Azure AI Agents handle instructions through a sophisticated thread-based architecture that maintains context and guidance throughout conversations. The service separates core instructions (defining the agent’s general behavior) from task-specific instructions (guiding individual interactions).

Task

Tasks in Azure AI Agents are implemented through a combination of messages and runs that work together to accomplish specific goals. The service breaks down complex tasks into manageable steps, coordinating tool usage and maintaining progress through the run system.

Planning

The planning component in Azure AI Agents handles tool selection, execution steps, and resource coordination. The service automatically plans the actions needed to complete a task, adapting to changing requirements and handling complex workflows.

Memory

Memory management in Azure AI Agents combines thread persistence, vector stores, and automatic context management. The service maintains short-term memory (within thread contexts) and long-term memory (through vector stores and file attachments).

Tools

Azure AI Agents implement tool support through a flexible integration system. Built-in tools provide core functionality, while custom tools can be added through Azure Functions and OpenAPI specifications.

Delegation

Azure AI Agents support delegation by integrating multi-agent orchestration frameworks like AutoGen and Semantic Kernel. While direct agent-to-agent delegation isn’t built into the service, you can achieve multi-agent workflows by combining Azure AI Agents with these frameworks. This enables complex scenarios where multiple agents collaborate on tasks. AutoGen or Semantic Kernel is recommended for more complex multi-agent scenarios, providing dedicated agent collaboration and task delegation features.

Conclusion

The modular architecture of Azure AI Agents enables developers to build sophisticated AI applications by combining various components. Each component has a clear purpose and seamlessly enables intelligent, stateful conversations. Whether you’re building a simple chatbot or a complex multi-agent system, understanding these components and their relationships is crucial for creating effective AI solutions.

In the next part of this series on Azure AI Agents, we will build an end-to-end agentic workflow. Stay tuned!