Building Agentic AI with Azure AI Foundry: Agents, Tools & Orchestration

In the last few articles I walked from SharePoint search to RAG to low-code Copilot Studio agents. This is the developer's end of that same arc: building agentic AI with code on Azure AI Foundry. Where Copilot Studio hands you a low-code surface, Foundry gives you the full platform — models, tools, orchestration, tracing, and evaluation — to ship production agents that reason, call systems, and act. Let's separate the hype from the engineering.

What "agentic" actually means

A chatbot answers. An agent pursues a goal: it reasons about what to do, calls tools to gather information or take action, observes the results, and loops until the task is done. The model is no longer just generating text — it's the decision-making core of a system that touches the real world.

The three things that turn a model into an agent:

Tools — functions, APIs, knowledge sources, or code the model can invoke (via function/tool calling).
Memory / state — a conversation thread the agent reasons over across turns.
An orchestration loop — the runtime that executes the reason → act → observe cycle until completion.

Azure AI Foundry is Microsoft's unified platform for building, evaluating, and operating AI apps and agents — the evolution of what was Azure AI Studio. It bundles the model catalog, the Foundry Agent Service, knowledge/grounding, tracing, evaluations, and content safety behind one project resource and SDK.

The anatomy of a Foundry agent

The Agent Service follows a clean, durable object model. You create an agent (a model + instructions + tools), open a thread (the conversation), add messages, and start a run — the service executes the orchestration loop server-side, calling tools as the model decides, and returns the result:

Foundry Agent Service — run lifecycle

User message → Thread"Summarise open contracts over $50k and flag risks"

Model reasons (Run)Plans which tools to call

Tool: query contractsFunction calling

Tool: grounding searchAzure AI Search knowledge

Grounded answer → ThreadCited, with risk flags

Reason → act → observe: the run loop executes server-side, invoking tools the model selects until the goal is met.

Your first agent in code

The Foundry SDK centres on an AIProjectClient pointed at your project endpoint, authenticated with Entra ID via DefaultAzureCredential — no API keys to leak. From there, the agents namespace creates the agent, thread, messages, and run:

from azure.ai.projects import AIProjectClient
from azure.identity import DefaultAzureCredential

project = AIProjectClient(
    endpoint="https://<your-resource>.services.ai.azure.com/api/projects/<project>",
    credential=DefaultAzureCredential(),
)

# 1) Define the agent: a model + instructions
agent = project.agents.create_agent(
    model="gpt-4o",
    name="contracts-analyst",
    instructions=(
        "You are a contracts analyst. Use the provided tools to look up "
        "contract data and ground your answers. Always cite the contract ID. "
        "If you are unsure, say so rather than guessing."
    ),
)

# 2) Open a conversation thread and post a message
thread = project.agents.threads.create()
project.agents.messages.create(
    thread_id=thread.id,
    role="user",
    content="Summarise open contracts over $50k and flag any renewal risks.",
)

# 3) Run it — the service drives the reasoning + tool loop
run = project.agents.runs.create_and_process(
    thread_id=thread.id,
    agent_id=agent.id,
)

if run.status == "failed":
    print("Run failed:", run.last_error)
else:
    messages = project.agents.messages.list(thread_id=thread.id)
    for m in messages:
        print(m.role, "→", m.text_messages[-1].text.value if m.text_messages else "")

create_and_process is the easy path — it submits the run and polls to completion, automatically executing any tool calls the agent decides on. When you need to stream tokens or handle tool calls yourself, use runs.create + runs.stream instead and react to events as they arrive.

Tools: where agents get their power

An agent with no tools is just a chat completion. Foundry agents support several tool categories, and you usually mix them:

Tool type	What it does
Function calling	Invoke your code/APIs — the model emits a structured call, you execute and return the result.
Knowledge (grounding)	Retrieve from Azure AI Search, files, SharePoint, or Microsoft Fabric — RAG, built in.
Code Interpreter	Run sandboxed Python for data analysis, math, and file generation.
OpenAPI / connected tools	Call any REST API described by an OpenAPI spec, or other Azure services.

Function calling is the workhorse. You describe a Python function, attach it as a tool, and the agent decides when to call it — the SDK marshals arguments and return values:

from azure.ai.agents.models import FunctionTool
import json

def get_contract(contract_id: str) -> str:
    """Return contract details as JSON for a given contract ID."""
    # ...query Dataverse / SQL / SharePoint here...
    record = {"id": contract_id, "value": 48000, "status": "Open", "renewsOn": "2026-09-01"}
    return json.dumps(record)

# Register the function as a tool the agent may call
functions = FunctionTool(functions={get_contract})

agent = project.agents.create_agent(
    model="gpt-4o",
    name="contracts-analyst",
    instructions="Use get_contract to fetch live data before answering.",
    tools=functions.definitions,
)

Treat tools as the security boundary. The model decides when to call a tool, but your code decides what it's allowed to do. Validate and authorize every argument inside the function, scope database queries to the caller's permissions, and never let a tool execute free-form input. The model can be prompt-injected; your tool implementation must assume it.

Grounding: RAG without the plumbing

The single most common enterprise agent is "answer questions over our content, with citations." Foundry's knowledge tools wire an agent to an Azure AI Search index (or files you upload) so the agent retrieves relevant chunks and grounds its response — the RAG pattern from the search-to-RAG article, now a configuration step rather than a pipeline you hand-build:

from azure.ai.agents.models import AzureAISearchTool

search = AzureAISearchTool(
    index_connection_id=conn_id,   # a project connection to Azure AI Search
    index_name="contracts-index",
)

agent = project.agents.create_agent(
    model="gpt-4o",
    name="contracts-kb",
    instructions="Answer only from retrieved documents and cite the source.",
    tools=search.definitions,
    tool_resources=search.resources,
)

From one agent to many

Single agents handle a lot, but complex work benefits from multi-agent orchestration — specialised agents, each with a narrow role and toolset, coordinated by an orchestrator. Foundry supports this through connected agents (one agent calling another as a tool) and, for code-first patterns, the Microsoft Agent Framework and the open Semantic Kernel SDK for explicit planners and group-chat style collaboration.

Multi-agent orchestration

Orchestratorplans & delegates

callsas tools

Specialist agentsretriever · analyst · writer

Divide and conquer: an orchestrator delegates to specialist agents, each with a focused role and tools.

You can't ship what you can't see: tracing & evaluation

This is the part that separates a demo from production, and where Foundry genuinely shines. Agents are non-deterministic — the same prompt can take different tool paths — so observability and measurement are not optional.

Tracing — Foundry emits OpenTelemetry traces of every run: the model calls, the tool invocations, inputs/outputs, tokens, and latency. Wire it to Azure Monitor / Application Insights and you can see exactly why an agent did what it did.
Evaluation — run your agent against a test dataset and score it with built-in evaluators for groundedness, relevance, coherence, fluency, plus agent-specific ones like intent resolution and tool-call accuracy. Track scores across versions so you know a prompt change improved things instead of hoping it did.

from azure.ai.evaluation import evaluate, GroundednessEvaluator
from azure.ai.evaluation import IntentResolutionEvaluator

results = evaluate(
    data="eval_dataset.jsonl",               # query / response / context rows
    evaluators={
        "groundedness": GroundednessEvaluator(model_config),
        "intent_resolution": IntentResolutionEvaluator(model_config),
    },
)
print(results["metrics"])   # aggregate scores to gate a release

Build the eval set from real traces. Once tracing is on, curate interesting and failed runs into your evaluation dataset, so every regression you find becomes a permanent test. This trace → dataset → eval loop is the agentic equivalent of writing a unit test for every bug.

Responsible AI is part of the build

Production agents touch real data and take real actions, so guardrails ship with the agent, not after it:

Azure AI Content Safety — filter harmful content and detect prompt injection / jailbreak attempts on inputs and tool outputs.
Entra ID + least privilege — authenticate with managed identity, and scope each tool's data access to the acting user.
Human-in-the-loop — for consequential actions (sending money, deleting records), require an approval step rather than letting the agent act unattended.

The takeaway

Agentic AI isn't a bigger chatbot — it's a system where a reasoning model orchestrates tools to accomplish goals. Azure AI Foundry gives you the whole stack to build it responsibly: the Agent Service with its thread/run model, function and knowledge tools, multi-agent orchestration, and — crucially — the tracing and evaluation that make agents trustworthy enough to ship. The grounding discipline from SharePoint search, the action-wiring from Power Automate, and the data modeling from Dataverse all converge here. Same engineering judgment, pointed at the most capable surface yet.

Dhaval Shah

Cloud Solution Architect — 18+ years across SharePoint, Microsoft 365, and Azure AI.

Back to Blog