Frontend Tools
Create frontend tools and use them within your Microsoft Agent Framework agent.
What is this?
Frontend tools enable you to define client-side functions that your agent can invoke, with execution happening entirely in the user's browser. When your agent calls a frontend tool, the logic runs on the client side, giving you direct access to the frontend environment.
This can be utilized to let your agent control the UI, power generative UI, or for Human-in-the-loop interactions.
In this guide, we cover the use of frontend tools for generative UI.
When should I use this?
Use frontend tools when you need your agent to interact with client-side primitives such as:
- Reading or modifying React component state
- Accessing browser APIs like localStorage, sessionStorage, or cookies
- Triggering UI updates or animations
- Interacting with third-party frontend libraries
- Performing actions that require the user's immediate browser context
Implementation
You'll need to run your agent and connect it to CopilotKit before proceeding.
If you don't already have CopilotKit and your agent connected, choose one of the following options:
Create a frontend tool
First, you'll need to create a frontend tool using the useFrontendTool hook. Here's a simple one to get you started that says hello to the user.
import { useFrontendTool } from "@copilotkit/react-core"
export function Page() {
// ...
useFrontendTool({
name: "sayHello",
description: "Say hello to the user",
parameters: [
{
name: "name",
type: "string",
description: "The name of the user to say hello to",
required: true,
},
],
handler({ name }) {
// Handler returns the result of the tool call
return { currentURLPath: window.location.href, userName: name };
},
render: ({ args }) => {
// Renders UI based on the data of the tool call
return (
<div>
<h1>Hello, {args.name}!</h1>
<h1>You're currently on {window.location.href}</h1>
</div>
);
},
});
// ...
}Modify your server
Ensure your AG-UI server receives the frontend tools and passes them to your agent logic as needed.
Receiving frontend tools on the server
When CopilotKit calls your AG-UI server, frontend tools registered with useFrontendTool are forwarded to your AG-UI server
and passed to the underlying agent via the AgentRunOptions on the server. They are automatically available by default to the
underlying chat client.
using Microsoft.Agents.AI;
using Microsoft.Agents.AI.Hosting.AGUI.AspNetCore;
using OpenAI;
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddAGUI();
var app = builder.Build();
var githubToken = builder.Configuration["GitHubToken"]!;
var openAI = new OpenAIClient(
new System.ClientModel.ApiKeyCredential(githubToken),
new OpenAIClientOptions { Endpoint = new Uri("https://models.inference.ai.azure.com") });
// Create the agent - frontend tools are automatically available
var agent = openAI.GetChatClient("gpt-4o-mini")
.CreateAIAgent(name: "SampleAgent", instructions: "You are a helpful assistant.");
// Map the AG-UI endpoint
app.MapAGUI("/", agent);
app.Run("http://localhost:8000");from __future__ import annotations
import os
from fastapi import FastAPI
from dotenv import load_dotenv
from agent_framework import ChatAgent
from agent_framework import ChatClientProtocol
from agent_framework.azure import AzureOpenAIChatClient
from agent_framework.openai import OpenAIChatClient
from agent_framework.ag_ui import add_agent_framework_fastapi_endpoint
from azure.identity import DefaultAzureCredential
load_dotenv()
def _build_chat_client() -> ChatClientProtocol:
if bool(os.getenv("AZURE_OPENAI_ENDPOINT")):
return AzureOpenAIChatClient(
credential=DefaultAzureCredential(),
deployment_name=os.getenv("AZURE_OPENAI_CHAT_DEPLOYMENT_NAME", "gpt-4o-mini"),
endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
)
if bool(os.getenv("OPENAI_API_KEY")):
return OpenAIChatClient(
model_id=os.getenv("OPENAI_CHAT_MODEL_ID", "gpt-4o-mini"),
api_key=os.getenv("OPENAI_API_KEY"),
)
raise RuntimeError("Set AZURE_OPENAI_* or OPENAI_API_KEY in agent/.env")
chat_client = _build_chat_client()
# Frontend tools registered with useFrontendTool are automatically available
# to the agent through the AG-UI protocol.
agent = ChatAgent(
name="sample_agent",
instructions="You are a helpful assistant.",
chat_client=chat_client,
)
app = FastAPI(title="AG-UI Server (Python)")
add_agent_framework_fastapi_endpoint(app=app, agent=agent, path="/")Frontend tools registered with useFrontendTool are automatically forwarded to your agent through the AG-UI protocol. The agent can invoke them, and execution happens on the frontend.
Give it a try!
You've now given your agent the ability to directly call any frontend tools you've defined. These tools will be available to the agent where they can be used as needed.
