Function / Tool Calling#

Tool calling lets a model ask an application to run a function.

The model does not execute the function. It returns the tool name and arguments; the application validates and runs them.

Videos#

How it works#

sequenceDiagram
    participant U as User
    participant M as Model
    participant A as Application
    participant T as Tool
    U->>M: What is the weather in Pune?
    M-->>A: call get_weather(city=Pune)
    A->>T: Validate and run
    T-->>A: 29°C
    A->>M: Tool result
    M-->>U: It is 29°C in Pune.

Tool definition#

A tool needs three things:

• Name: what the function is called.
• Description: when the model should use it.
• Schema: which arguments are allowed.

weather_tool = {
    "name": "get_weather",
    "description": "Get current weather for one city",
    "parameters": {
        "city": "string",
        "units": ["metric", "imperial"],
    },
}

Real model APIs use JSON Schema, but the simple example shows the idea.

Built-in tools and custom tools#

Providers can offer built-in tools: the provider operates the tool and the application enables it. Examples include search, file retrieval, and a managed code environment. You can also define a custom function whose code and permissions you operate.

This is a guide to the categories, not a promise that every tool is available for every model, region, account, or API. Check the provider documentation and the tool’s data, billing, and retention rules before using it.

Built-in does not mean harmless. A web result, fetched page, or uploaded file is still untrusted input. A managed code environment still needs a review of what data it receives and what files it returns. For a payment, production change, or customer-data operation, keep the final permission and confirmation check in your own application.

A provider-neutral example#

The orchestration pattern is the same whether the tool is built in or custom:

User asks for this week's policy changes
→ model chooses web search
→ provider returns sources and excerpts
→ application records sources and checks its citation policy
→ model writes a cited summary

For a custom refund_order function, replace the provider step with your application validating the arguments, checking the user and order, obtaining approval, and calling the payment service. The model never becomes the permission system.

Good tool design#

Parallel calls#

Independent reads can run together:

weather(Pune) ─┐
weather(Delhi) ├─ run together → combine results
weather(Kochi) ┘

Dependent actions must stay in order:

find_order → show_refund_preview → user_approval → refund_order

Tool calling is a contract#

A tool call has two separate parts:

1. The model proposes a tool name and arguments.
2. The application decides whether the call is valid, allowed, and safe to execute.

The model can choose the wrong city, invent an order ID, or be influenced by untrusted text. Therefore validation belongs in normal application code. Parse the arguments against a strict schema, reject extra fields when appropriate, enforce ranges and formats, then check the current user’s permissions.

Model proposal: refund_order(order_id="A-19", amount=50000)
Application: validate type → look up order → check ownership → check amount
             → show preview → require approval → call payment service

The model should never receive a general “run arbitrary code” or “make any HTTP request” tool when a narrow operation will do.

Design useful results#

Tool results are context for the next model step. Return compact structured data and a clear status, not a large HTML page or a human-only sentence.

{
  "status": "ok",
  "order_id": "A-19",
  "eligible_for_refund": true,
  "maximum_amount": 799,
  "currency": "INR"
}

For a failure, use a safe category such as NOT_FOUND, FORBIDDEN, INVALID_ARGUMENT, CONFLICT, or TEMPORARY_FAILURE. Include a short correction when it is safe to do so. Avoid returning raw database errors, internal paths, tokens, or stack traces.

Read tools and write tools#

Do not rely on a prompt saying “ask first.” The tool implementation should make an approval token or confirmed action ID mandatory for high-impact operations.

Reliability patterns#

• Timeout: stop waiting for a slow dependency.
• Retry: retry only safe, temporary failures with a small limit and backoff.
• Idempotency key: lets the service recognize a repeated write request.
• Pagination and result limits: prevent one tool result from consuming the whole context window.
• Audit record: store who requested the tool, sanitized arguments, outcome, external ID, and time.

When a tool result is untrusted—for example, web search, email, or a document— tell the model to treat it as data. It may contain text attempting to change the agent’s rules or persuade it to call another tool.

Testing a tool#

Test valid input, missing input, malformed input, a user without access, a timeout, duplicate requests, and approval rejection. Also test the model-facing description: can a human reading it predict exactly when it should be used and what it cannot do? Clear contracts make tool calling safer and easier to debug.

Safety checklist#

• Allow only approved tools for the current user.
• Validate every argument before execution.
• Check permissions in application code, not in the prompt.
• Ask for confirmation before payments, deletion, messages, or publishing.
• Add timeouts and safe error messages.
• Use an idempotency key so a retry cannot repeat a payment or email.
• Treat web pages, files, and tool results as untrusted input.

References#

• JSON Schema object reference
• OpenAI tools guide
• Gemini API tools
• Claude tool use
• MCP tools specification
• OWASP Prompt Injection guidance