AI Agent Tool Use and Function Calling: Production Design Patterns

Introduction

AI agents are only as useful as the tools they can wield. An agent that can chat about ideas but cannot interact with APIs, databases, or external systems is limited to conversation — not execution. Tool use, also called function calling, is what transforms an AI model from a text generator into an action-capable agent.

This guide covers production-grade patterns for designing, selecting, and integrating tools into AI agents. Whether you’re building a customer service agent that checks order status or an internal operations agent that runs database queries, the principles below will help you avoid the most common failure modes.

What Is Tool Use and Function Calling?

Function calling is a structured interface between an LLM and external capabilities. Instead of generating free-form text, the model outputs a JSON payload describing which tool to call and with what arguments. The runtime then executes the tool and feeds the result back to the model.

{
  "tool": "get_weather",
  "arguments": {
    "location": "San Francisco",
    "unit": "celsius"
  }
}

The key difference from traditional API calls is that the model decides **which** tool to call and **what** arguments to pass — based on natural language context. This is powerful, but it introduces new failure modes that do not exist in deterministic code.

Tool Design Principles

Principle 1: Keep Tools Atomic and Single-Purpose

Each tool should do one thing well. Avoid monolithic tools like `execute_task()` that accept a free-text instruction. Instead, break functionality into discrete, predictable functions:

• `search_knowledge_base(query)`
• `create_ticket(title, description, priority)`
• `check_order_status(order_id)`
• `send_notification(recipient, message)`

Atomic tools are easier to test, debug, and reason about. They also produce more reliable function calling because the model has clearer boundaries for when to invoke each one.

Principle 2: Define Clear, Constrained Schemas

Tool arguments should use strict schemas with typed fields, required parameters, and enumerated values where possible. Loose schemas lead to ambiguous calls and runtime errors.

Good schema example:

{
  "type": "object",
  "properties": {
    "order_id": {
      "type": "string",
      "pattern": "^ORD-[0-9]{6}$",
      "description": "Order ID in format ORD-123456"
    },
    "include_tracking": {
      "type": "boolean",
      "default": false
    }
  },
  "required": ["order_id"]
}

Bad schema example:

{
  "type": "object",
  "properties": {
    "query": {
      "type": "string",
      "description": "Any search terms"
    }
  }
}

The constrained schema gives the model explicit validation rules and reduces the chance of generating invalid arguments.

Principle 3: Provide Rich Tool Descriptions

The model relies entirely on your tool description to understand when and how to use each tool. A vague description like `”Get weather information”` leads to inconsistent usage. A detailed description like `”Retrieve current weather and 5-day forecast for a city. Accepts city name or ZIP code. Returns temperature in specified unit (celsius/fahrenheit).”` produces far more reliable calls.

Include in every tool description:

• What the tool does
• When to use it (trigger conditions)
• What parameters it accepts and their formats
• What the response looks like
• Any limitations or error conditions

Common Failure Modes and Fixes

Failure Mode 1: Hallucinated Arguments

The model invents arguments that do not match the schema or fabricates values. This happens when schemas are loose or descriptions are unclear.

**Fix:** Add strict validation at the tool execution layer. Never trust unvalidated model output. Use Pydantic models, JSON Schema validators, or equivalent to catch invalid arguments before they reach your backend.

Failure Mode 2: Missing Required Tools

The model calls a tool that does not exist or omits a required tool because its description was unclear.

**Fix:** Maintain a tool registry and validate tool names before execution. Log all tool calls for auditability. Review logs periodically to identify patterns of missing or misused tools.

Failure Mode 3: Over-Tooling

Too many tools create decision fatigue for the model. When presented with 20+ tools, the model struggles to select the right one and may call irrelevant tools or skip necessary ones.

**Fix:** Group related tools under a single composite tool where appropriate. Use tool categories or namespaces. Prioritize the most commonly needed tools and hide specialized ones behind conditional loading.

Failure Mode 4: Non-Deterministic Output Formats

Different invocations of the same tool produce inconsistently formatted responses, making it hard for the model to parse results reliably.

**Fix:** Standardize tool response formats. Always return structured JSON with consistent field names. Include success/failure indicators and error messages in a predictable structure.

Tool Integration Patterns

Pattern 1: Direct API Calls

The simplest pattern: the agent calls external APIs directly through tool wrappers. This works well for read-heavy operations like fetching data, checking status, or retrieving documents.

Advantages:

• Low latency
• Full control over authentication and error handling
• Easy to add caching layers

Disadvantages:

• Requires maintaining API client code
• Each integration adds complexity

Pattern 2: Internal Service Abstraction

Wrap internal services (databases, CRMs, ERP systems) behind a consistent tool interface. This decouples the agent from backend implementation details.

Example: Instead of letting the agent query the database directly, provide a `query_customer_data(customer_id)` tool that encapsulates the database logic, caching, and permission checks.

Advantages:

• Centralized security and access control
• Easier to modify backend without changing agent logic
• Consistent error handling

Pattern 3: Chain-of-Tools Workflows

Complex tasks often require multiple tools called in sequence. The model should be able to chain tool calls naturally, using the output of one tool as input to another.

Example workflow:

1. `search_orders(customer_email)` → returns order list
2. `get_order_details(order_id)` → returns specific order info
3. `check_inventory(product_id)` → verifies stock
4. `create_shipment(order_id, product_id)` → fulfills order

Enable this by ensuring each tool’s response is structured and self-describing, so the model can parse results and decide on the next action.

Security and Permission Management

Tools give AI agents the ability to perform actions. This power requires careful security controls.

Key Security Practices

1. **Principle of Least Privilege:** Each tool should have the minimum permissions necessary. A customer service agent should not need admin-level database access.

1. **Input Sanitization:** Treat all tool arguments as untrusted input. Validate, sanitize, and parameterize queries to prevent injection attacks.

1. **Audit Logging:** Log every tool call with timestamp, agent identity, tool name, arguments (sanitized), and result. This is essential for debugging and compliance.

1. **Rate Limiting:** Apply rate limits per tool and per agent to prevent abuse and protect downstream systems.

1. **Sensitive Data Filtering:** Ensure tool responses do not leak PII or secrets. Filter sensitive fields before returning results to the model.

Testing Tool Integration

Test your tool integration rigorously before deploying to production.

Unit Tests for Tools

Write tests for each tool that verify:

• Schema validation catches invalid arguments
• Expected outputs match the documented format
• Error conditions are handled gracefully
• Edge cases (empty results, timeouts, rate limits) produce correct responses

Integration Tests

Simulate real agent workflows end-to-end:

• Feed natural language prompts and verify correct tool selection
• Check that chained tool calls produce coherent results
• Measure latency and error rates under load
• Validate security controls (unauthorized tool access is blocked)

Fuzz Testing

Inject malformed or unexpected tool arguments to verify your validation layers catch them. This helps identify schema gaps and edge cases that manual testing might miss.

Monitoring and Observability

Production tool use requires observability. Track these metrics:

• **Tool call success rate:** Percentage of calls that complete without error
• **Argument validation failure rate:** How often the model generates invalid arguments
• **Latency distribution:** Time from tool call to response
• **Tool selection accuracy:** Are the right tools being called for the right tasks?
• **Error patterns:** Common failure modes that need tool or prompt fixes

Use structured logging and integrate with your existing monitoring stack (Prometheus, Datadog, etc.) for alerting on anomalies.

Best Practices Summary

FAQ

**Q: How many tools should an AI agent have?**

A: Start with 5–10 core tools that cover the most common tasks. Add specialized tools only when needed. More than 15–20 tools tends to degrade selection accuracy unless you implement smart routing or tool grouping.

**Q: Should I let the agent call tools directly or route through a supervisor?**

A: For simple agents, direct tool calls work fine. For complex multi-agent systems, a supervisor or router can make better tool selection decisions by considering context that individual agents might miss.

**Q: How do I handle tool failures in agent workflows?**

A: Implement retry logic with exponential backoff for transient errors. For permanent failures, return a clear error message to the model so it can adapt its approach. Always log failures for debugging.

**Q: Can tools call other tools?**

A: Yes, this is called tool chaining. It is powerful for complex workflows but adds latency and complexity. Use it judiciously and ensure each tool in the chain has clear success/failure signals.

**Q: How do I update tool definitions without restarting the agent?**

A: Design your tool registry to support dynamic reloading. Store tool definitions in a database or configuration file that can be updated at runtime. Most production frameworks support hot-reloading of tool schemas.

**Q: What is the difference between function calling and agents?**

A: Function calling is a mechanism — a way for the model to invoke external code. An agent is a system that uses function calling (along with memory, planning, and other capabilities) to autonomously accomplish goals. Function calling is a tool agents use, not the agent itself.

**Q: How do I prevent agents from calling tools too frequently?**

A: Set rate limits at the tool level. Implement throttling middleware. Use circuit breakers to temporarily disable tools that are failing or being overused. Monitor call frequency and adjust limits based on actual usage patterns.

Conclusion

Tool use is the bridge between AI conversation and AI action. Designing reliable tool integrations requires attention to schema quality, input validation, security, and observability. Start with atomic tools, strict schemas, and rich descriptions. Test thoroughly. Monitor continuously. And remember: the better your tools are designed, the more capable and reliable your agent will be.

For practical guidance on building production AI agents with robust tool integration, explore the resources available at [SmaugBrain](https://www.smaugbrain.com/).

For practical guidance on building production AI agents with robust tool integration, explore the resources available at SmaugBrain.