AI Tools & Function Calling
How to use AI APIs and enable function calls in LLMs.
Last updated: July 2026
Prompt
The user asks for something that needs live data or an action - like the weather or booking a table.
Prompt
Listen: AI Tools & Function Calling (student & teacher)StudentTeacher
0:000:00
Function calling (also called tool calling or tool use) lets a large language model notice when a request needs outside data or an action it can't do on its own. Instead of guessing, it produces a structured, machine-readable output (usually JSON) that triggers a real external function, API, or service. The application then runs that function and feeds the result back to the model.
“Without function calling, a model is a closed box that only generates text. With it, the model becomes an active helper that can search the web, query a database, book a calendar slot, or run code on demand.”
| Component | What it does | Example |
|---|---|---|
Tool Definition (Schema) | Registers the functions the model can use, with a name, a description, and a JSON parameter schema added to the model's context | A travel agent registers search_flights(origin, destination, date, seat_class) with required IATA codes and an optional cabin class enum ['economy','business','first'] |
Tool Decision (Model Reasoning) | The model receives the message plus all tool definitions and decides whether to answer directly or output a structured tool call | User: 'Remind me to take medication at 7pm.' → Model outputs create_reminder(time='19:00', message='Take medication') |
Tool Execution (Developer Code) | The application layer, not the model, runs the actual function and maps the structured output to real code | Backend receives lookup_order(order_id='ORD-88421') from the model, queries the internal database, returns JSON result |
Observation (Tool Result) | The function's output is added back as a tool result message and sent to the model so its reply is grounded in real data | After order lookup returns {status: 'shipped', eta: '2025-05-07', carrier: 'FedEx'}, model generates: 'Your order has shipped via FedEx and arrives May 7th.' |
The Agent Loop | In multi-step tasks the call cycle repeats. Each new observation is added to the context so the model can reason about the results so far | Research assistant: web_search → fetch_url → create_summary → grounded cited response (4 chained tool calls before final answer) |
Full assembled example
User → Model sees: message + tool schemas → Model outputs: {"type":"function_call","name":"get_stock_price","arguments":{"ticker":"AAPL"}} → App executes real API call → Result $213.47 returned to model → Model responds: 'Apple (AAPL) is currently trading at $213.47.'Core Principles
- 1The model decides; the application executes - The model only picks which tool to call and with which arguments. It never runs code itself. That boundary is a key security and control feature.
- 2Tool descriptions are the only guide the model has - 'search_database' with no context leads to missed or wrong calls. Rewrite it as 'Search internal customer records. Use ONLY for queries about specific accounts, orders, or support tickets.' and the model's choices improve a lot.
- 3The tool schema is added on every call, so keep it lean - A logistics platform cut from 25 tools to 8 core tools, dropped average token usage by 40%, and trimmed response latency by 600ms.
- 4Fail informatively, not silently - A database search that returns '' invites the model to make things up. Return 'No matching records found for order ID ORD-99999. Verify the ID and retry, or use search_orders() with a broader query.' instead.
- 5Always validate and clean tool inputs - A model might output delete_file(path='../../../etc/passwd') if inputs aren't checked. Enforce path restrictions and input types before calling any file-system or database function.
- 6Parallel tool calls improve throughput; sequential calls handle dependencies - For 'weather in Paris and London', call both at once. For 'search flights then book the cheapest', get the search results before calling the booking tool.
Single Tool Call
The model calls exactly one tool in one turn to handle the request. The simplest and most common pattern.
Prompt
User: 'What is the current price of Bitcoin?' → Model calls get_crypto_price(symbol='BTC') once and returns the live result.
Output
Simple atomic lookup completed in a single round-trip
Use when:Simple one-shot lookups: weather checks, price lookups, unit conversions, or sending a single notification.
Multi-Step / Chained Tool Calls
The model makes a series of tool calls where each step builds on the last, using earlier results to decide what to do next.
Prompt
'Find the CEO of Stripe and check their recent LinkedIn activity.'
1. web_search('Stripe CEO') → learns it's Patrick Collison
2. search_linkedin(name='Patrick Collison') → gets profile data
3. Synthesizes both into a final responseOutput
Research tasks and multi-step workflows where each call informs the next
Use when:Research tasks, multi-step workflows, and tasks that need conditional logic based on earlier results.
Parallel Tool Calls
The model issues several independent tool calls at once in a single turn, which cuts latency a lot compared with calling them one by one.
Prompt
'Compare the weather in New York, London, and Tokyo right now.' → All three get_weather() calls issued at once in a single model response turn.
Output
All results returned together, then synthesized into a comparison: 3x faster than sequential
Use when:Any request that needs data from several independent sources at the same time. It cuts latency a lot.
Forced Tool Use
The developer sets the model to always call a specific tool with tool_choice: 'required', whether or not the model would pick it on its own.
Prompt
A compliance chatbot must always call log_query(query, user_id) before responding to maintain a mandatory audit trail.
Output
Guaranteed tool execution, useful for auditing, logging, and guardrail enforcement
Use when:Auditing, logging, guardrail enforcement, and structured-output extraction where the tool call must be guaranteed.
Agentic / Autonomous Tool Use
The model works in a self-directed loop: it makes tool calls, checks the results, plans the next step, and repeats without a person stepping in.
Prompt
'Research and write a competitor analysis on the top 3 AI coding assistants.' → Agent autonomously runs web searches, fetches pages, extracts data, structures comparison, writes report.
Output
Complex multi-step task completed with minimal human input
Use when:Complex, multi-hour tasks, software agents, and data pipelines. It needs solid error handling and human-in-the-loop checkpoints for high-stakes actions.
MCP (Model Context Protocol)
An open, standard protocol for how AI models connect to outside tools and data sources. Build it once and connect to any MCP-compliant agent framework. Think of it as USB-C for AI tools.
Prompt
A developer builds one MCP server for their internal knowledge base. It connects to agents built on Claude, OpenAI, or any MCP-compatible framework, without rewriting the integration each time.
Output
Reusable tool implementation across multiple AI platforms and frameworks
Use when:Reusable enterprise tool libraries, and sharing one tool implementation across several AI platforms and agent frameworks.
Best Practices
- 1Write tool descriptions that say when to use the tool, not just what it does - Bad: 'Search the web'. Good: 'Search the web for current events, recent news, or live prices. Do NOT use for questions answerable from general knowledge.'
- 2Use enums and strict types to limit parameter values - "status": {"type": "string", "enum": ["open","in_progress","resolved","closed"]} stops the model from inventing invalid values at the source
- 3Mark only the truly required parameters as required; use defaults for the optional ones - A send_email tool marks to and subject as required but leaves cc, bcc, and priority optional with documented defaults, which keeps calls lean
- 4Return structured, helpful error messages from tools - If an order isn't found, return 'No order found with ID X. Verify the ID or use search_orders() to find it.' rather than an empty string or a raw exception
- 5Limit the tool set to what the task actually needs - A billing support agent only reaches payment- and invoice-related tools. It can't call delete_account() or push_code_to_repo()
- 6Put usage guidance in the system prompt, not only in tool descriptions - 'Always check knowledge_base first for internal company questions before using web_search. Use calculator for any arithmetic - never compute numbers yourself.'
- 7Log and inspect intermediate steps while developing - Turn on verbose, step-by-step logging so you can see exactly which tools were called, with what arguments, and what came back. This is essential for debugging agents
Common Mistakes
| Mistake | Real-world scenario | Fix |
|---|---|---|
| Vague tool descriptions | A model with 'description: Get data' calls the wrong database tool for a customer lookup. | Rewrite: 'Query customer account records by email or customer ID. Use for account-specific questions only.' |
| Too many tools registered at once | An agent with 30 tools wastes ~3,000 tokens per request on definitions; latency and cost spike. | Load tools dynamically: serve only the 5-8 tools relevant to the current task. |
| No input validation before execution | Model generates delete_file(path='../../config.json'); app executes it without sanitization. | Whitelist allowed paths and actions; validate every model-generated argument before calling any function. |
| Silent failures (empty or null returns) | Tool returns None when a record isn't found; model hallucinates a fake result to compensate. | Always return a string with context: 'No record found for X. Suggest trying Y.' |
| Treating tool use as guaranteed | Developer assumes the model will always call the weather tool; model answers from training instead. | Use tool_choice: 'required' when a tool call is mandatory; test with varied prompts to confirm behavior. |
| Ignoring token costs of tool schemas | A chatbot with 20 tools at 150 tokens per schema adds 3,000 tokens to every single API call. | Trim schemas; combine tools that share parameters; load large libraries dynamically. |
| No error handling in agent loop | A tool times out; agent loop crashes instead of retrying or responding gracefully. | Wrap every tool call in try/except; return clear error strings; retry with backoff. |
| Storing sensitive data in tool results | Full credit card numbers returned in tool output and stored in conversation context. | Mask PII in tool responses; return only last 4 digits, tokenized IDs, or redacted summaries. |
| Over-automating high-stakes actions | An agent autonomously deletes 200 customer records based on ambiguous instructions. | Add human-in-the-loop confirmation checkpoints for irreversible, high-impact actions (delete, send, pay). |
| Using function calling for simple lookups | A model calls a complex API for 'What is 2+2?' when it could answer directly. | Save tool calls for things the model genuinely can't do without external data. |
Use Cases by Industry
| Use case | Real-world example | Best approach |
|---|---|---|
| Live data retrieval | A financial dashboard assistant answering 'What is Tesla's P/E ratio right now?' | Single tool call to a financial data API (get_stock_fundamentals) |
| Customer support | A retail bot that checks order status, initiates returns, and escalates tickets | Multi-step chained calls: lookup_order → create_return → notify_agent |
| Calendar & scheduling | An AI assistant that finds a free slot for 3 attendees and books the meeting | Parallel tool calls to fetch availability + sequential call to create event |
| Code execution | A coding assistant that runs user-submitted Python snippets in a sandbox | Single forced tool call to a sandboxed code interpreter tool |
| Research & summarization | An analyst agent that gathers quarterly earnings from 5 companies and writes a comparison | Parallel web searches + chained data extraction + text generation |
| Healthcare | A clinical decision support tool that pulls patient lab history before suggesting a diagnosis prompt | Chained secure EHR API calls with strict input validation and PII masking |
| E-commerce | A shopping assistant that checks inventory, applies promo codes, and places orders | Agentic loop: search → filter → add-to-cart → validate promo → checkout confirmation |
| DevOps / Engineering | A Slack bot that creates GitHub issues from bug reports and assigns them | Multi-step: parse_bug_report → create_github_issue → assign_team → post_slack_update |
| Legal / Compliance | A document review agent that checks contracts against a rule database and flags clauses | Chained calls to internal clause extractor + compliance rules engine |
| HR & Recruiting | An onboarding bot that creates accounts, sends welcome emails, and schedules orientation | Agentic loop with human confirmation checkpoint before account provisioning |
| IoT / Smart Home | A voice assistant that turns off lights, adjusts thermostat, and locks the door on 'goodnight' | Parallel independent tool calls to separate device-control APIs |
| Education | An AI tutor that retrieves practice problems matching the student's current performance level | Chained: get_student_profile → fetch_problems(difficulty=level, topic=subject) |
1
Single Tool Call: Live Weather Lookup
Weak prompt
User: 'What's the weather like in Tokyo right now?' → Without function calling: 'As of my training, Tokyo typically has mild weather, but I cannot retrieve live data.'
Better prompt
Strong prompt
Tool registered: get_current_weather(city, unit)
User asks: 'What's the weather in Tokyo right now?'
Step 1: Model outputs → get_current_weather(city='Tokyo', unit='celsius')
Step 2: App executes real weather API → {city: 'Tokyo', temperature: 22, condition: 'Sunny'}
Step 3: Model receives result and generates grounded responseOutput
The current weather in Tokyo is 22°C (Sunny). It looks like a pleasant day!
2
Multi-Step Chained Tool Calls: Order Status Bot
Weak prompt
User: 'My order ORD-4821 hasn't arrived, it's been 2 weeks.' → Without tools: 'I'm sorry to hear that. Please contact our support team.' (no real data or action taken)
Better prompt
Strong prompt
Turn 1: User message → Model calls lookup_order(order_id='ORD-4821')
Result: {status: 'in_transit', last_update: '2025-04-18', eta_was: '2025-04-22'}
Turn 2: Model sees order is 13 days overdue → calls create_support_ticket(order_id='ORD-4821', issue='delayed')
Result: {ticket_id: 'TKT-00293', message: 'Specialist will contact you within 24 hours.'}
Turn 3: Model synthesizes both observations into final responseOutput
I looked into your order ORD-4821. It shipped via USPS on April 18th and was expected by April 22nd - so it is overdue. I've created a support ticket (TKT-00293) on your behalf. A specialist will follow up within 24 hours.
3
Parallel Tool Calls: Multi-City Weather Comparison
Weak prompt
Sequential approach: get_weather(New York), wait → get_weather(London), wait → get_weather(Sydney). Each call adds a full round-trip, so you pay 3x the latency.
Better prompt
Strong prompt
Model receives message → outputs all 3 tool calls in a single turn (parallel):
- get_current_weather('New York', 'fahrenheit') [call_1]
- get_current_weather('London', 'celsius') [call_2]
- get_current_weather('Sydney', 'celsius') [call_3]
All 3 execute simultaneously → results returned together → model synthesizes comparisonOutput
Here's the current weather comparison: - New York: 68°F - Partly Cloudy - London: 14°C - Overcast - Sydney: 21°C - Clear Sydney and New York are enjoying nice conditions, while London is living up to its reputation.
4
MCP Tool Definition: Reusable Internal Knowledge Base
Prompt
{
"name": "search_knowledge_base",
"description": "Search internal company docs, FAQs, and policies. ALWAYS use this before web_search for internal questions.",
"inputSchema": {
"type": "object",
"properties": {
"query": { "type": "string", "description": "Specific query, e.g. 'refund policy for digital products'" },
"filter_category": { "type": "string", "enum": ["policies","technical","billing","general"] }
},
"required": ["query"]
}
}
Deploy as an MCP server - works with Claude, OpenAI, and any MCP-compatible framework without rewriting the integration.Output
Per our Subscription Refund Policy, you can request a full refund within 30 days of your initial purchase. Go to Settings > Billing to initiate the process. Would you like me to walk you through the steps?
Tools & Platforms
| Tool / Platform | Best for | Link |
|---|---|---|
| OpenAI API (GPT-5.5) | Production-grade function calling with parallel tool calls, tool_choice control, and JSON schema validation | https://platform.openai.com/docs/guides/function-calling |
| Anthropic Claude API | Tool use with strong reasoning; great at deciding when NOT to call a tool; MCP-native | https://docs.anthropic.com/en/docs/tool-use |
| Google Gemini API | Parallel function calling, multi-modal tool use (vision + functions), Google ecosystem integration | https://ai.google.dev/gemini-api/docs/function-calling |
| LangChain | Building agent pipelines with tool orchestration in Python/JS; large pre-built tool ecosystem | https://python.langchain.com/docs/concepts/tools/ |
| LlamaIndex | RAG + tool use for structured data retrieval agents; strong document-centric workflows | https://docs.llamaindex.ai/ |
| n8n | Visual no-code/low-code agent workflows with built-in tool nodes; great for debugging intermediate steps | https://n8n.io/ |
| Vercel AI SDK | Full-stack TypeScript/Next.js apps with streaming tool calls and React hooks | https://sdk.vercel.ai/docs/ai-sdk-core/tools-and-tool-calling |
| Fireworks AI | Fast open-source model inference with function calling support (Llama, Mixtral, etc.) | https://fireworks.ai/blog/function-calling |
| Groq | Ultra-low-latency inference for function-calling-capable open models | https://console.groq.com/docs/tool-use |
| Model Context Protocol (MCP) | Standardized tool server protocol: build once, connect to any MCP-compatible agent framework | https://modelcontextprotocol.io/ |
| Semantic Kernel (Microsoft) | Enterprise .NET and Python agent orchestration with function calling and Azure AI integration | https://learn.microsoft.com/en-us/semantic-kernel/ |
| Mistral AI API | Function calling on open-weight Mistral models; good for privacy-sensitive or self-hosted deployments | https://docs.mistral.ai/capabilities/function_calling/ |
Official Guides
OpenAI Function Calling Guide
Production-grade function calling with parallel calls and schema validation
Anthropic Tool Use Docs
Tool use with strong reasoning and MCP-native support
Anthropic: Writing Effective Tools for AI Agents
Best practices for tool descriptions, schemas, and agent tool design
Google Gemini Function Calling
Parallel and multi-modal function calling in the Google ecosystem
Model Context Protocol (MCP) Specification
Standardized open protocol for connecting AI models to tools and data sources
Frequently asked questions
What is function calling in AI models?
Function calling in AI models allows a large language model to recognize when a task requires external data or actions it cannot perform by itself. It generates a structured output, typically in JSON, to trigger an external function, API, or service, which then executes the task and returns the result to the model.
How do AI models decide which tool to use?
AI models decide which tool to use based on predefined tool definitions that include a name, description, and JSON parameter schema. The model evaluates these definitions to determine whether to respond directly or make a structured tool call, depending on the task.
What are the best practices for using AI tool calls?
Best practices for AI tool calls include writing clear tool descriptions that specify when to use them, using enums and strict types to limit parameter values, and ensuring that only necessary tools are registered to reduce token usage. Additionally, validate inputs and provide structured error messages to improve reliability.
What are common mistakes when implementing AI function calls?
Common mistakes include vague tool descriptions, registering too many tools at once, and failing to validate inputs before execution. These can lead to incorrect tool usage, increased latency, and security risks. It's important to dynamically load relevant tools and ensure robust error handling.
When should parallel tool calls be used in AI models?
Parallel tool calls should be used when a request requires data from multiple independent sources simultaneously. This approach significantly reduces latency compared to sequential calls, making it ideal for tasks like comparing weather in different cities or retrieving information from various databases at once.