07. Function Calling & Tool Execution

1. The Tool Execution Loop

A tool execution sequence requires a multi-turn lifecycle. The LLM acts as an orchestrator, signaling tool calls back to the application server for physical execution before ingesting the returned payload to formulate a final user-facing response.

2. Tool Declaration Schema Architecture

Tools are declared using JSON Schema specifications. The model reads the descriptions to determine suitability during inference:

3. Parallel Tool Calling vs. Sequential Execution

Modern models support Parallel Tool Calling, emitting multiple tool invocations within a single inference response.

4. Dual-Stack Implementations: Full Function Calling Loop

Registering a mock database lookup tool, processing the model's call request, executing the handler, and returning the final synthesized answer:

import json
from openai import OpenAI

client = OpenAI()

# 1. Local Tool Implementation
def query_user_tier(user_id: str) -> dict:
    # Simulated database lookup
    database = {"usr_42": {"tier": "PLATINUM", "retention_score": 0.94}}
    return database.get(user_id, {"tier": "FREE", "retention_score": 0.10})

# 2. Tool Schema Declaration
tools = [{
    "type": "function",
    "function": {
        "name": "query_user_tier",
        "description": "Fetch user loyalty tier and retention metrics by user ID.",
        "parameters": {
            "type": "object",
            "properties": {
                "user_id": {"type": "string", "description": "The unique user identifier (e.g. usr_42)"}
            },
            "required": ["user_id"]
        }
    }
}]

# 3. Initial Inference Turn
messages = [{"role": "user", "content": "What is the membership tier of usr_42?"}]
response = client.chat.completions.create(model="gpt-4o-mini", messages=messages, tools=tools)
choice = response.choices[0]

# 4. Handle Tool Calling Loop
if choice.message.tool_calls:
    messages.append(choice.message) # Append assistant's tool-request turn
    
    for tool_call in choice.message.tool_calls:
        if tool_call.function.name == "query_user_tier":
            args = json.loads(tool_call.function.arguments)
            result = query_user_tier(args["user_id"])
            
            # Append tool result turn
            messages.append({
                "role": "tool",
                "tool_call_id": tool_call.id,
                "content": json.dumps(result)
            })

    # 5. Final Synthesis
    final_response = client.chat.completions.create(model="gpt-4o-mini", messages=messages)
    print(final_response.choices[0].message.content)

import dev.langchain4j.agent.tool.Tool
import dev.langchain4j.memory.chat.MessageWindowChatMemory
import dev.langchain4j.model.openai.OpenAiChatModel
import dev.langchain4j.service.AiServices

// 1. Define Tool Service with Annotations
class UserMembershipTools {
    @Tool("Fetch user loyalty tier and retention metrics by user ID")
    fun queryUserTier(userId: String): String {
        return if (userId == "usr_42") {
            """{"tier": "PLATINUM", "retention_score": 0.94}"""
        } else {
            """{"tier": "FREE", "retention_score": 0.10}"""
        }
    }
}

// 2. High-Level AI Service Interface
interface CustomerSupportAgent {
    fun chat(message: String): String
}

fun main() {
    val chatModel = OpenAiChatModel.builder()
        .apiKey(System.getenv("OPENAI_API_KEY"))
        .modelName("gpt-4o-mini")
        .build()

    // 3. Bind Service with Reflective Tool Dispatch
    val agent = AiServices.builder(CustomerSupportAgent::class.java)
        .chatLanguageModel(chatModel)
        .tools(UserMembershipTools())
        .chatMemory(MessageWindowChatMemory.withMaxMessages(10))
        .build()

    // LangChain4j automatically extracts JSON schema, detects tool calls, 
    // runs queryUserTier() on JVM, and feeds the response back to the LLM.
    val response = agent.chat("What is the membership tier of usr_42?")
    println(response)
}

5. Exception Handling & Sandboxed Execution

• Graceful Degradation: If a downstream tool (such as a database query or external REST API) fails with a 500 error or timeout, do not crash the LLM conversation loop. Package the exception into a JSON error response: {"error": "Database query timed out after 3000ms"} and pass it back in the tool turn. The model can then apologize to the user or attempt an alternate strategy.
• Least Privilege Execution: Tools should never run un-sanitized raw arbitrary SQL or shell commands. Wrap operations in typed functions with strict parameter bounds to avoid privilege escalation.

6. Progressive Glossary

Sources & Reference Standards

• OpenAI Developer Platform: Function Calling and Structured Tool Execution
• Anthropic Documentation: Tool Use (Function Calling) Implementation Patterns
• LangChain4j Architecture: Declarative Tool Annotations and Dispatch Loop