> ## Documentation Index
> Fetch the complete documentation index at: https://stem-docs.intellectualpoint.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Tool calls
> Learn how an LLM can control real things by calling functions, and how this connects the model's brain to the robot's body.
## Tool calls: how an LLM controls real things
**Time**: 10:05 AM to 10:25 AM
You just learned that an LLM is a brain in a jar — it can only generate text. So how do you make it drive a robot, read a sensor, or speak out loud?
The answer is **tool calling** (also called function calling). Instead of just writing words, the model can trigger real actions in your code.
***
### The problem
Without tool calls, here is what happens when you tell an LLM to move your robot:
> **You:** "Drive forward for 3 seconds."
>
> **LLM:** "Okay, I'm driving forward for 3 seconds!"
The robot does not move. The model generated the *words* "I'm driving forward" but has no way to actually control the motors. It is like telling someone to turn on the lights over a phone call — they can say "done!" but the lights stay off.
Tool calling fixes this by giving the model a menu of real functions it can ask your code to run.
***
### How tool calling works
```mermaid theme={null}
sequenceDiagram
participant You
participant YourCode as Your Python Code
participant GPT as GPT-4o-mini
participant Robot as PiCar-X
You->>YourCode: "Drive forward for 3 seconds"
YourCode->>GPT: Send message + list of available tools
GPT->>GPT: Decide which tool to call
GPT->>YourCode: Tool call: move_forward(duration=3)
YourCode->>Robot: px.forward(30); sleep(3); px.stop()
Robot-->>YourCode: Done
YourCode->>GPT: Result: "Drove forward for 3s"
GPT->>YourCode: "I just drove forward for 3 seconds!"
YourCode->>Robot: espeak("I just drove forward!")
```
You give the LLM a list of functions it can call, along with descriptions of what each function does and what arguments it accepts.
When the model determines it needs to take an action, it outputs structured JSON specifying which function to call and what arguments to pass. It does **not** run the function itself.
Your Python program reads the JSON, calls the actual function (which controls the robot's motors, servos, or speaker), and gets a result.
The function result is sent back to the model. The model incorporates the result and generates its final spoken response.
***
### What a tool definition looks like
When you send a message to GPT, you also send a list of tools. Each tool is a JSON object that describes a function. Here is one from the program you will run later today:
```json theme={null}
{
"type": "function",
"function": {
"name": "move_forward",
"description": "Drive the robot forward.",
"parameters": {
"type": "object",
"properties": {
"speed": {
"type": "integer",
"description": "Speed percentage (10-60). Default 30."
},
"duration": {
"type": "number",
"description": "How many seconds to drive. Default 2."
}
}
}
}
}
```
The model reads the `name`, `description`, and `parameters` to understand what the function does and what arguments to pass. It never sees your Python code — only this JSON description.
The quality of your descriptions matters. If you write a vague description like "does a thing", the model will not know when to use it. Clear, specific descriptions lead to better tool use.
***
### Your robot's tool menu
Your robot has **10 tools** the LLM can call. Here is the full menu:
| Tool | What it does | Physical action |
| --------------- | ---------------------------------------------------- | ------------------------------------ |
| `move_forward` | Drive forward at a given speed for a given duration | Wheels spin forward |
| `move_backward` | Drive backward at a given speed for a given duration | Wheels spin backward |
| `turn_left` | Turn left by a given number of degrees | Steering locks left, drives forward |
| `turn_right` | Turn right by a given number of degrees | Steering locks right, drives forward |
| `stop_car` | Stop all movement and straighten wheels | Everything stops |
| `speak` | Say text out loud through the speaker | Speaker plays espeak audio |
| `nod` | Nod the camera up and down | Camera tilts yes |
| `shake_head` | Shake the camera side to side | Camera pans no |
| `celebrate` | Do a happy dance with camera and wheels | Camera wiggles, car sways |
| `look_around` | Pan the camera left and right to scan | Camera sweeps side to side |
The model can call **multiple tools in one response**. If you say "make a square", it might call `move_forward` then `turn_right` four times in sequence.
***
### The brain and body analogy
```mermaid theme={null}
flowchart LR
Brain["LLM\n(the brain)"] -->|"tool calls\n(motor nerves)"| Body["Robot\n(the body)"]
Body -->|"results\n(sensory nerves)"| Brain
```
Think of it this way: the LLM is the **brain**, and tool calls are the **motor nerves** connecting it to a body. Without tool calls, the brain can think but cannot act. With tool calls, thoughts become physical motion.
The results that come back are like **sensory nerves** — they tell the brain what happened so it can decide what to do next.
***
### Why this matters beyond robots
Tool calling is not just for robots. It is the same pattern behind:
* **ChatGPT plugins** — the model calls web search, code execution, or image generation
* **AI assistants** — Siri and Alexa use function calling to set timers, play music, and control smart home devices
* **AI agents** — autonomous programs that browse the web, write code, and manage tasks
Every time an AI "does something" instead of just "says something", tool calling is what makes it happen. You are learning the exact same architecture that powers the most advanced AI systems in the world.
After this section, take a 10-minute break (10:25 AM to 10:35 AM).