Skip to main content

​
Tool calls: how an LLM controls real things

Time: 10:05 AM to 10:25 AM
By default, an LLM can only generate text. Tool calling (also called function calling) lets the model do more: instead of just writing words, it can trigger real actions.

​
How tool calling works

1

Define available tools

You give the LLM a list of functions it can call, along with descriptions of what each function does and what arguments it accepts.
2

Model decides to use a tool

When the model determines it needs to take an action, it outputs structured JSON specifying which function to call and what arguments to pass.
3

Your code runs the function

Your program reads the JSON, calls the actual function, and gets a result.
4

Result goes back to the model

The function result is sent back to the model, which incorporates it and continues generating its response.

​
Robot tool examples

For your robot, this means defining functions like:
  • move_forward(speed, duration) - drive forward
  • turn_left(angle) - steer left by a specific angle
  • speak(text) - say something through the speaker
The LLM reads your voice command, decides which function to call, and your code executes the physical action.

​
The analogy

Think of it this way: the LLM is the brain, and tool calls are the motor nerves connecting it to a body. Without tool calls, the brain can think but cannot act. With tool calls, thoughts become physical motion.
After this section, take a 10-minute break (10:25 AM to 10:35 AM).