> ## Documentation Index > Fetch the complete documentation index at: https://stem-docs.intellectualpoint.com/llms.txt > Use this file to discover all available pages before exploring further. # Robot reporter > Build a robot that tours the room, captures photos at each stop, sends them to GPT-4o-mini vision, and narrates what it sees aloud. ## Robot reporter: the final project **Time**: 10:50 AM to 11:40 AM This project ties together everything from the entire week into a single script. The robot drives to a series of positions, captures a photo at each stop, sends the image to GPT-4o-mini vision, and speaks a description of what it sees. ### What this combines | Capability | Where you built it | How it is used | | --------------------- | ------------------ | -------------------------------------- | | Movement | Days 1-2 | Robot drives between tour stops | | Text-to-speech | Day 2 | Robot narrates each stop aloud | | OpenAI API | Day 3 | Sends images and receives descriptions | | Vision language model | Day 4 | GPT describes what the camera sees | Everything converges here. One script, all five layers. ### The tour loop ```mermaid theme={null} flowchart TD Start["Speak intro"] --> Move["Drive to\nnext stop"] Move --> Capture["📷 Capture\nphoto"] Capture --> Send["Send to\nGPT-4o-mini vision"] Send --> Narrate["🔊 Speak\ndescription"] Narrate --> More{"More stops?"} More -->|Yes| Move More -->|No| Outro["Speak\noutro"] ``` *** ## Building the program ### Step 1 — Define the tour stops The tour is a list of stops. Each stop has a description (for your terminal), a movement action, and movement parameters: ```python theme={null} TOUR_STOPS = [ ("Stop 1: Straight ahead", "forward", {"speed": 25, "duration": 2}), ("Stop 2: Look right", "turn_right", {"degrees": 90}), ("Stop 3: Move forward", "forward", {"speed": 25, "duration": 2}), ("Stop 4: Look left", "turn_left", {"degrees": 90}), ("Stop 5: Final position", "forward", {"speed": 25, "duration": 1}), ] ``` Available movement actions: `"forward"`, `"backward"`, `"turn_left"`, `"turn_right"`, `"pause"`. Customize these to match your room layout. Add more stops, change angles, or extend distances. ### Step 2 — Define the reporter persona The system prompt controls how the robot narrates. This is where you get creative: ```python theme={null} REPORTER_PERSONA = """You are a news reporter doing a live broadcast from inside a room. Describe what you see in 1-2 exciting sentences as if reporting live on TV. Be specific about objects, colors, and anything interesting. Always start with 'And here we can see...' or 'Looking around, I notice...'""" ``` ### Step 3 — The movement function A simple function that executes each movement action: ```python theme={null} def do_movement(px, action, args): if action == "forward": px.forward(args.get("speed", 25)) time.sleep(args.get("duration", 2)) px.stop() elif action == "turn_right": degrees = args.get("degrees", 90) px.set_dir_servo_angle(35) px.forward(25) time.sleep(degrees / 30.0) px.stop() px.set_dir_servo_angle(0) # ... similar for backward, turn_left, pause ``` ### Step 4 — The vision function Capture a frame and ask GPT to describe it using the reporter persona: ```python theme={null} def ask_vision(client, image_b64, persona): response = client.chat.completions.create( model="gpt-4o-mini", messages=[ {"role": "system", "content": persona}, { "role": "user", "content": [ {"type": "text", "text": "Describe what you see in this image."}, {"type": "image_url", "image_url": { "url": f"data:image/jpeg;base64,{image_b64}", "detail": "low", }}, ], }, ], max_tokens=150, ) return response.choices[0].message.content.strip() ``` ### Step 5 — The tour loop The main loop drives to each stop, captures, describes, and narrates: ```python theme={null} speak(INTRO_LINE) for i, (description, action, args) in enumerate(TOUR_STOPS): print(f" ── {description} ({i+1}/{len(TOUR_STOPS)}) ──") # Move to position do_movement(px, action, args) # Capture and describe image_b64 = capture_frame(cam) narration = ask_vision(client, image_b64, REPORTER_PERSONA) # Narrate print(f" Reporter: {narration}") speak(narration) speak("And that concludes our tour! This is PiCar X, signing off.") ``` No keyboard input needed after launch — the robot runs the entire tour autonomously. ### Run it ```bash theme={null} sudo python3 ~/camp/day5/robot_reporter.py ``` Press **Enter** to start the tour. Press **Ctrl-C** at any time to stop. Test the stop control (Ctrl-C) before running the full tour. Make sure the robot has clear space to move through all its stops. Expected output: ``` ================================================== PiCar-X — Robot Reporter ================================================== Initializing robot... Robot ready. Starting camera... Camera ready. OpenAI client ready. Tour has 5 stops. Press Ctrl-C at any time to stop. Press ENTER to start the tour... Reporter: Good afternoon! This is PiCar X reporting live. Let me take you on a tour! ── Stop 1: Straight ahead (1/5) ── Moving: forward {'speed': 25, 'duration': 2} Capturing photo... Sending to GPT-4o-mini vision... Reporter: And here we can see a long hallway with fluorescent lights. There's a door at the far end with a green exit sign above it. ── Stop 2: Look right (2/5) ── Moving: turn_right {'degrees': 90} Capturing photo... Sending to GPT-4o-mini vision... Reporter: Looking around, I notice a whiteboard covered in colorful sticky notes and a desk with several laptops. ``` ```python theme={null} #!/usr/bin/env python3 """ Robot Reporter — tours the room, captures photos, narrates what it sees. Combines movement (Days 1-2), TTS (Day 2), OpenAI API (Day 3), VLM (Day 4). Usage: sudo python3 robot_reporter.py """ import sys, os, io, time, base64 sys.path.insert(0, os.path.expanduser("~/camp")) from secret import OPENAI_KEY from openai import OpenAI from picamera2 import Picamera2 from picarx import Picarx # ── CUSTOMIZE THESE ────────────────────────────────────────── REPORTER_PERSONA = """You are a news reporter doing a live broadcast from inside a room. Describe what you see in 1-2 exciting sentences as if reporting live on TV. Be specific about objects, colors, and anything interesting. Always start with 'And here we can see...' or 'Looking around, I notice...'""" INTRO_LINE = "Good afternoon! This is PiCar X reporting live. Let me take you on a tour!" TOUR_STOPS = [ ("Stop 1: Straight ahead", "forward", {"speed": 25, "duration": 2}), ("Stop 2: Look right", "turn_right", {"degrees": 90}), ("Stop 3: Move forward", "forward", {"speed": 25, "duration": 2}), ("Stop 4: Look left", "turn_left", {"degrees": 90}), ("Stop 5: Final position", "forward", {"speed": 25, "duration": 1}), ] # ── END CUSTOMIZATION ──────────────────────────────────────── def speak(text, speed=150): safe = text.replace("'", "'\\''").replace('"', '\\"') os.system(f"espeak -s {speed} '{safe}' 2>/dev/null") def capture_frame(cam): stream = io.BytesIO() cam.capture_file(stream, format="jpeg") stream.seek(0) return base64.b64encode(stream.read()).decode("utf-8") def ask_vision(client, image_b64, persona): response = client.chat.completions.create( model="gpt-4o-mini", messages=[ {"role": "system", "content": persona}, {"role": "user", "content": [ {"type": "text", "text": "Describe what you see in this image."}, {"type": "image_url", "image_url": { "url": f"data:image/jpeg;base64,{image_b64}", "detail": "low"}}]}], max_tokens=150) return response.choices[0].message.content.strip() def do_movement(px, action, args): if action == "forward": px.forward(args.get("speed", 25)); time.sleep(args.get("duration", 2)); px.stop() elif action == "backward": px.backward(args.get("speed", 25)); time.sleep(args.get("duration", 2)); px.stop() elif action == "turn_left": d = args.get("degrees", 90); px.set_dir_servo_angle(-35) px.forward(25); time.sleep(d / 30.0); px.stop(); px.set_dir_servo_angle(0) elif action == "turn_right": d = args.get("degrees", 90); px.set_dir_servo_angle(35) px.forward(25); time.sleep(d / 30.0); px.stop(); px.set_dir_servo_angle(0) elif action == "pause": time.sleep(args.get("duration", 2)) time.sleep(0.5) def main(): print("=" * 50) print(" PiCar-X — Robot Reporter") print("=" * 50) px = Picarx(); px.stop(); px.set_dir_servo_angle(0) cam = Picamera2() cam.configure(cam.create_still_configuration(main={"size": (640, 480)})) cam.start(); time.sleep(2) client = OpenAI(api_key=OPENAI_KEY) print(f"\n Tour has {len(TOUR_STOPS)} stops.") print(" Press Ctrl-C at any time to stop.\n") input(" Press ENTER to start the tour...") print(f"\n Reporter: {INTRO_LINE}") speak(INTRO_LINE); time.sleep(1) try: for i, (desc, action, args) in enumerate(TOUR_STOPS): print(f"\n ── {desc} ({i+1}/{len(TOUR_STOPS)}) ──") print(f" Moving: {action} {args}") do_movement(px, action, args) print(" Capturing photo...") image_b64 = capture_frame(cam) print(" Sending to GPT-4o-mini vision...") try: narration = ask_vision(client, image_b64, REPORTER_PERSONA) except Exception as e: narration = f"Technical difficulties at stop {i+1}!" print(f" ERROR: {e}") print(f" Reporter: {narration}") speak(narration); time.sleep(1) outro = "And that concludes our tour! This is PiCar X, signing off." print(f"\n Reporter: {outro}") speak(outro) except KeyboardInterrupt: print("\n\n Tour interrupted!") speak("Tour stopped. Signing off!") finally: px.stop(); px.set_dir_servo_angle(0) cam.stop(); cam.close() print("\n Tour complete!\n") if __name__ == "__main__": main() ``` *** ## What you modify You are changing three things in the program: ### 1. Tour stops and movement Add more stops, change turn angles, or extend distances. Map out a route through your workspace: ```python theme={null} TOUR_STOPS = [ ("Entrance", "pause", {"duration": 1}), ("Down the hall","forward", {"speed": 30, "duration": 4}), ("Check left", "turn_left", {"degrees": 90}), ("Scan room", "pause", {"duration": 1}), ("Continue", "turn_right", {"degrees": 90}), ("End of hall", "forward", {"speed": 25, "duration": 3}), ("Turn around", "turn_right", {"degrees": 180}), ("Return", "forward", {"speed": 30, "duration": 4}), ] ``` ### 2. Reporter persona Change the system prompt to give the robot a different voice: | Persona | System prompt | | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------ | | Nature documentary | *You are a nature documentary narrator. Describe what you see as if observing wildlife in its natural habitat. Use a calm, reverent tone.* | | Sports commentator | *You are a sports commentator. Describe what you see with high energy and play-by-play excitement. Everything is thrilling!* | | Art critic | *You are a snooty art critic visiting a gallery. Critique everything you see with dramatic flair and strong opinions.* | | Detective | *You are a detective investigating a crime scene. Describe everything you see as potential evidence, noting details others might miss.* | ### 3. Intro and outro lines Write an opening and closing line that matches your persona: ```python theme={null} INTRO_LINE = "Welcome to National Geographic. Today we explore... the classroom." # At the end of the tour: outro = "And that's a wrap from the field. This is PiCar X, returning to the studio." ``` *** ### Wrap-up discussion This single script uses movement from Days 1-2, text-to-speech from Day 2, the OpenAI API from Day 3, and GPT-4o vision from Day 4. The robot perceives, reasons, and acts — all in one loop. That is the full stack of embodied AI, built by you over five days. The same architecture (perceive → reason → act) is used in self-driving cars, warehouse robots, and drone navigation systems.