Skip to main content

​
Foundational risks of AI in society

Time: 9:30 AM to 10:10 AM
This week you built a robot that sees, hears, speaks, and acts on LLM instructions. Today starts with a direct question: what could go wrong? This is not a hypothetical exercise. Every risk discussed here uses technology you have already built and tested.

​
Bias and hallucination

LLMs confidently generate incorrect information. The model does not know it is wrong — it is predicting likely tokens, not verifying facts. This is called hallucination.
You saw hallucination firsthand on Day 4 — the VLM sometimes described objects that were not in the image. Now imagine that same error rate applied to millions of decisions per day.
Discussion: When your robot hallucinated a description of something it could not see, was it lying? Can a machine lie if it has no concept of truth?

​
Surveillance and misuse

The same camera and VLM stack you built this week could be used to:
  • Identify faces in a crowd and track their movements
  • Classify behavior as “suspicious” based on training data biases
  • Monitor employees, students, or citizens at scale with no human oversight
The technology is identical. The difference between your helpful robot and a surveillance system is intent and governance, not hardware. Discussion: Should there be a law requiring a sign when AI-powered cameras are watching? Who decides what counts as “suspicious” behavior?

​
Concentration of power

The most capable AI models are controlled by a handful of companies: Your robot depends on OpenAI’s API. If OpenAI raises prices, changes terms, or goes offline, your robot stops thinking. This week you experienced this directly — without an API key from your facilitator, none of the Day 3-4 programs would have worked. Discussion: Is it a problem that critical AI infrastructure depends on private APIs? What happens when a company decides to discontinue a model you built a product on?

​
Autonomous weapons

The architecture you built this week — perceive, reason, act — is the same foundation used in lethal autonomous weapon systems (LAWS). This is not theoretical. Systems that can identify targets and take action without human approval already exist.
Your robot’s architecture:
  1. See — camera captures the environment
  2. Think — LLM reasons about what to do
  3. Act — motors execute the decision
A lethal autonomous weapon uses the exact same loop. The difference is what the “act” step does. Discussion: Should machines ever make targeting decisions without human approval? Where do you draw the line between a helpful robot and a weapon?

​
The dual-use problem

The same research that enables a helpful robot enables a harmful one. There is no technical separation between the two. The code is identical — the difference is intent and governance.
Ground this in what you built: you just built a robot that can identify objects and move toward them. Now imagine that at scale, with different intent. These are real stakes, not abstract ones.

​
What can be done?

There are no easy answers, but there are concrete actions:
1

Transparency

Systems should disclose when AI is making decisions. Users should know when they are talking to a model, not a human.
2

Accountability

Someone must be responsible when AI systems cause harm. “The algorithm did it” is not acceptable.
3

Access and competition

Open-source models (like Meta’s Llama) ensure no single company controls the technology. Open research prevents knowledge monopolies.
4

Regulation

Governments are actively developing AI regulation. The EU AI Act, US executive orders on AI safety, and international LAWS treaties are all in progress.
5

Individual responsibility

You — the person building these systems — make ethical choices every day about what to build, how to test it, and who it serves.

​
Discussion questions

Take 10 minutes to discuss 2-3 of these as a group:
  1. You built a robot that follows voice commands without question. Should AI systems ever refuse an instruction? When?
  2. Your robot’s training data came from the internet. What biases might it have absorbed? How would you test for them?
  3. If someone used your exact code to build something harmful, are you responsible? Is the API provider? Is anyone?
  4. Is open-source AI safer (everyone can inspect it) or more dangerous (everyone can misuse it)?