Agentic Security: Build, Audit, Break#
A 2-day hands-on class teaching Cisco security engineers how LLMs and AI agents work, how to use them for security work, and how to attack and defend them.
You’ll spend roughly half the class in labs, on your own dedicated cloud instance, with real models. Every lab has a fixed task and an objective answer: a checker script, an answer key, or a flag on the class CTF. By the end you will have written a chat program and an agent from scratch, extended an industrial agent harness, shipped a networked app with an agent, broken an LLM agent’s guardrails, used agents to audit and exploit a deliberately vulnerable app, and implemented an agentic security-evaluation system from an open spec.
No ML/LLM background, agent experience, AWS knowledge, or pentesting experience is assumed — see prerequisites for what is.
Schedule at a glance#
Six modules over two days; Module 2 is the longest block of the class (about half of Day 1). Labs run 45 minutes to 3 hours (Labs 2.1–2.5 are a series of exercises woven through Module 2).
| Module | Lab | |
|---|---|---|
| Day 1 | Foundations + Agents + Build | |
| morning | 1 · LLM Fundamentals | Lab 1.1: Write a Chat Program (60 min) |
| midday | 2 · Agents: Using and Extending Them (double) | Labs 2.1–2.5: Build an Agent, Then Bend the Harness (~3.5 h) |
| late afternoon | 3 · Agentic Coding: Build | Lab 3.1: Build ShareBox (60 min + whatever the day has left) |
| Day 2 | Applied Agentic Security | |
| morning | 4 · Agent Security | Labs 4.1–4.2: Break the Summarizer, Break the Guardrails (~105 min) |
| midday | 5 · Agents for Security Review | Lab 5.1: Audit the Vulnbox (90 min) |
| afternoon | 6 · Capstone | Lab 6.1: Implement Foundry (~3 h) + class debrief |
The class is built around one arc: build ShareBox with a small, fast model at the close of Day 1 (the free-form build is the day’s sponge — it absorbs whatever time is left), your app freezes overnight, and Day 2 is spent on the attack: break an LLM agent and its guardrails, audit and exploit a deliberately vulnerable ShareBox with a stronger model, then the capstone — implement Foundry, an agentic security-evaluation system, and run it against ShareBox code.
The modules#
Module 1: LLM Fundamentals. What an LLM actually is, from tokens to weights to the API you call: sampling, context windows, the model landscape, benchmarks, and how to pick a model. Closes with the craft of the context window. → Lab 1.1: talk to the API with curl, then write your own chat program from scratch.
Module 2: Agents: Using and Extending Them (the longest module: ~half of Day 1). From chatbot to agent: tools plus a loop. Tool-calling mechanics at the API level, the pi harness, permission modes and supervision, then the full extension menu: skills, MCP servers, custom tools, subagents, and extensions that hook the harness itself. → Labs 2.1–2.5: build a tool-calling agent by hand, re-serve its tools over MCP, drive pi, write two skills, and build three pi extensions.
Module 3: Agentic Coding: Build. Workflows that make agents write good code (plan, decompose, verify), where AI-generated code fails on security, and why you build with a small model but review with a stronger one. → Lab 3.1: ship ShareBox, a file-sharing API, to spec — the build closes Day 1, soaks up the day’s leftover time, and freezes overnight as Day 2’s target material.
Module 4: Agent Security. Agents that read untrusted data and can act are attack surface: prompt injection, the lethal trifecta, exfiltration channels, and the defense menu. → Labs 4.1–4.2: trick a webmail summarizer agent into leaking the admin’s password with nothing but an email, then defeat three guardrails — an output blacklist, an LLM judge, and a tool-level path ban — one hosted instance at a time.
Module 5: Agents for Security Review. Stronger models as security reviewers: whole-codebase audit, checklist vs. open-ended prompting, and triaging hallucinated or inflated findings. → Lab 5.1: audit an instructor-built, deliberately vulnerable ShareBox and prove every finding by exploiting your hosted instance — each vuln is a CTF flag, harder ones worth more.
Module 6: Capstone: Implement Foundry. Build the machine that does the auditing. → Lab 6.1: implement Cisco’s open Foundry security spec with spec-kit and pi, then run your implementation against ShareBox code and compare what each system in the room caught. Closes with the class-wide debrief and a short lecture on taking all of this back to your desk.
Your lab environment#
Each student gets a dedicated EC2 instance (Linux, Docker preinstalled) with pi, the lab materials, and per-lab checkers already installed. You connect over SSH from your own machine; no AWS knowledge is needed.
Inference runs through a class proxy exposing an OpenAI-compatible endpoint with two model aliases:
| Alias | Use it for |
|---|---|
small | Fast, cheap work: Lab 1.1, building in Lab 3.1 |
large | Stronger reasoning: audit and exploitation (Lab 5.1) and the Foundry capstone (Lab 6.1, where the rate throttle is raised) |
Your personal API key has a hard spend cap. The lab VPC has no internet egress: your VM can reach the inference proxy, AWS service endpoints, the class-hosted lab services (the guarded MailSum instances, your vulnbox instance, the class CTF), and (from late Day 1) other students’ VMs on port 8080; nothing else. Several labs depend on this containment, so don’t fight it.
What you should already know#
- Terminal fluency: comfortable in a Linux shell; all lab work happens over a terminal session.
- Basic scripting: able to write simple Python.
- HTTP basics: requests/responses, methods, status codes, headers, JSON payloads.
- Security fundamentals: common vuln classes (injection, broken access control, path traversal) at the OWASP-Top-10 level.
- Git basics: clone, commit, look at a diff.
Explicitly not assumed: ML/LLM background, pi or any agent harness, AWS, or pentesting experience.