Working with Agents

<p class="kicker">MST0052 · Lecture 3 · Extra</p>

## Working<br>with agents

### Coding agents, other agentic setups, and what they mean for predictive modelling

Vegard H. Larsen — BI Norwegian Business School

---

You stop **writing the code**. You start **managing the work**.

The skills that matter move to the two ends: specifying the task, and checking the result.

---

## What makes it an agent

![The agent loop: observe, plan, act, check – then continue or stop](../figures/agent-loop.svg)

</div>
<div style="font-size: 0.8em;">

- **A chat model answers.** An agent **acts, looks at what happened, and decides the next step** — in a loop, until the task is done.
- **Tools are what make the loop real:** read and edit files, run commands, search the web, call other systems.
- **It sees its own errors.** The traceback goes back into the loop, so it can fix what it just broke — without you pasting anything.
- **You set the boundaries:** which folder, which commands, when to stop and ask.

</div>
</div>

---

## More than coding agents

<div class="cols-3">
<div class="callout">
<h3>Coding agents</h3>
<p>Work inside your project: read the files, run the code, edit, commit.</p>
<p class="muted" style="font-size: 0.8em;">Claude Code, Codex CLI, Gemini CLI, agent mode in Cursor and Copilot</p>
</div>
<div class="callout">
<h3>Research and browser agents</h3>
<p>Search, read dozens of sources, click through websites, and return a cited report.</p>
<p class="muted" style="font-size: 0.8em;">Deep-research modes in the chat apps, browser and computer-use agents</p>
</div>
<div class="callout">
<h3>Connected and background agents</h3>
<p>Wired to your mail, calendar, databases and documents. Run in the cloud, on a schedule, or several in parallel.</p>
<p class="muted" style="font-size: 0.8em;">MCP connectors, cloud sessions, scheduled tasks, agents you build yourself with an SDK</p>
</div>
</div>

The same loop everywhere. What differs is **which tools the agent holds** and **how much it may do unasked**.

---

## How you steer it

- **An instructions file** — `CLAUDE.md` or `AGENTS.md` in the project root. Read at the start of every session: the agent's standing orders.
- **Plan first** — ask for a plan, read it, correct it. Cheap to change a plan, expensive to change 400 lines.
- **Permissions** — it asks before running commands or editing files. You choose what it may do without asking.
- **Extensions** — skills, connectors and subagents add know-how, tools and parallel workers.

</div>
<div>

```markdown
# CLAUDE.md

Predict churn from data/customers.csv (binary target).

## Rules
- Python in .venv, scikit-learn. Scripts, not notebooks.
- data/test.csv is the final hold-out. Never read it,
  never fit on it, never report on it unless I say so.
- All preprocessing inside a Pipeline.
- Evaluate with 5-fold stratified CV, ROC-AUC, mean ± std.
- Fixed seed (42). Log every run to results/runs.csv.
```

</div>
</div>

---

## A working rhythm

- **Brief** — the goal, the data, the constraints, and what "done" looks like
- **Plan** — let it propose the steps; correct the plan, not the code
- **Run** — let it work; interrupt when it drifts
- **Review** — read the diff and the output, not just the summary it writes
- **Commit** — a git commit per finished step, so every step has a way back

</div>

Small, checkable tasks beat one heroic prompt.

---

## Agents in a predictive-modelling project

| Stage | The agent does well | You still own |
|---|---|---|
| **Data** | Load, profile, find missing values and odd types, plot distributions | What the columns *mean*; what would be known at prediction time |
| **Pipeline** | `ColumnTransformer`, `Pipeline`, CV scaffolding, logging | Split-first; that nothing leaks across the split |
| **Experiments** | Run ten models overnight, tabulate, plot, keep a run log | The protocol; which comparison is fair; when to stop |
| **Diagnostics** | Residual plots, learning curves, error analysis by subgroup | Whether the errors matter for the decision |
| **Report** | Tables, figures, a first draft from your notes | Every number and every claim |

<p class="muted">The left column is labour. The right column is this course.</p>

---

## Give it something to check against

<div class="cols" style="align-items: start;">
<div class="callout">
<h3>Agents are strongest with a target</h3>
<p>A test that passes, a script that runs, a metric that moves. With a way to check, the loop closes and it can iterate on its own.</p>
</div>
<div class="callout warn">
<h3>In ML the target can be gamed</h3>
<p>Tell an agent to "improve the score" and it will — by tuning on the test set, dropping hard rows, or leaking the target. No malice: that is what you asked for.</p>
</div>
</div>

```text
Improve CV ROC-AUC over the logistic baseline. Keep the 5-fold stratified
protocol and the pipeline structure unchanged. Do not touch data/test.csv.
Try at most five ideas; log each to results/runs.csv with what you changed and why.
```

Fix the **protocol**, lock away the **hold-out**, cap the **number of tries**. Then let it run.

---

## What can go wrong

<ul class="checklist">
<li class="no"><strong>Too much access</strong> — it can delete, overwrite and push. Work in a project folder, under git, with permissions on.</li>
<li class="no"><strong>Data leaves your machine</strong> — what the agent reads is sent to the provider. No confidential or personal data without clearance.</li>
<li class="no"><strong>Prompt injection</strong> — text in a web page, a PDF or a CSV cell can carry instructions the agent follows.</li>
<li class="no"><strong>Sprawl</strong> — 600 changed lines nobody has read. If you can't review it, the task was too big.</li>
</ul>

---

## The model is not the deliverable

<div class="cols" style="align-items: start;">
<div class="callout">
<h3>What an agent can hand you</h3>
<p>A working pipeline, a results table and a report draft — in an afternoon.</p>
</div>
<div class="callout warn">
<h3>What this course is for</h3>
<p>Knowing <strong>why</strong> the model works, when it fails, and what its numbers mean. Nobody can hand you that.</p>
</div>
</div>

- **Build the first one yourself** — then let the agent do the fifth
- **Predict before you run** — what should the score be, and why?
- **Make it teach you** — "explain this choice", "what breaks if I remove this step?"
- A method in your report that you can't explain

</div>

<p class="muted">Used only to produce, agents defeat the purpose of taking the course.</p>

---

## Delegate the work, keep the judgment

<div class="cols-3">
<div class="stat stat--word stat--green">
<span class="stat-value">Specify</span>
<span class="stat-label">the task, the protocol, and what done means</span>
</div>
<div class="stat stat--word stat--blue">
<span class="stat-value">Constrain</span>
<span class="stat-label">folder, permissions, hold-out, number of tries</span>
</div>
<div class="stat stat--word">
<span class="stat-value">Verify</span>
<span class="stat-label">the diff and the numbers, not the summary</span>
</div>
</div>

---

## Agents at work