AI Tools in a Machine-Learning Workflow

<p class="kicker">MST0052 · Lecture 2 · Fall 2026</p>

## AI Tools in a<br>Machine-Learning Workflow

### Vegard H. Larsen — BI Norwegian Business School

---

## Where we are

L1 set up the course. Today: **how** you'll work all semester.

![Semester timeline with four phases](../figures/semester-timeline.svg)

---

AI is good at **code**. ML is **statistical reasoning**.

The dangerous failures live in the reasoning layer — that's why this gets a whole lecture.

---

## Today's plan

<p class="kicker">90 minutes · one live demo</p>

- Where AI tools genuinely **help** — and where they **fail silently**
- **Live demo:** Claude Code on a real ML task
- Using AI **defensibly** in your project

---

## AI as a junior collaborator

---

## The mental model

A fast, well-read **junior** — who has never seen your data.

![AI-human workflow loop](../figures/ai-workflow-loop.svg)

---

## Where AI helps

- **Scaffolding** — project skeletons, `requirements.txt`, pipeline boilerplate
- **Debugging** — paste the traceback with context
- **Explaining** — unfamiliar code, methods, error messages
- **Learning** — methods beyond the syllabus

</div>

You still need to understand what it does and why you used it.

---

## Where AI fails silently

---

## The category that matters most

The failures that matter throw **no errors**.

![Silent failures iceberg](../figures/silent-failures-iceberg.svg)

---

## Three silent failures

- **Statistical reasoning** — syntactically correct, statistically wrong
- **Your data** — assumes i.i.d., never spots a leaked feature
- **References** — confident, plausible, nonexistent

---

## The cross-validation leakage pattern

<p class="kicker">Wrong — the scaler sees every fold</p>

```python
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)      # leakage!
scores = cross_val_score(model, X_scaled, y, cv=5)
```

<p class="kicker">Right — scaling inside the pipeline</p>

```python
pipe = make_pipeline(StandardScaler(), LogisticRegression())
scores = cross_val_score(pipe, X, y, cv=5)
```

This is exactly the code an LLM writes for "evaluate with cross-validation."

---

## Red flags you should never ignore

<ul class="checklist">
<li class="no">The accuracy that looks <strong>too good</strong></li>
<li class="no">The CV score that matches the test score to three decimals</li>
<li class="no">The result that contradicts a sanity check</li>
<li class="no">The code you <strong>can't explain</strong> line-by-line</li>
</ul>

---

## Live demo — Claude Code

---

## The 2026 landscape

**Editor**   `Copilot` `Cursor`

**Terminal**   `Claude Code` `Codex CLI` `Gemini CLI`

**Chat**   `ChatGPT` `Claude` `Gemini`

</div>

<p class="muted">They overlap heavily — full detail on the <strong>AI page</strong> of the course site.</p>

---

## Why the terminal

<div class="cols">
<div class="callout">
<h3>Chat window</h3>
<p>You paste context. It never sees your actual project.</p>
</div>
<div class="callout">
<h3>Terminal agent</h3>
<p>Reads your files, runs your code, iterates — in your project.</p>
</div>
</div>

The closer the AI is to your **workflow**, the more useful it becomes.

---

## Prompt, verify, document

---

## A weak prompt vs a strong prompt

<div class="cols">
<div class="callout warn">
<h3>Weak</h3>
<p>"Help me classify this data."</p>
</div>
<div class="callout">
<h3>Strong</h3>
<p>"12k rows, binary churn target, 30 mixed features. Baseline logistic regression in a sklearn pipeline, 5-fold stratified CV, ROC-AUC ± std."</p>
</div>
</div>

The strong prompt is what you'd ask a **thoughtful colleague**.

---

## Verification habits

- **Run it** — don't just read it
- **Check** shapes, ranges, NaNs after every step
- **Ask** — "what would I see if this were wrong?"
- **Push back** — "what could be wrong with this analysis?"

</div>

Cross-check the official docs — 5 seconds of verification saves a week of confusion.

---

## Wrap-up

---

## Before Lecture 3

- **Pick one or two AI tools** — set up and working
- **GitHub Student Developer Pack** — free Copilot Pro for students
- **Have a candidate dataset** — L3 is preprocessing
- **Write out the leakage fix** — the pipeline pattern, on any dataset

</div>

Access and cost: the AI page on the course site.

---

## Three habits for the semester

<div class="cols-3">
<div class="stat stat--word stat--green">
<span class="stat-value">Read</span>
<span class="stat-label">what the AI writes</span>
</div>
<div class="stat stat--word stat--blue">
<span class="stat-value">Verify</span>
<span class="stat-label">what you can't explain</span>
</div>
<div class="stat stat--word">
<span class="stat-value">Document</span>
<span class="stat-label">what you used</span>
</div>
</div>

---

## Questions

Backup slides below — press down ↓

## Second leakage example: target encoding

```python
from category_encoders import TargetEncoder

# Wrong: fit on full data, then cross-validate
X["city_encoded"] = TargetEncoder().fit_transform(X["city"], y)
scores = cross_val_score(model, X, y, cv=5)
```

The encoder saw **target values** from the validation fold during fit.

Fix: wrap the encoder inside the pipeline that goes into CV.

## A worked debugging session

<p class="kicker">Error → context → ask → verify</p>

```text
ValueError: Input contains NaN, infinity
or a value too large for dtype('float64').
```

**Good prompt:** "I get this error on `pipe.fit(X_train, y_train)`. Here is `X_train.info()`: [paste]. Here is the pipeline: [paste]. What's most likely wrong?"

Why it works: full context — error, data, code.

## Comparing two tools on the same task

> "Add 5-fold CV with ROC-AUC to my logistic regression pipeline on this CSV."

<p class="muted">Terminal agents fail less on data-specific judgment — but <strong>neither replaces your review</strong>.</p>

## Hallucinated sklearn API

```python
from sklearn.ensemble import GradientBoostingClassifier
model = GradientBoostingClassifier(early_stopping=True)
```

`GradientBoostingClassifier` has **no `early_stopping` parameter** — you're thinking of `HistGradientBoostingClassifier`.

Always check the official API docs.

## AI for writing the report itself

- Drafting structure or paragraphs you then edit and verify
- Tightening prose you already wrote
- Turning your bullet-point notes into a section
- Analysis or numbers you can't reproduce from your code 
- AI-drafted text submitted verbatim, undisclosed 
- "Interpreting" results you haven't read yourself

</div>

<p class="muted"><strong>BI's misconduct rules apply</strong> to unattributed use and text you cannot account for. When in doubt: disclose it.</p>

## Notebooks vs scripts with AI agents

<div class="cols">
<div class="callout">
<h3>Notebooks</h3>
<p>Good for exploration. Bad for agents — cell state is invisible.</p>
</div>
<div class="callout">
<h3>Scripts</h3>
<p>Agents can read, edit, and run them end-to-end.</p>
</div>
</div>

Explore in notebooks; ship the final pipeline as a clean script.

---

## What's next

**Lecture 3:** Foundations and preprocessing pipelines

- Why preprocessing is part of the model
- scikit-learn Pipelines and ColumnTransformers
- The split-first rule