The Stack Under the Model

<p class="kicker">MST0052 · Lecture 2 · Extra</p>

## The stack<br>under the model

### The tools a working ML engineer uses, beyond scikit-learn

Vegard H. Larsen — BI Norwegian Business School

---

The model is the **smallest part** of the system.

Everything this course teaches is one box in a much larger picture.

---

## The map

![The machine-learning stack, from the operating system up to operations](../figures/ml-stack-layers.svg)

<p class="muted">Colour says <strong>when</strong> a layer is worth learning, not how important it is. We take them from the bottom up.</p>

---

## Linux, and which one

Every server, container, CI runner and rented GPU you will meet runs **Linux**. macOS is Unix underneath, so nearly everything transfers.

| Distribution | Where you meet it | Packages |
|---|---|---|
| **Ubuntu** | The default — cloud images, WSL, most tutorials | `apt` |
| **Debian** | Ubuntu's upstream, and the base of `python:3.11-slim` | `apt` |
| **Fedora** | Leading edge, on a six-month cycle; upstream of Red Hat | `dnf` |
| **Alpine** | Very small container images | `apk` |
| **Arch, NixOS** | Enthusiasts; NixOS for exact reproducibility | `pacman`, `nix` |
| **Gentoo** | Built from source and tuned per machine | `emerge` |

Same kernel, same filesystem, same shell. What differs is the **package manager** and how fast things change — so learning one gets you all of them.

<p class="muted">If nobody tells you otherwise, assume Ubuntu. It is what <code>wsl --install</code> gives you, what most cloud images boot, and what the tutorial you are reading was written against.</p>

---

## The terminal: three things people confuse

<div class="cols-3">
<div class="callout">
<h3>The emulator</h3>
<p>The window that draws the text. <strong>Ghostty</strong> and <strong>WezTerm</strong> are the current favourites; <strong>iTerm2</strong> on macOS, <strong>Windows Terminal</strong> on Windows.</p>
</div>
<div class="callout">
<h3>The shell</h3>
<p>The program reading what you type. <code>bash</code> is what scripts assume, <code>zsh</code> is the macOS default, <code>fish</code> is friendlier but not POSIX.</p>
</div>
<div class="callout">
<h3>The commands</h3>
<p><code>ls</code>, <code>grep</code>, <code>ssh</code> are separate programs, identical in every window and every shell. These are what you actually learn.</p>
</div>
</div>

```bash
ls -lh data/              # what is here, and how big
head -3 train.csv         # look at the data before you load it
wc -l train.csv           # 1.2M rows — so don't open it in Excel
du -sh ~/.cache           # what is eating the disk
ps aux | grep python      # is my training run still alive?
ssh vegard@gpu-01         # work on a machine that isn't mine
```

<p class="muted">Small programs, one job each, composed with <code>|</code> — the design that outlived every GUI built to replace it. Worth learning early: tab completion, <code>Ctrl-R</code> to search your history, and <code>tmux</code> so a remote training run survives your laptop closing.</p>

---

## Windows: put Linux inside it

<p class="kicker">WSL 2 · one-time setup, then you are on Linux</p>

```powershell
wsl --install            # PowerShell as administrator, then reboot
```

```bash
sudo apt update && sudo apt install python3-venv python3-pip
mkdir ~/project && cd ~/project        # in ~, NOT in /mnt/c
python3 -m venv .venv && source .venv/bin/activate
code .                   # VS Code renders on Windows, runs in Linux
```

- **Keep project files in `~`, not `/mnt/c`** — crossing the filesystem boundary is many times slower
- **One Python, not two** — a venv built by Windows Python will not run inside WSL
- **Install the VS Code "WSL" extension** — editor on Windows, interpreter and terminal in Linux
- **GPUs work** — install the Windows NVIDIA driver only; never a Linux GPU driver inside WSL

</div>

---

## Two habits make work reproducible

<div class="cols">
<div class="callout">
<h3>Version the code</h3>
<p><code>git</code> — every change has an author, a message and a way back. Commit code and configuration. Never commit data, and never commit keys.</p>
</div>
<div class="callout">
<h3>Declare the environment</h3>
<p><code>python -m venv .venv</code> and a <code>requirements.txt</code>. Another machine has to be able to rebuild yours from the file alone.</p>
</div>
</div>

"It works on my machine" is not an inconvenience in a graded project. It is an **unverifiable result**.

---

## Docker: the environment as an artifact

```dockerfile
FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["python", "train.py"]
```

An **image** is that filesystem, frozen. A **container** is a running copy of it. Those six lines behave identically on your laptop, a colleague's laptop and a rented GPU.

<p class="muted"><code>docker build -t churn .</code> then <code>docker run --rm churn</code>. Underneath, every managed cloud training and serving service runs your code exactly like this.</p>

---

## The cloud is three things you rent

<div class="cols-3">
<div class="callout">
<h3>Compute</h3>
<p>A Linux VM by the hour. A GPU box for training. Spot instances cheaper, if your job can survive being interrupted.</p>
</div>
<div class="callout">
<h3>Storage</h3>
<p>A bucket — S3, GCS, Azure Blob. Cheap and effectively unlimited, but not a filesystem: you <code>get</code> and <code>put</code> whole objects.</p>
</div>
<div class="callout">
<h3>Managed services</h3>
<p>Someone else operates the training job, the endpoint, the database. Dearer per hour, far cheaper in your time.</p>
</div>
</div>

<div class="callout warn">
<p><strong>The bill is what people actually get wrong.</strong> A GPU instance runs roughly $1–4 an hour depending on provider — a forgotten one is a four-figure bill by the end of the month. Set a budget alert on day one, and stop instances you are not using.</p>
</div>

---

## SQL is the layer you will use most

In most organisations the data does not arrive as a CSV. It sits in a warehouse, and the way in is SQL.

```sql
SELECT customer_id,
       COUNT(*)        AS n_orders,
       SUM(amount)     AS total_spend,
       MAX(order_date) AS last_order
FROM orders
WHERE order_date < '2026-01-01'   -- nothing from after the cutoff
GROUP BY customer_id;
```

`GROUP BY` is feature engineering. That `WHERE` clause is **leakage control** — and it is the first place leakage gets built in, long before any Python runs.

<p class="muted">Warehouses you will hear named: BigQuery, Snowflake, Databricks, Postgres. On disk expect <strong>parquet</strong> rather than CSV — columnar, typed, compressed, often several times smaller. <code>pandas.read_sql</code> and <code>pandas.read_parquet</code> are the bridges back to this course.</p>

---

## Your model behind a URL

An API is just this: a **request** arrives over HTTP, your code answers. Two halves of one conversation.

<p class="kicker">What some other system sends</p>

```bash
curl -X POST https://api.yourcompany.com/predict \
     -H "Authorization: Bearer $API_KEY" \
     -d '[{"bill_length_mm": 39.1, "island": "Torgersen"}]'
```

<p class="kicker">Your side, answering</p>

```python
@app.post("/predict")
def predict(rows: list[dict]):
    return {"prediction": pipe.predict(pd.DataFrame(rows)).tolist()}
```

<p class="muted">Start it with <code>uvicorn app:app</code>. That is the whole difference between a model <strong>you</strong> have and a model <strong>other people</strong> can use.</p>

---

## Give your work a command line

```python
# train.py
import argparse

parser = argparse.ArgumentParser()
parser.add_argument("--data", required=True)
parser.add_argument("--model", default="ridge", choices=["ridge", "rf"])
parser.add_argument("--seed", type=int, default=42)
args = parser.parse_args()
```

A notebook you ran in an order you can no longer reconstruct is not a result. **A command is.**

<p class="muted">Run it as <code>python train.py --data churn.csv --model rf --seed 7</code> — and that line goes straight into the methods section of your report. Same shape as <code>git commit -m</code>, <code>docker run --rm</code>, <code>aws s3 cp</code>: a verb, then flags. Learn to read one and you can read all of them.</p>

---

## The rest of the iceberg

Names worth recognising. You are not expected to learn these now.

<div class="cols-3">
<div class="callout">
<h3>Experiment tracking</h3>
<p>MLflow, Weights & Biases — which run produced that number, and with what settings?</p>
</div>
<div class="callout">
<h3>Orchestration</h3>
<p>Airflow, Prefect, dbt — something has to run the pipeline at 04:00 every morning.</p>
</div>
<div class="callout">
<h3>Monitoring and drift</h3>
<p>The world keeps changing; your training data does not. Accuracy decays quietly.</p>
</div>
<div class="callout">
<h3>CI/CD</h3>
<p>GitHub Actions — tests and checks run on every commit, not when someone remembers.</p>
</div>
<div class="callout">
<h3>Model registry</h3>
<p>Which version is live, who approved it, and how do you roll it back?</p>
</div>
<div class="callout">
<h3>Kubernetes</h3>
<p>How containers get scheduled across machines. Almost certainly someone else's job.</p>
</div>
</div>

<p class="muted">Only one of these is a statistics problem — and <strong>drift</strong> is the one most likely to bite a model you built yourself.</p>

---

## What to learn, and in what order

<div class="cols-3">
<div class="callout">
<h3>This semester</h3>
<p>The terminal · virtual environments · git · your pipeline as a script</p>
</div>
<div class="callout">
<h3>For a first job</h3>
<p>SQL · Docker · one cloud, any cloud · calling and serving an API</p>
</div>
<div class="callout">
<h3>Later, or never</h3>
<p>Orchestration · model registries · Kubernetes</p>
</div>
</div>

For the project you need exactly four things: a terminal you are comfortable in, a virtual environment, a git repository, and your pipeline in a `.py` file you can run from the command line.

<p class="muted">Setup instructions for all four are on the <strong>Tools</strong> page of the course site.</p>