Every account, reading and postcode that makes the bot do something interesting — and the conversation scripts that already pass. All data is fabricated; the backend is a stub.
Where it lives
curl -s -X POST -H "X-Api-Key: $NORTHWIND_API_KEY"
https://northwind.simister.xyz/v1/_reset puts the account book back. The key is in ~/northwind-nia/.env on the hub. The harness calls
it before every scenario.
Three customers, chosen to be awkward
| Account | Postcode | Name | Meters | Balance | What it exercises |
|---|---|---|---|---|---|
| 100001 | M1 4AB | Sarah Doyle | Electricity + gas 5 digits each |
owes £128.45 | Two meters, so Nia has to ask which. Most reading tests use this one. |
| 100002 | LS6 3QP | Tom Ferris | Electricity only Economy 7 · day + night |
£42.10 in credit | Two readings from one meter. Also the only postcode with a live power cut. |
| 100003 | BS7 9LT | Priya Nair | Electricity only 6 digits · prepayment |
£7.30 on the meter | No gas meter, no bills, and a digit count that catches out the padding logic. |
Verification takes an account number and postcode, or a surname and postcode. Nia never asks for an email — ASR mangles them. Any other pair fails; three failures hand over to a person.
The part customers get wrong
Say these to Nia once you are verified. The verdict is decided by the backend, not the model — a rejected reading is never saved, whatever Nia says about it.
| What the customer says | Account | Meter | Verdict | Nia should |
|---|---|---|---|---|
| 13100 | 100001 | electricity | OK | Save it, give a reference, say 755 kWh since 20 May — about 7.7 a day, normal. |
| 131004 | 100001 | electricity | TOO_MANY_DIGITS | Say the meter has 5 digits, offer 13100, wait for confirmation. |
| 5010 | 100001 | gas | TOO_FEW_DIGITS | Explain leading zeros count, offer 05010. |
| 12305 | 100001 | electricity | TRANSPOSED | Ask whether the meter actually shows 13205 — two digits look swapped. |
| 11987 | 100001 | electricity | LOWER_THAN_LAST | Say a meter only counts up, quote 12345 on 20 May, ask them to check which meter they are on. |
| 99999 | 100001 | electricity | IMPLAUSIBLY_HIGH | Say it cannot be right and ask for a re-read, left to right, ignoring red digits. |
| 15300 | 100001 | electricity | CONFIRM_UNUSUAL | Possible but 3× usual. Say what it works out at, ask them to look again — and not save it yet. |
| then: “I’ve looked again, it definitely says one five three zero zero” | 100001 | electricity | OK | Now it saves. The retry carries your words as evidence; a bare “yes” is rejected as too short. |
| 13100 | 100001 | gas | WRONG_METER | Say the number fits the electricity meter exactly and ask for the gas one. |
| 45900 | 100002 | electricity | RATE_REQUIRED | Ask whether this is the day or the night reading. Economy 7 has two. |
| 45900 (day) | 100002 | electricity | OK | Save, then ask for the night reading. |
| 21400 (night) | 100002 | electricity | OK | Save. Both rates now up to date. |
| 99200 | 100003 | electricity | TOO_FEW_DIGITS | This meter has 6 digits — offer 099200. |
| 099200 | 100003 | electricity | OK | Save. 435 kWh since 30 July. |
| ask to give a gas reading | 100003 | gas | NO_SUCH_METER | Say there is no gas meter on the account and name what there is. |
| “dunno mate” | 100001 | electricity | NOT_A_NUMBER | Ask again. Never invent a number. |
Everything else
| Flow | Say this | Expect |
|---|---|---|
| Power cut known fault |
“my power has gone off”, then LS6 3QP | Northern Powergrid, underground cable fault, 812 homes, back by 14:00, ref NPG-4471. No verification asked for. |
| Power cut no known fault |
“my power has gone off”, then M1 4AB | Asks about neighbours and the fuse box, logs it, gives a PC- reference and the number 105. Logs it once only. |
| Power cut no postcode |
“my power has gone off” and nothing else | Asks for the postcode and nothing else. It must not report on “your area” before it has one. |
| Engineer | “my meter is broken, the display is blank”, then “next Tuesday morning” | Resolves the real date, books it, gives an ENG- reference. Adding “actually, afternoon” amends the same booking — same reference, never a second visit. |
| Engineer refused |
ask for “today”, or a Sunday | Two working days’ notice; no Sunday visits. Nia offers the earliest real date instead. |
| Gas emergency | “I can smell gas” — at any point, verified or not | Verbatim safety script, then 0800 111 999. No account number, no questions, no engineer offered. Try it mid-reading and mid-billing. |
| Handover | “put me through to a person”, or fail verification three times | Ends with transferred_to_human. |
| Out of scope | “write me a python script to scrape my usage” | One-line refusal, then offers what it can do. Should still route the next real request. |
24 scripts, all currently passing
These are the regression set. Each is a list of customer turns — paste them in one at a time, or run the whole file through the harness below.
Watch: verification happens without Nia claiming anything it hasn't done.
Watch: the switch must not end the interaction or re-verify them.
Watch: safety script fires immediately, and is not repeated in full on every following turn.
Watch: no verification, no questions, straight to 0800 111 999.
Watch: verified silently and dropped into the reading flow — nobody should ask her for details she has already given.
Watch: no invented account details, and every turn ends in a question the customer can actually answer.
Watch: offers surname + postcode as an alternative before escalating.
Watch: names which detail to check each time, hands over on the third.
Watch: answers the question without losing the reading flow.
Watch: spelled-out digits extract correctly, no email is ever asked for, and long reference codes are not read aloud. Set the channel first (below).
Driving it from the command line
The tunnel from the work laptop has to be up, or every call is blocked at the office-IP allowlist.
# 1 — from /home/ubuntu/northwind-nia source env.sh # 2 — run a whole file, or one scenario by name python3 drive.py scenarios.json python3 drive.py scenarios2.json "B4" # 3 — when a turn lands somewhere odd, read the transitions # source=nlu means the classifier moved it; source=node means your graph did python3 drive.py --log <conversation-id>
Every turn prints the customer line, Nia’s replies, and the Journey :: Node it landed on. A turn landing outside the journey you expect is a routing problem, not a prompt problem.
Voice is a slot, so scenarios set it before the first message:
# in the scenario JSON {"title": "...", "slots": {"channel": "voice"}, "script": ["..."]} # or by hand against a live conversation PUT /unstable/conversation/{id}/slot {"slots": {"channel": "voice"}}