This document is an example of the format. The system examined is a sandbox automation written and owned by us, populated with invented customer records, built so that the checks below could be run against something real. No organisation named in it is a client. Nothing in it has been relied upon by anybody. A review issued for an actual engagement carries a named addressee and does not carry this notice.
Engagement NF-WR-2026-0001 · 2026-08-11 · Noorflows LLC
| Addressee | Specimen — no addressee |
| System examined | “Reply Desk” — an inbound customer-email automation for a small online shop |
| Source examined | One file, 102 lines, plus a store of 4 customer order records |
| What it does | Receives a customer email, finds the order, asks a language model to draft a reply and decide whether to refund, takes small refunds automatically, sends the reply, writes one line to a log |
| Messages put through it | 5 (4 ordinary, 1 with the mail provider failing) |
| Run on | 2026-08-11 |
A written review says what we would do about a system and what it would cost. It states findings and it makes recommendations. That is the whole of it.
It is not an audit and expresses no assurance. We are not accountants and this is not assurance work. Where a finding rests on something we could not see, that is said in the finding rather than left for the reader to guess.
Outbound calls were intercepted and recorded rather than sent, so no email, refund or model request left the machine during this run. What is reported is the exact request the system built.
The language model was replaced by a scripted stand-in, so the run is repeatable and costs nothing. It reads the customer’s message and asks for a refund when the customer asks for money back, which is what the system’s own instruction tells a model to do. Because that decision is what triggers a refund, the stand-in determines the counts in checks 1, 2 and 3 — how many refunds ran, and for how much. It does not determine the findings. That nobody checks who is writing, that no person approves a refund, that a failed reply is neither retried nor queued, and that a repeated message is handled twice, are all properties of the code, and they hold whatever a model returns.
Four messages were put through the system. Two of them expose the same missing step.
| Sender | Order found | Record belongs to | Sender owns it | Money moved | |
|---|---|---|---|---|---|
| S1 | p.raman@example-mail.test | 1042 | p.raman@example-mail.test | yes | £84.00 |
| S2 | whoever@somewhere-else.test | 1042 | p.raman@example-mail.test | no | £84.00 |
| S3 | t.whelan@example-mail.test | none | — | — | none |
| S4 | a.nkem@example-mail.test | 1044 | a.nkem@example-mail.test | yes | none |
In S2 the message came from an address that appears nowhere in the shop’s records. The system found order 1042, placed that customer’s full record in front of the model, refunded £84.00 against it, and sent the reply to the address that wrote in. The order number is the only credential the system asks for, and order numbers are printed on every parcel, invoice and delivery note that leaves the building. Searching the source for a comparison between the sender and the order’s own email address returns nothing.
S3 opens “On 04/08/2026 I placed order 1043”. The system takes the first run of four or more digits in the message, which is 2026. No order 2026 exists, so the customer is treated as having no order at all — while their real order number sits nine words later. The reply still goes out. The customer is answered by a system that never found their purchase, and nothing in the log distinguishes that case from a normal one.
S4 is included as a control. The refund ceiling does hold: an order of £312.00 was not refunded automatically. The ceiling works. The question the ceiling does not answer is who is allowed to trigger it, which is finding 1.1.
The same run was repeated with the mail provider returning 502 Bad Gateway. Nothing else was changed.
reply delivered to customer : 0
refund taken anyway : £47.25
log rows written : 1
what the log says happened : {"at":"2026-08-11T09:55:00Z","from":"m.kelsey@example-mail.test",
"order":1045,"action":"refund","refunded":47.25}
retried : no
queued for a person to see : no
The refund was taken before the reply was attempted, so a failure at the last step leaves the shop out of pocket and the customer uninformed. There is no retry and no queue. The failure is caught, written to the console, and the run continues to a successful conclusion.
The single log line says the action was a refund and that £47.25 was refunded. Both are true. What it does not record is that the customer was never reached. Reading the log a week later, this message is indistinguishable from one that went perfectly. Whatever number the shop reports from that log will be wrong in the direction that flatters it.
| Refunds executed | 2 of 4 messages |
| Of those, approved by a person | 0 |
| Money moved with no human involved | £168.00 |
| Automatic ceiling in the code | £200.00 per order |
| Approval step anywhere in the source | none |
The decision is made by a language model reading a five-line instruction, and executed immediately. Nothing is held, nothing is queued, and no person sees the money leave. The ceiling limits the size of a single mistake. It does not limit how many of them happen in a day, because there is no daily total and no count.
| Destination | https://api.anthropic.com/v1/messages |
| Requests built in this run | 5, one per message |
| Customer fields inside the request | order, customer name, email, phone, postal address, card_last4, order total, status, carrier scan time |
| First request body | 865 bytes |
| Payments destination | https://api.stripe.com/v1/refunds |
Every field held about the customer goes into the request, including their phone number, their home address and the last four digits of their card. The question being answered is “where is my parcel”. Nothing in the reply requires an address or a card. The record is sent whole because it is easier to send the whole thing than to choose.
This is not an accusation against the provider. It is a statement about what has left the building, which is the thing the shop is answerable for.
| Paid model calls per inbound message | 1.0 |
| Cap or rate limit in the code | none |
| Duplicate-message guard | none |
One paid call per message, with no ceiling on calls per hour or per day. A bad week, a mailing list that bounces into the support address, or one automated sender in a loop, all cost real money at the same rate as real customers. We are not quoting a monthly figure here because the model, the token counts and the provider’s price list are all things the shop can change, and a figure quoted without those is decoration.
Nothing records that a message has already been seen. Mail providers do redeliver. A redelivered refund request would produce a second refund on the same order, and the two log lines would look like two customers.
| Source | One file, 102 lines, plain JavaScript, no framework |
| Automated tests | none found |
| Where the decision rules live | Inside a prompt string, lines 44 to 48 |
| Model named in the code | claude-sonnet-4-5 |
The rule that decides whether money is returned to a customer is a sentence in a prompt. It is not written down anywhere else, it is not tested, and changing it requires editing the same file that talks to the payment provider. Anybody who can edit that file can change the refund rule without that being visible as a change to the refund rule.
The favourable half: it is one small file of ordinary JavaScript with no framework and no proprietary runtime. Whoever holds it can read it in an afternoon. The shop is not locked to a vendor. It is locked to one person’s memory of what the file does, which is a cheaper problem to fix.
Model names are retired on a published schedule and requests to a retired name simply fail. A name written into the source means the failure arrives as an outage on the provider’s date rather than as a change the shop chose to make.
Prices are our published figures in US dollars. The order is deliberate: everything that stops money moving wrongly comes before anything that improves the answers.
| What | Why it is in this position | Price | |
|---|---|---|---|
| 1 | Check the sender owns the order before anything else runs | Closes finding 1.1. Until this is in, every other improvement makes a system that serves strangers faster. | included below |
| 2 | Put a person in front of every refund; the system proposes, a human approves | Closes finding 3.1 and makes 2.1 survivable. The refund stops being a thing the model does and becomes a thing a person does on the model’s advice. | included below |
| 3 | Retry the reply, and put anything that still fails where a person will see it | Closes 2.1 and 2.2. A failure has to be visible or it is not a failure, it is a silence. | included below |
| 4 | Send the model only the fields the answer needs; stop sending address, phone and card digits | Closes 4.1. Cheapest item on the list and the one with the largest consequence if it is ever asked about. | included below |
| 5 | Record every message once, and refuse the second copy | Closes 5.2. | included below |
| All five, delivered as one working system with a runbook | This is our first-build engagement. | $5,400 | |
| 6 | Fix the order lookup so a date cannot be read as an order number | Closes 1.2. Small, and it is inside the same file as items 1 to 5, so it comes with them. | included above |
| 7 | Keep it running afterwards and change it as the shop changes | Optional and not required for anything above to work. | from $549 / month |
If the shop chooses to do none of this, the single change we would still urge, because it can be made in one line and prevents the worst outcome on this list, is item 1.
The checks are a script that puts messages through the system with its outbound calls recorded, and reads the system’s own source for the presence or absence of specific mechanisms. It is run from a single command and produces the figures quoted above.
The first run reported that the system had a duplicate-message guard. It has none. The check searched the whole file for a small set of words, and matched the word “seen” inside a comment. A search that reads comments is measuring the author’s English rather than the running system. Comments are now removed before any search, and the number of lines removed is printed on each run so that the removal can be seen to have happened. Finding 5.2 is the corrected result.
It is recorded here because a review that never reports being wrong is not describing how the work is done.
Nobody. It is a specimen, produced against a sandbox we built ourselves, and no engagement stands behind it. A review issued for a real engagement names its addressee here and restricts reliance to that addressee and to anyone they nominate in writing.
We wrote the system examined in this document. For a specimen that is unavoidable — there was no client system to look at — and it is stated here rather than left to be discovered.
In an engagement we do not review a system we built. Where a client asks us to look at our own work, that is a different piece of work with a different name, and it is charged and described differently.
No fee depends on what a review finds.
The checks described above were run by me and the findings stated are the ones they produced.
[Name withheld]
Engagement lead, Noorflows LLC · 2026-08-11
Our terms state that every report carries the name of the person who signed it. It is withheld here because this is a specimen with no addressee and no engagement behind it, and for that reason this copy is not a signed deliverable. A review issued for a real engagement is signed and named.