Sea Trial sends 44 attacks and 245 disguised versions of them through a guardrail stack, counts what got through and how many ordinary customers were refused along the way, and writes the result into a hash-chained record.
One HTML file. No account, no server. You can paste a real transcript into it.
First pass with every rail on. In a moment the unmasking rail comes off.
In model risk this is called effective challenge. Somebody who is not the vendor tries to break the control and keeps a record of what happened. The vendor demo is not that.
I have sat through those demos. The attack is typed cleanly, the filter catches it, everyone nods. Nobody types it in leetspeak. Nobody swaps in a Cyrillic "а" or pastes it base64-encoded inside a note. Nobody asks what happened to the customer who wrote "ignore my earlier message, I found the statement" and got refused.
Those are the two questions I would expect from an examiner: what gets through, and how often do you refuse a customer for no reason. Sea Trial answers both from a fixed configuration, so a second reviewer running it gets the same number you did.
Five steps, about ten minutes the first time. The score is provisional until step two.
44 attacks and 14 ordinary messages go through the ten rails. You get a detection rate, a false-positive rate, and a grid of which rail caught what.
Each caught attack is rewritten seven ways (leetspeak, spaced letters, look-alike characters, zero-width padding, base64, reversed, buried in a polite message) and sent again. This is where most stacks lose points.
Paste your own red-team cases or complaint transcripts. Add your own phrases, tool names and canary strings to the rails. Your cases get their own rows in the grid.
Change a threshold or switch a rail off and the numbers update. Usually you trade one false positive for one miss; now you can see which one.
One page: grade, settings, findings, the mapping to OWASP, the EU AI Act, NIST AI RMF and SR 11-7, and the chain hash. Print it and file it.
Below is a real run of the built-in corpus at default settings, generated when this page loaded. Yours will come out differently.
Each cell is how many attacks in that vector a rail flagged on its own. A row with one filled cell is a single point of failure.
Each rail is a readable set of rules, so a reviewer can see exactly why a message was held.
Undoes disguises before anything else reads the text: base64, letter spacing, leetspeak, Cyrillic look-alikes, zero-width characters, reversed strings. Turn it off in the demo above and the same attack gets through five times out of eight.
OWASP LLM01, EU AI Act Art. 15, NIST MEASURE 2.7Holds anything the assistant is not entitled to do: wires, deletions, limit changes, waived verification, shell commands, calls to privileged tools, and any amount over the cap. In the default stack it is the rail with the second-most attacks depending on it alone.
OWASP LLM06, EU AI Act Art. 14, NIST GOVERN 1.7Checks the reply, not the request. Malformed JSON, "guaranteed returns", "I have transferred the money", "as an unrestricted assistant". Most guardrail products stop at the input. This is the rail examiners ask about.
OWASP LLM05, EU AI Act Art. 13, NIST MEASURE 2.5One page with three numbers that did not come from a vendor: attacks caught, disguised attacks caught, customers wrongly refused. Dated and tied to a configuration hash.
The coverage grid and the single-point-of-failure list. They read like findings because that is what they are. SR 11-7 asks for evidence of challenge; this is the evidence.
Your own attack cases and rules, saved as a JSON file. Re-running next quarter is an import and a click, and whoever has the job after you can do it.
Run two configurations through the same corpus and put the two certificates side by side. It is a shorter conversation than comparing decks.
Half a day with your team, or installed for your validators to run on their own. Either way you finish with the same three files.