How to use Sea Trial

A bank's AI assistant sits behind ten guardrails. Sea Trial attacks those guardrails on purpose, measures what got through and what it cost ordinary customers, and seals the result into a record that can't be edited afterwards. Nothing here reaches a live model.

Five steps

1. Run sea trialSends 44 known attacks and 14 ordinary member messages through the stack. You get the detection rate, the false-positive rate, a coverage matrix showing which rail catches which attack type, and the single points of failure.
2. Run mutation stormRewrites every caught attack seven ways an attacker would (leetspeak, spaced letters, look-alike characters, invisible characters, base64, reversed, hidden inside a polite message) and re-sends them. A red dot is a disguised attack that walked through. This turns the score from provisional to final.
3. Build your own trialOpen "Build your own trial" at the top of the deck. Paste messages from your red-team notes, complaint logs or chat transcripts, or pick from the examples, and mark each as an attack type or ordinary traffic. Teach a rail your own phrases, tool names, canaries or forbidden claims. Your cases get their own rows in the matrix; untick the built-in corpus to test only your own material.
4. Tune and re-runMove a threshold or switch a rail off in the left column; the deck recomputes at once. Try switching the Normaliser off and running the storm again to see resilience collapse. Every configuration you try is sealed into the chain.
5. Issue certificateA one-page Certificate of Sea Trial: grade, configuration, findings as conditions, regulatory crosswalk and the chain head hash. Print or save as PDF for the validation file. Export evidence downloads the full chain as studio.evidence.v1.

The screen

The stack (left)

The ten guardrails under test, with switches and the five tunable settings. The link at the bottom maps each rail to OWASP, EU AI Act, NIST AI RMF and SR 11-7.

Trial deck (middle)

Results: headline numbers, coverage matrix, misses, false positives, single points of failure, mutation storm.

Interceptor (right)

Test one message at a time. Type anything or load an example, and see each rail's ruling, the exact words it flagged and the time it took.

Evidence chain (bottom)

Every action hashed and linked to the one before it with SHA-256.

The score

Seaworthiness out of 100: 60% detection, 25% resilience to disguise, 15% not holding real customers. A is 95 or better, B 85, C 70.