Laya Agent Guard

🛡️ Laya Agent Guard

A guardrail for AI agents. Paste anything an agent might read — a user message, an email, a web page, a tool result — and it decides in one forward pass whether it is benign, a prompt injection, a jailbreak or a harmful request.

🤗 Model 💻 Code 📚 Dataset 421M params · ModernBERT-large + decision head

Text to check

Try an example:0 / 4000

Block if P(attack) ≥ 0.50

Result

Run a check to see the verdict, class probabilities and timing.

Your checks this session

Text	Verdict	P(attack)	Model	Total
No checks yet.

Benchmark results

Model	F1	AUROC	False blocks	T4 latency
Laya Agent Guard (this)	0.88	0.97	7%	34 ms
Jev (hosted API)	0.78	0.92	17%	n/a
Llama Guard 3 1B	0.65	0.81	19%	98 ms
Laya base (no fine-tune)	0.58	0.69	34%	44 ms

18,617 test prompts across 11 benchmarks, threshold 0.5. Weak spots: it trails Jev on XSTest, JailbreakBench and deepset. Full results →

How it works

1 · One sheetThe question, the four labels and your text go into one input. Each label gets a [MASK] "answer box".

2 · ReadModernBERT-large reads everything bidirectionally; each box absorbs how well its label fits.

3 · CompareTwo extra Transformer layers let the boxes compete; one shared scorer gives each a number.

4 · CalibrateSoftmax with a fitted temperature turns scores into honest probabilities.

Fine-tuned on 78k examples from 13 public datasets plus synthetic indirect injections, on free Kaggle GPUs.