Message Safety Check looks at a user message, a model reply, or a tool call and tells you whether it is safe to continue.
It checks for jailbreaks, attempts to trick the model, harm, and policy risk. You get a simple action — pass, review, block, or send to support — plus a score for each kind of risk. Your code decides what to do with that action.
Put this in front of a chat model or a dangerous tool. It is a cheap check, not a full conversation.
What you can do
- screen incoming prompts for jailbreaks, harm, medical advice, self-harm, and secret-seeking
- screen model replies before a user sees them
- screen tool calls that might delete data, steal secrets, or take over the session
Who it's for
Chatbots, agent workflows, and anyone who should not spend a large model call just to decide if this turn is safe.
How to use it
- Call screen with the text and
side: input, output, or tool_call - Follow the returned action in your own code
- Start with policy=strict until you have your own labelled examples
Getting started
Screen a jailbreak-style prompt with side=input. If the action is block, do not send it to the model.
