Self-hosted docs
Guardrails
Guardrails define what safe, compliant, and reliable execution looks like in production.
Runtime guardrails
Catch unsafe output, runaway latency, and cost spikes at the point of execution before incidents spread.
- unsafe output detection
- latency and error thresholds
- budget and token variance checks
- Example: unsafe_output_detection with confidence_threshold: 0.85
- Example: latency_spike_threshold: {critical: 2000ms, warning: 1500ms}
- Example: cost_variance_alert: {percentage: 15%, min_absolute: $0.50}
- Request: POST /v1/policies/runtime/evaluate {route_id: "prod-chat", model: "claude-3-opus", latency_ms: 1800}
- Response: {verdict: "warn", reason: "latency_spike_threshold_exceeded", confidence: 0.92}
- API Reference: See [Runtime Policy Evaluation](/docs/api#runtime-policy-evaluation) for full contract details
Change-event guardrails
Evaluate provider-side changes against active production routes to flag reliability and policy risk quickly.
- Example: pricing_change_guardrail for >10% increase on active route models
- Example: status_degradation_guardrail for provider outages affecting production
- Example: release_notes_guardrail for breaking changes in dependent services
- Request: POST /v1/policies/change/evaluate {provider: "anthropic", change_type: "pricing", affected_models: ["claude-3-opus"]}
- Response: {verdict: "block", reason: "pricing_increase_exceeds_threshold", action: "trigger_review_workflow"}
- API Reference: See [Change Policy Evaluation](/docs/api#change-policy-evaluation) for full contract details
Containment policies
Automated responses to SLA breaches, policy violations, and failure conditions.
- Automatic failover when latency exceeds defined thresholds
- Traffic throttling during performance degradation
- Circuit breaking for problematic routes
- Alert escalation based on incident severity
- Example: failover_policy with latency_threshold: 2000ms and fallback_endpoint: "backup-chat"
- Example: throttling_policy with max_rps: 1000 and burst_limit: 50
- Request: POST /v1/policies/containment/activate {route_id: "prod-chat", policy: "failover", reason: "latency_degradation"}
- Response: {status: "activated", original_endpoint: "primary-chat", fallback_endpoint: "backup-chat", timestamp: "2024-03-15T14:30:00Z"}
Change-event guardrails
Evaluate provider-side changes against active production routes to flag reliability and policy risk quickly.
- Example: pricing_change_guardrail for >10% increase on active route models
- Example: status_degradation_guardrail for provider outages affecting production
- Example: release_notes_guardrail for breaking changes in dependent services
- Request: POST /v1/policies/change/evaluate {provider: "anthropic", change_type: "pricing", affected_models: ["claude-3-opus"]}
- Response: {verdict: "block", reason: "pricing_increase_exceeds_threshold", action: "trigger_review_workflow"}
- API Reference: See [Change Policy Evaluation](/docs/api#change-policy-evaluation) for full contract details