THE FIELD GUIDE / AI & THROUGHPUT
AI throttling, daily caps & unthrottled mode
Control pacing, simultaneous work and estimated spending as three separate settings.
Choose an admission mode
Administrators choose rate throttling, a daily USD spend cap, or unthrottled admission in AI settings. Rate pacing and a daily cap can be combined. Configured concurrency is a separate resource bound; it is not a spending allowance.
| Mode | What happens |
|---|---|
| Rate throttle | Smoothly pace eligible work; keep excess work queued until refill. |
| Daily spend cap | Admit paid calls only when a conservative reservation fits the day’s budget. |
| Unthrottled | No Mr Patchy rate or spend allowance; concurrency, resources and provider limits still apply. |
Apply one policy to paid calls
Shared admission covers investigations and follow-ups, retries, scoring batches, manual/automatic scanner and startup plans, release-configuration reviews and billable saved-settings model tests. Manual requests and background workers use the same policy. Budget/rate deferral remains queued work, not a failed attempt or discarded evidence.
See why a request is waiting
A durable fleet-wide provider floor honors Retry-After and classified transient failures before rate or spend admission. Scoring, incident/startup plans and release planning retain timed eligibility or a pause/resume wait; the scheduler rechecks current policy and evidence before resuming. Inspect Activity, Packages and AI worker status rather than repeatedly submitting the same request. Read the AI wait and recovery guide.
Reserve before sending
Atomically reserve conservatively for the selected model/tier, bounded input and output/reasoning usage, supported charges and any required follow-up stage. Outstanding reservations across workers and purposes count against headroom. Reconcile returned usage with a retained price snapshot; partial, failed and refused calls may still cost money. Unknown prices, missing usage and lost responses retain uncertainty and can block capped admission when no safe bound is available.
Define the day
A daily cap has a positive USD amount, explicit IANA timezone and next calendar-day reset, including daylight-saving behavior. Audited, revision-checked settings changes, controller restarts and mode changes do not silently grant a second budget. Cross-midnight calls retain consistent attribution, and uncertain possibly billed reservations are not discarded merely because a day or lease ended.
Read queue and spending state
Settings, Investigations, AI activity and worker health show selected mode, effective concurrency/rate, active/queued work, oldest wait, blocker, next eligibility/reset, estimated usage, reserved and uncertain charges and remaining headroom. The durable budget ledger survives ordinary AI-log pruning. Administrators review trailing 1–90 day history by retained reset window and timezone. Capped admission checks pricing freshness and blocks when the reviewed catalog expires.
Review a policy change
Migration from hourly allowances presents an explicit pacing policy and conservative defaults for administrator review. Saving a change shows its effect first and records an audit entry; zero is not silently reinterpreted as unlimited.
Open the application reference for capability scope and technical details.
Open Open SECURITY_IMPLEMENTATION.md on GitHub ↗