AGENTS.md Template: Request Throttling Strategy

Overview

This AGENTS.md template defines a request throttling strategy managed by AI coding agents. It governs a multi-agent orchestration pattern that coordinates planning, policy generation, enforcement, telemetry, and validation. It supports both single-agent operation and distributed, multi-agent handoffs with a clear source of truth and auditable decisions.

Direct answer: Use this template to formalize throttling policies, enforce them at the gateway, and continuously validate against load and quota constraints in a controlled, auditable manner.

When to Use This AGENTS.md Template

• When building or updating a distributed rate-limiting policy for microservices.
• When integrating third-party API quotas with internal service quotas.
• When you need repeatable agent handoffs and explicit escalation paths.
• When governance, security, and auditing of throttling rules are required.
• When you want a copyable, project-scoped operating context for AI coding agents.

Copyable AGENTS.md Template

# AGENTS.md

Project role
- Throttling Orchestrator and Policy Agent responsible for defining, propagating, and supervising the throttling policy across services.

Agent roster and responsibilities
- PlannerAgent: designs throttle policy (per API/service), time-slice windows, and concurrency limits.
- ImplementerAgent: translates policy into gateway/config updates and enforces limits at the entry points.
- TelemetryAgent: collects rate, error, and quota metrics; surfaces drift and policy violations.
- EvaluatorAgent: validates policy against load patterns; triggers policy re-optimization when needed.
- ReviewerAgent: security/compliance checks and approvals for policy changes.
- OrchestratorAgent: coordinates handoffs, stores policy state, and ensures memory consistency across agents.

Supervisor or orchestrator behavior
- The Orchestrator maintains a single source of truth for policy, enforces idempotent changes, and triggers re-plans when drift is detected. It coordinates handoffs between Planner, Implementer, Telemetry, Evaluator, and Reviewer.

Handoff rules between agents
- PlannerAgent -> ImplementerAgent: deliver policy in a versioned artifact and a change request.
- ImplementerAgent -> TelemetryAgent: enable monitoring and metrics collection for the new policy.
- TelemetryAgent -> EvaluatorAgent: provide observed data and drift signals for validation.
- EvaluatorAgent -> PlannerAgent: return validation results and recommended adjustments.
- If security or compliance concerns arise, escalate to ReviewerAgent before deployment.

Context, memory, and source-of-truth rules
- Maintain policy in config/policies/throttle_policy.json and document in docs/policies.md.
- Cache current active policy in memory with a 60s refresh window and persistence in a durable store.
- All decisions must reference the current policy version as the source of truth.

Tool access and permission rules
- Grant read access to policy docs and metrics endpoints; restrict writes to Planner and Implementer via controlled CI.
- Secrets used for external throttling services must be retrieved from a secrets vault; never commit keys in code.

Architecture rules
- Use a centralized throttle service with token-bucket or leaky-bucket algorithm; support per-API quotas.
- Ensure idempotent policy application and deterministic outcomes for same input.
- Implement circuit-breaking and safe-fallbacks if throttle service is unavailable.

File structure rules
- Only include folders that relate to throttling workflow: config/policies, gateways, infra, services, docs, tests, and agents.

Data, API, or integration rules when relevant
- All quotas pulled from policy API and local cache; external API calls must be wrapped with retry/backoff logic bounded by policy.
- Validate quotas against service SLAs and provide fallback paths.

Validation rules
- Use synthetic load tests to verify response times, error rates, and quota adherence.
- Validate policy drift and regression with automated checks.

Security rules
- Never expose secret keys in logs or policy artifacts.
- Enforce least-privilege for all agent credentials and use vault-backed retrieval.

Testing rules
- Unit tests for policy computation; integration tests for gateway enforcement; end-to-end tests for throttling under peak load.
- Include negative tests for policy rollback and unsafe state transitions.

Deployment rules
- Deploy throttling policy changes via canary or feature-tlag; require approval for permanent changes.
- Rollback plan with automated revert to the previous version if metrics breach thresholds.

Human review and escalation rules
- Security and compliance reviews are mandatory for any policy changes.
- Escalate to SRE on production incidents requiring throttle policy hotfixes.

Failure handling and rollback rules
- If throttle service fails, fall back to a safe default (e.g., 60 requests per second per API) with telemetry of degraded state.
- Roll back to prior policy version on verification failure or policy drift beyond thresholds.

Things Agents must not do
- Do not bypass policy checks; do not apply policies without versioned changes; do not mutate the source of truth without orchestration.
- Do not expose internal quotas to external clients; do not operate outside agreed SLAs.

Recommended Agent Operating Model

The agent operating model defines roles, decision boundaries, and escalation paths for throttling governance. Planner decides policy envelopes; Implementer enforces them; Telemetry validates outcomes; Evaluator confirms performance; Reviewer approves changes; Orchestrator coordinates all handoffs and maintains policy memory across the lifecycle. Decisions must reference the authoritative policy version, and escalations must follow the security/compliance workflow.

Recommended Project Structure

throttling-workflow/
  config/
    policies/
      throttle_policy.json
  gateways/
    api-gateway-config.yaml
  infra/
    throttle-service/
  services/
  docs/
    policies.md
  tests/
    unit/
    integration/
  agents/
    planner/
    implementer/
    telemetry/
    evaluator/
    reviewer/
    orchestrator/

Core Operating Principles

• Single source of truth for policy versioning.
• Deterministic decisions and auditable reasoning.
• Strict separation of concerns among planner, implementer, telemetry, and evaluator.
• Safe defaults and explicit rollback paths.
• Least-privilege access to tools and secrets.

Agent Handoff and Collaboration Rules

• Planner → Implementer: deliver versioned policy artifact and a change request.
• Implementer → Telemetry: enable monitoring for the policy change.
• Telemetry → Evaluator: provide observed load and drift signals.
• Evaluator → Planner: return validation results and recommendations.
• If issues arise, escalate to Reviewer before deployment.

Tool Governance and Permission Rules

• Enforce least-privilege for credentials; secrets stored in vaults; no plaintext keys in code or docs.
• Writes to policy must go through CI/CD gates with review) and approval workflow.
• API calls to throttle services must be traced, rate-limited, and auditable.
• Any production change requires supervisor approval and feature flag management.

Code Construction Rules

• Idempotent changes and deterministic policy application.
• Versioned policy artifacts; no in-place, untracked edits.
• Clear unit tests for policy logic; integration tests for gateway enforcement.
• Avoid hard-coding quotas; use config-driven values sourced from policy store.

Security and Production Rules

• Secrets must be retrieved from a vault; no secret leakage in logs or artifacts.
• Throttle service must default to safe, conservative behavior in failure; circuit breakers in place.
• All changes must pass security and compliance reviews before deployment.

Testing Checklist

• Unit tests for policy computation and edge cases (zero quota, negative values).
• Integration tests for gateway enforcement and policy propagation.
• Load/stress tests to confirm policy under peak traffic.
• Canary/deployment tests with rollback capability.

Common Mistakes to Avoid

• Bypassing the policy or applying unversioned changes.
• Overly aggressive defaults causing customer-visible failures.
• Missing audit trails for policy decisions.
• Inconsistent memory/state across agents.

Related implementation resources: AI Use Case for Policy Documents and Internal Question Answering and AI Use Case for Corporate Event Managers Using Slack To Orchestrate Day-Of Venue Tasks Across Multi-Department Teams.

FAQ

What is AGENTS.md Template used for in a throttling workflow?

It provides a copyable operating manual that defines roles, handoffs, and governance for a request throttling strategy in AI coding agents.

How does multi-agent orchestration apply to throttling?

Multiple agents collaborate to plan, implement, monitor, and validate throttling policies, with explicit handoffs and shared memory of policy versions.

What if the throttle service fails?

Fallback to a safe default policy and escalate to human review while preserving telemetry for analysis.

How are changes approved and deployed?

Changes go through a planner→implementer→telemetry→evaluator flow, with reviewer approval and CI/CD gated deployment.

What must be avoided in this workflow?

Avoid undocumented policy changes, secret leakage, and policy drift without an auditable path back to the source of truth.

How is success measured?

Success is measured by adherence to quotas, low latency, low error rates, and minimal drift between observed and policy-reported behavior.