Rate Limiting Architecture AGENTS.md Template

Overview

Direct answer: this AGENTS.md Template governs a rate limiting architecture using multi-agent orchestration to enforce per-tenant quotas, distribute counters, and coordinate handoffs between planners, implementers, researchers, and reviewers. It supports both single-agent and multi-agent workflows and defines the operating context for AI coding agents involved in tool governance and human review.

In practice, this AGENTS.md template anchors policy design, enforcement, validation, and escalation for rate limiting at an API gateway or gateway-enabled service mesh, with a clear separation of concerns among agents and a single source of truth for policy and telemetry.

When to Use This AGENTS.md Template

• You need a repeatable, auditable rate limiting workflow across services.
• You require multi-agent collaboration for policy design, implementation, testing, and deployment.
• You must govern access to shared rate limit state (counters, quotas, and policies) with strict tool governance.
• You want explicit handoff rules and escalation paths for policy changes or failures.

Copyable AGENTS.md Template

# AGENTS.md
# Rate Limiting Architecture - AGENTS.md Template

Project: Rate Limiting Architecture for API Gateway

Agent roster and responsibilities
- Planner: designs policy, token bucket strategy, per-tenant quotas, and orchestrates dependency graph
- Implementer: builds limiter middleware, integrates with gateway configs, enforces token consumption
- PolicyResearcher: analyzes bucket types, concurrency models, and policy implications
- StateManager: stores and reads quotas/counters in Redis or similar store; ensures consistency and eviction rules
- Auditor: checks security, privacy, and policy compliance
- Reviewer: validates changes, approves PRs, and ensures test coverage
- Tester: runs unit, integration, and end‑to‑end tests for limiter behavior
- Operator: handles deployment, feature flags, and canary rollout

Supervisor or orchestrator behavior
- Central orchestrator coordinates plans, tracks handoffs, and triggers escalations when policy or safety constraints are violated
- All changes require traceable rationale and test evidence before promotion

Handoff rules between agents
- Planner -> Implementer: policy doc, acceptance criteria, and implementation plan
- Implementer -> Reviewer: code changes, tests, and config changes
- Reviewer -> Tester: validated patch and test results
- Tester -> Orchestrator: test results and risk assessment
- Orchestrator -> Planner: decision and next iteration guidance

Context, memory, and source-of-truth rules
- Context: current rate limit policy, per-tenant quotas, and gateway configuration
- Memory: policy service, Redis counters, gateway caches, and audit logs
- Source of truth: policy repository, config store, and observability dashboards

Tool access and permission rules
- Access to Redis, gateway config, policy service; secrets stored in vault with tight RBAC
- All tool actions are logged and replayable; no inline secret exposure

Architecture rules
- Distributed rate limiter with per-tenant quotas and sharded counters
- Prefer token bucket with leaky bucket fallback for burst control
- Idempotent operations and deterministic decisions

File structure rules
- rate-limiter-architecture/
  configs/
  policies/
  src/
    planner/
    implementer/
    researcher/
    state/
    orchestrator/
    auditor/
    tester/
  tests/
  docs/

Data, API, or integration rules
- Public APIs for quota queries must be read-only unless explicitly allowed
- All changes must pass data validation and integrity checks

Validation rules
- Unit tests for limiter math and edge cases
- Integration tests with gateway and caching layer
- End-to-end tests simulating burst traffic

Security rules
- Secrets in vault, rotation policies, least privilege
- Encrypted transport and auditable decision logs

Testing rules
- 80%+ unit test coverage; 90% end-to-end coverage where feasible
- CI checks for code quality and security linting

Deployment rules
- Canaries and feature flags for policy changes
- Rollback plan and automated rollback if errors exceed threshold

Human review and escalation rules
- Escalate to Platform Architect for policy ambiguities or security concerns
- All escalations require documented rationale and risk assessment

Failure handling and rollback rules
- If limiter fails open, degrade gracefully to allow traffic with reduced guarantees
- Rollback policy changes to last known good state on failure

Things Agents must not do
- Do not bypass the rate limiter under any circumstance
- Do not mutate policy without a plan and approvals
- Do not operate on production systems outside approved channels

Recommended Agent Operating Model

The operating model assigns clear responsibilities and decision boundaries for each role. The Planner sets policy constraints and acceptance criteria; the Implementer translates policy into code; the Researcher informs policy choices with empirical findings; the StateManager maintains counters and quota data; the Auditor ensures security and policy compliance; the Reviewer validates changes; the Tester verifies behavior; and the Operator handles deployment. Handoffs follow explicit signals, and the Orchestrator maintains a canonical plan and coordinates parallel work streams. Escalation paths to human review exist for ambiguous or high-risk decisions.

Recommended Project Structure

rate-limiter-architecture/
configs/
policies/
src/
  planner/
  implementer/
  researcher/
  state/
  orchestrator/
  auditor/
  tester/
tests/
docs/

Core Operating Principles

• Single source of truth for policy and telemetry
• Idempotent, deterministic decisions
• Explicit, replayable handoffs with clear signals
• Observability, traceability, and auditable actions
• Least privilege, secure secrets, and strict access control

Agent Handoff and Collaboration Rules

• Planner communicates policy intent and acceptance criteria to Implementer with a plan doc
• Implementer shares implementation details, tests, and config changes with Reviewer
• Researcher surfaces findings that may alter policy or architecture
• Auditor validates security and policy compliance before deployment
• Ongoing collaboration requires ongoing observability feedback to the Orchestrator
• Domain specialists provide context for compliance with regulatory constraints when applicable

Tool Governance and Permission Rules

• Commands to modify policy or config must pass governance reviews
• All changes to production routes require approval gates and canary verification
• Secrets must never be hard-coded; use vault or secret manager
• All API calls and edits are logged with trace IDs

Code Construction Rules

• Write modular, testable limiter components with clear interfaces
• Avoid hard-coded values; use configuration-driven inputs
• Provide unit and integration tests for all modules
• Ensure idempotence and deterministic behavior under retry
• Document decisions and rationale in code comments and PRs

Security and Production Rules

• Encrypt data in transit and at rest where applicable
• Rotate credentials and enforce short-lived tokens
• Audit and monitor rate limiter decisions and anomalies
• Canary deployments and automatic rollback on detected issues

Testing Checklist

• Unit tests for limiter math and edge cases
• Integration tests with policy service and store
• Performance tests simulating bursts and multi-tenant load
• End-to-end tests across API gateway flows
• Canary validation in staging before production

Common Mistakes to Avoid

• Skipping explicit handoffs and relying on implicit intent
• Overloading a single agent with cross-domain responsibilities
• Bypassing the policy source of truth or secret management
• Ignoring observability during rollout and rollback planning

Related implementation resources: AI Use Case for Policy Documents and Internal Question Answering and AI Use Case for Leasing Agents Using Zendesk To Answer Tenant Faq Queries Instantly Via Ai Chatbot.

FAQ

What is this AGENTS.md Template for rate limiting architecture?

It defines a repeatable, governance-driven workflow for planning, implementing, and validating a distributed rate limiter using multi-agent orchestration.

Who are the core agents and their responsibilities?

Planner designs policy; Implementer builds limiter; Researcher informs strategy; StateManager tracks counters; Auditor checks security; Reviewer validates changes; Tester verifies; Operator deploys.

How are agent handoffs enforced?

Handoffs are driven by explicit signals in the AGENTS.md template: plan to implement, implement to review, review to test, test to orchestrator, and escalation to the orchestrator as needed.

What are the security constraints?

Secrets must be stored in a vault, access scoped by role, and rate-limit decisions must be auditable with immutable logs.

How is failure handled and rolled back?

On failure, the system degrades gracefully, rolls back to the last known good policy, and notifies the orchestrator for manual review if needed.