Real-time Chat Architecture AGENTS.md Template

Overview

Direct answer: This AGENTS.md Template defines the operating model for real-time chat architecture using AI coding agents, enabling both single-agent execution and multi-agent orchestration with explicit handoffs, tool governance, and safety controls.

The template codifies how to structure agent roles, responsibilities, and interactions to handle live chat scenarios where latency, accuracy, and safety are critical. It provides a project-level operating context that supports scalable coordination, auditability, and human review where needed.

When to Use This AGENTS.md Template

• When building a real-time chat system that relies on AI coding agents to ingest, reason, respond, and act on user messages.
• When you need explicit multi-agent orchestration, including planner, implementer, reviewer, tester, and domain specialists.
• When enforcing tool governance, secret handling, and production safeguards in a chat workflow.
• When you require a clear source of truth, memory model, and audit trail for live conversations.

Copyable AGENTS.md Template

# AGENTS.md

# Real-time Chat Architecture AGENTS.md Template

Project Role: Platform Engineering Lead; Product Manager; AI-Centric Chat Engine Team

Agent roster and responsibilities:
- Orchestrator (ChatOrchestrator): coordinates message flow, timing, and cross-agent handoffs.
- IngestAgent: normalizes and tokenizes incoming messages; persists raw input to a session store.
- ModerationAgent: applies content safety and policy checks; flags for escalation.
- RoutingAgent: determines routing to responders or specialized tools.
- RespondAgent: generates natural language responses using LLMs or retrieval augmented generation.
- MemoryManager: maintains session memory, context, and reference sources.
- ToolAccessAgent: interfaces with external services (APIs, databases) under policy constraints.
- DomainSpecialistAgent: handles domain-specific logic (finance, healthcare, etc.).
- ReviewerAgent: performs manual or automated review for high-risk interactions.
- TesterAgent: validates outputs before delivery to the user.

Supervisor / Orchestrator behavior:
- Maintain global state per chat session and enforce completion of each handoff before proceeding.
- Enforce tool governance rules, permission checks, and secret handling on every action.
- Trigger human review when policy or safety flags are raised.
- Provide escalation path to on-call engineers for production incidents.

Handoff rules between agents:
- IngestAgent → ModerationAgent: after ingest, pass content for safety checks.
- ModerationAgent → RoutingAgent: route based on policy outcome.
- RoutingAgent → RespondAgent or ToolAccessAgent: depending on whether a tool is required.
- RespondAgent → MemoryManager: update memory with user-visible content.
- DomainSpecialistAgent → ReviewerAgent: for high-risk or specialized domains.

Context, memory, and source-of-truth rules:
- Use a centralized session store as source of truth for user messages and agent decisions.
- MemoryManager maintains per-session context with a retention policy (e.g., 24h);
- All agents reference the memory snapshot before acting.
- All external data references are tagged with provenance metadata.

Tool access and permission rules:
- ToolAccessAgent can invoke only whitelisted APIs with scoped credentials.
- Secrets must be retrieved via a vault; never hard-coded.
- All tool calls require an approval gate if the action modifies user data or state.

Architecture rules:
- Event-driven microservices with a per-session state machine.
- Stateless agents where possible; state is stored in the MemoryManager.
- Idempotent operations and retry policies for reliability.

File structure rules:
- agents/
  - orchestrator/
  - ingestor/
  - moderation/
  - routing/
  - responder/
  - memory/
  - tools/
  - domain/
  - tester/
  - reviewer/
- configs/
- workflows/
- tests/
- docs/

Data, API, or integration rules:
- Use standard message formats; attach metadata: {sessionId, timestamp, provenance}.
- Respect user consent for data usage in training or analytics.

Validation rules:
- Each agent must emit a validated action object with explicit intent and outcome.
- All memory writes must be confirmed by a validation step.
- Automated tests must cover at least two end-to-end scenarios per release.

Security rules:
- Secrets stored in vaults; never in code.
- Enforce least privilege on tool access.
- Audit logs retained for at least 90 days.

Testing rules:
- Unit tests for each agent wrapper; integration tests for end-to-end flows.
- Mock external services; include error injection tests.
- End-to-end tests must simulate user interactions with latency budgets.

Deployment rules:
- CI/CD gated merges; blue/green deployments for live chat components.
- Feature flags for riskier capabilities.

Human review and escalation rules:
- All high-risk outputs require human review before user delivery.
- Escalate critical incidents to on-call with incident tickets.

Failure handling and rollback rules:
- If a tool call fails, retry once; on second failure, escalate.
- Rollback to previous safe memory state if a chain of actions fails.

Things Agents must not do:
- Do not bypass safety checks or skip policy evaluation.
- Do not perform production changes without approvals.
- Do not persist raw user data beyond policy retention.

Recommended Agent Operating Model

Roles, responsibilities, decision boundaries, and escalation paths define a predictable, auditable flow. The Orchestrator enforces cross-agent handoffs, policy checks, and memory updates; Domain Specialists provide deep, constraint-aware reasoning; Reviewers and Testers gate content quality and safety before user delivery. Escalations route to humans when risk exceeds automatic thresholds.

Recommended Project Structure

real_time_chat/
├── agents/
│   ├── orchestrator/
│   │   └── orchestrator.py
│   ├── ingestor/
│   │   └── ingest.py
│   ├── moderation/
│   │   └── moderation.py
│   ├── routing/
│   │   └── router.py
│   ├── responder/
│   │   └── respond.py
│   ├── memory/
│   │   └── memory_store.py
│   ├── tools/
│   │   └── api_client.py
│   ├── domain/
│   │   └── domain_specialist.py
│   ├── tester/
│   │   └── tests.py
│   └── reviewer/
│       └── reviewer.py
├── workflows/
│   └── real_time_chat_workflow.md
├── configs/
│   ├── airflow/
│   └── k8s/
├── tests/
│   ├── unit/
│   ├── integration/
│   └── e2e/
└── docs/
    └── runbooks.md

Core Operating Principles

• Single source of truth per chat session with strict memory boundaries.
• Explicit, auditable agent handoffs with contract-style interfaces.
• Least-privilege tool access and credential management.
• Continuous validation and human-in-the-loop when risk is detected.
• Idempotent actions, deterministic outputs, and retry strategies.

Agent Handoff and Collaboration Rules

Rules for planner, implementer, reviewer, tester, researcher, and domain specialist agents:

• Planner -> Implementer: provide a detailed plan and data requirements; include success criteria and rollback steps.
• Implementer -> Reviewer: submit outputs with provenance, memory updates, and test results.
• Reviewer -> Planner: approve, modify, or request additional data; log decision rationale.
• Researcher -> Domain Specialist: supply domain-context and fetch domain-safe references; domain specialist confirms domain-appropriate behavior.
• Orchestrator ensures all handoffs meet policy checks before progression; any failure triggers escalation.

Tool Governance and Permission Rules

• Only whitelisted APIs may be called; secrets accessed via vault; credentials rotated on a schedule.
• Production tool calls require approval gates and traceable approvals.
• All data in transit must be encrypted; logs preserve privacy and compliance.
• Editing core configuration or memory schemas requires a peer review and change window.
• Automated monitoring must detect anomalous tool usage and halt flows immediately.

Code Construction Rules

• Use typed interfaces for agent messages; validate schemas on ingestion and between handoffs.
• Avoid hard-coded secrets; rely on environment variables and secret stores.
• Keep state local to the memory store; avoid global mutable state across sessions.
• All external API calls are wrapped with retries, timeouts, and circuit breakers.
• Do not bypass safety or policy checks; always return explicit policy decisions.

Security and Production Rules

• Encrypt personal data; adhere to retention limits and data minimization.
• Implement role-based access control for agents and operators.
• Maintain audit logs for all agent actions with immutable storage where possible.
• Isolate chat sessions; prevent cross-talk between user sessions.
• Prepare runbooks for incident response and rollback procedures.

Testing Checklist

• Unit tests for each agent wrapper and function.
• Integration tests covering end-to-end chat flows with latency budgets.
• End-to-end tests simulating tool failures, timeouts, and policy triggers.
• Security tests including secret handling and access controls.
• Deployment tests like canary and blue/green deployments for risk mitigation.

Common Mistakes to Avoid

• Skipping policy checks or human review for high-risk content.
• Overloading agents with unbounded memory or state drift across sessions.
• Hard-coding secrets or secrets leakage through logs.
• Unclear handoff contracts leading to race conditions and missing provenance.

Related implementation resources: AI Use Case for Content Marketers Using Wordpress To Auto-Translate Blog Posts Into Multiple Languages and AI Use Case for Sales Pipeline Reviews and Deal Risk Scoring.

FAQ

What is this AGENTS.md Template for real-time chat architecture?

It provides a ready-to-paste operating manual that defines agent roles, handoffs, governance, memory, and security for live AI-driven chat systems.

How does multi-agent orchestration work in this template?

Agents operate in a coordinated lifecycle with a centralized orchestrator; each handoff is contract-driven with policy checks and memory updates before the next step.

What are the recommended handoff rules between agents?

Ingest → Moderation → Routing → Respond/ToolAccess, with memory updates and optional Domain Specialist review when needed.

How is memory and source-of-truth managed?

A central per-session memory store is the canonical source of truth; all agents read from and write to this memory under enforced provenance rules.

What security practices are included?

Secrets stored in vaults; least-privilege access; encrypted data in transit and at rest; audit logs with retention policies.