AGENTS.md Template for Large File Upload Systems

Overview

Direct answer: This AGENTS.md Template codifies a robust operating model for large file upload systems using AI coding agents, enabling both single-agent and multi-agent orchestration with clear handoffs, validation gates, and security controls.

This template defines an end-to-end workflow for chunked uploads, integrity verification, and durable storage, while ensuring tool governance and human review where needed. It focuses on concrete roles, expectations, and measurable outputs to prevent context drift and architecture drift in production systems.

When to Use This AGENTS.md Template

• Building a scalable large file upload pipeline that uses AI coding agents to orchestrate ingest, validation, and storage tasks.
• Implementing multi-agent coordination with explicit handoffs and escalation paths.
• Enforcing strict tool access, secrets, and production-ready governance for external services.
• Providing a project-level operating context that teams can paste into a new repository for consistent workflow behavior.

Copyable AGENTS.md Template

# AGENTS.md

Project role:
- Lead Architect (Orchestrator): defines policies, coordinates agents, and gates releases.
- IngestAgent: handles chunked file reception, resumption, and chunk ordering.
- ChecksumAgent: validates integrity with checksums and cross-checks totals.
- ValidationAgent: enforces policy checks (size limits, MIME types, virus scan markers).
- StorageAgent: writes to durable storage (object store) and emits id/paths.
- AuditorAgent: monitors, logs, and triggers human review on anomalies.
- HandoffAgent: implements explicit handoffs and ensures state consistency.

Agent roster and responsibilities:
- IngestAgent: receive chunks, deduplicate, reassemble, emit chunk metadata.
- ChecksumAgent: compute and verify checksums for each chunk and final file hash.
- ValidationAgent: validate metadata, access controls, and policy conformance.
- StorageAgent: persist combined file, update metadata registry.
- Orchestrator: sequence tasks, retries, timeouts, and escalation rules.
- AuditorAgent: ensure traceability, auditing, and anomaly reporting.

Supervisor or orchestrator behavior:
- Orchestrator enforces a strict plan with retries, backoff, and timeouts.
- It triggers handoffs when a previous step completes the required success criteria.
- It refuses unsafe shortcuts (no shortcutting checks, no secret leakage, no bypassing approvals).

Handoff rules between agents:
- DataHandOff: IngestAgent -> ChecksumAgent; include chunk IDs and partial hashes.
- IntegrityHandOff: ChecksumAgent -> ValidationAgent; include final hash and size.
- PersistHandOff: ValidationAgent -> StorageAgent; include final metadata and storage path.
- ReviewHandOff: AuditorAgent triggers human review when anomaly is detected.

Context, memory, and source-of-truth rules:
- Maintain a single source of truth: a structured manifest with file_id, total_size, chunk_count, and hashes.
- All agents read from and write to the manifest; no local snapshotting beyond allowed cache with TTL.
- Context memory must be cleared after successful completion unless explicitly archived.

Tool access and permission rules:
- Agents can call storage APIs, checksum computation services, and virus/malware scans with scoped tokens.
- Secrets must be injected via a secure vault; never log secrets.
- No direct user prompts; all actions are automated with explicit approvals where required.

Architecture rules:
- Stateless agents; state stored in the manifest and event store.
- Idempotent operations for chunk processing and finalization.
- Central orchestrator enforces idempotence and reproducible outcomes.

File structure rules:
- project/
  - agents/
    - ingest_agent/
    - checksum_agent/
    - validation_agent/
    - storage_agent/
    - auditor_agent/
    - orchestrator/
  - workflows/
  - services/
  - config/
  - tests/

Data, API, or integration rules:
- All metadata in a manifest.json with file_id, total_size, chunk_count, and hashes.
- API calls to storage and validation services must pass through a verified gateway.

Validation rules:
- Each chunk must hash to its stored value; final file hash must match manifest.
- Size & MIME type restrictions must be enforced before storage.
- All steps must log outcomes with timestamps.

Security rules:
- Secrets stored in vault; tokens rotated on schedule.
- Access controls restricted by role; no cross-project access.
- All external calls go through allowlists and mutual TLS.

Testing rules:
- Unit tests for each agent; integration tests for the end-to-end flow.
- Simulated network failures with retry logic validated.
- Regression tests for chunk reassembly and hash verification.

Deployment rules:
- CI checks pass before deployment; stages mirror production policy.
- Canary rollout for new agents; rollback triggers on failures.
- Observability: metrics, traces, and logs stored securely.

Human review and escalation rules:
- AuditorAgent raises tickets for anomalies; human review required for size anomalies or mismatched hashes.
- All escalations require manager approval and documented rationale.

Failure handling and rollback rules:
- On any failure, revert state to manifest; re-run from last successful handoff.
- Do not skip validation or storage steps during recovery.

Things Agents must not do:
- Do not leak secrets, bypass approvals, or mutate final manifests without audit.
- Do not re-interpret chunk ordering; do not drop chunks without re-assembly validation.
- Do not perform destructive deletes without explicit escalation.

Recommended Agent Operating Model

The operating model defines roles, decision boundaries, and escalation paths for large file uploads. It enables clear separation of concerns between Ingest, Validation, and Storage while preserving end-to-end traceability via a central Orchestrator and Auditor.

Recommended Project Structure

project/
  agents/
    ingest_agent/
    checksum_agent/
    validation_agent/
    storage_agent/
    auditor_agent/
    orchestrator/
  workflows/
  services/
  config/
  tests/

Core Operating Principles

• Single source of truth for file manifests and chunk metadata.
• Idempotent, auditable, and reproducible agent actions.
• Explicit handoffs with validated success criteria before progression.
• Least privilege and strict secrets management.
• Human-in-the-loop when anomalies or policy violations occur.

Agent Handoff and Collaboration Rules

• Planner (Orchestrator) schedules steps and defines success criteria for each handoff.
• IngestAgent must emit chunk IDs and a provisional manifest before checksums.
• ChecksumAgent completes must-have hashes before ValidationAgent proceeds.
• ValidationAgent enforces policy checks; upon success, StorageAgent persists data.
• Tester and Reviewer roles verify end-to-end integrity and policy conformance before production.
• Domain Specialist may assist when file types or regulatory constraints require it.

Tool Governance and Permission Rules

• Commands and API calls must be scoped, auditable, and approved by the Orchestrator.
• File edits and metadata updates require formal approvals and versioning.
• Secrets must never be logged or surfaced in outputs.
• Production system changes require change tickets and canary deployments.
• All external services must be allowlisted and accessed via a secure gateway.

Code Construction Rules

• All modules expose idempotent APIs and deterministic outputs.
• Use strongly typed schemas for manifests and chunk metadata.
• Document interfaces and memory boundaries; avoid hidden state.
• Handle retries with exponential backoff and bounded total time.
• Validation steps must be explicit; do not infer results without checks.

Security and Production Rules

• Only authenticated agents can access storage and validation services.
• Secrets are stored in a vault; rotation and access controls are required.
• Production changes require approvals and rollback plans.
• All data at rest and in transit must be encrypted.

Testing Checklist

• Unit tests for each agent with deterministic inputs.
• End-to-end integration tests simulating chunked uploads and reassembly.
• Failure mode tests: network drop, partial writes, hash mismatches.
• Canary and rollback tests for deployment changes.

Common Mistakes to Avoid

• Over-optimistic handoffs without verifying state consistency.
• Logging secrets or exposing internal traces in outputs.
• Skipping validation steps to accelerate flow.
• Not accounting for partial failures across multiple agents.

Related implementation resources: AI Use Case for Corporate Event Managers Using Slack To Orchestrate Day-Of Venue Tasks Across Multi-Department Teams and AI Use Case for Xero Reports and Business Performance Insights.

FAQ

What is the purpose of this AGENTS.md Template for large file upload systems?

To provide a complete, copyable operating manual for AI coding agents that orchestrate chunked uploads, ensuring governance, handoffs, and human review where needed.

How does multi-agent orchestration work in this template?

The Orchestrator coordinates IngestAgent, ChecksumAgent, ValidationAgent, StorageAgent, and AuditorAgent via explicit handoffs, retries, and stateful manifests to guarantee end-to-end integrity.

What are the escalation paths for failed uploads?

Failures trigger AuditorAgent review and, if unresolved, escalation to project leads with rollback to the last known good manifest.

How are secrets and tool access controlled?

Secrets live in a vault, are rotated on schedule, and never logged. Access is restricted by role and instrumented with approvals.

How do we validate and rollback?

Validation checks final integrity; on failure, the orchestrator replays to the last valid state and rolls back partially completed steps.