AGENTS.md Template for Horizontal Partitioning Design

Overview

Direct answer: This AGENTS.md template provides a complete operating manual for horizontal partitioning design, enabling both single-agent work and multi-agent orchestration to define shard keys, partitioning schemes, rebalancing plans, and governance across data stores. It codifies roles, handoffs, and guardrails so AI coding agents can operate safely and transparently at project scope.

When to Use This AGENTS.md Template

• When designing horizontal partitioning for a new data store, ensuring scalable writes and reads across shards.
• When coordinating multiple agents (planner, implementer, reviewer, tester, researcher, domain specialist) to converge on a shard strategy.
• When establishing tool governance, security, and deployment rules around partitioning migrations and rollbacks.

Copyable AGENTS.md Template

Below is a copyable AGENTS.md template block that you can paste into your project as the authoritative operating context. It governs both single-agent and multi-agent orchestration for horizontal partitioning design.

# AGENTS.md

Project role
- Data Platform Architect driving horizontal partitioning across distributed storage.

Agent roster and responsibilities
- Planner: defines shard strategy (hash vs range, shard keys, hot-key mitigation).
- Implementer: translates the plan into schema changes, migrations, and access rules.
- Reviewer: validates partition design against requirements and performance targets.
- Tester: runs correctness, performance, and failover tests.
- Researcher: gathers industry patterns, DB constraints, and vendor specifics.
- Domain Specialist: focuses on data governance, privacy, and regulatory considerations.
- Orchestrator: supervises handoffs, quality gates, and end-to-end coordination.

Supervisor or orchestrator behavior
- Orchestrator enforces entry criteria for each step, triggers handoffs, and records decisions in the design registry.
- All changes must pass validation gates and must be reviewed before production migration.

Handoff rules between agents
- Planner -> Implementer: provide final shard map, key decisions, and migration plan.
- Implementer -> Reviewer: present implemented schema changes and migration impact.
- Reviewer -> Tester: approve test suite and expected outcomes.
- Tester -> Orchestrator: report pass/fail and risk assessments.
- Researcher/Domain Specialist -> Planner: submit new constraints, governance considerations, or exceptions.

Context, memory, and source-of-truth rules
- Central design registry holds the canonical shard map, policy constraints, and validation results.
- All agents must reference the design registry for decisions and be traceable to the original design doc.
- Source of truth includes: shard_map.json, migration_plan.md, policy_matrix.md, and audit logs.

Tool access and permission rules
- Access to DB CLI, migration tooling, schema registry, and monitoring dashboards is granted to Implementer and Orchestrator only.
- Secrets and production credentials must be accessed via a vault, with approved request flows.
- All code changes must go through the repository with PR-based reviews.

Architecture rules
- Prefer horizontal partitioning with clear shard keys to minimize cross-shard joins.
- Avoid hotspot shards; plan rebalancing and shard splitting/merging as traffic grows.
- Ensure queries that span shards have defined corner cases and acceptable latency.
- Maintain deterministic shard placement and rollback paths.

File structure rules
- Keep a single source of truth in a design-registry folder with: shard_map.json, migration_plan.md, policy_matrix.md.
- Store architecture diagrams in docs/architecture/sharding.png or .svg.
- Use scripts/ or infra/ for tooling around migrations and validation.

Data, API, and integration rules
- All shard maps must be versioned; migrations must be reversible.
- APIs should support cross-shard reads with pagination and consistent read concerns.
- Integrations with analytics or BI must not expose shard-internal keys in k8s secrets or logs.

Validation rules
- Validate shard distribution against expected load curves.
- Verify cross-shard queries meet SLAs under simulated peak load.
- Confirm that rollback procedures restore original state in under target time.

Security rules
- Do not expose raw shard keys in logs; redact or alias where needed.
- Enforce least-privilege access for all agents and services.
- Encrypt data at rest and in transit for cross-shard replication.

Testing rules
- Unit tests for partition logic functions.
- Integration tests for migrations with a staging dataset.
- Performance tests for read/write latency across shards.
- Rollback tests to ensure safe revert paths.

Deployment rules
- Do not execute production migrations without explicit approval gates.
- Maintain a rollback window and monitor health after migration.
- Tag deployments with shard-compatibility status and risk level.

Human review and escalation rules
- Escalate high-risk design decisions to the architecture board.
- All production changes require a human review checkpoint before execution.

Failure handling and rollback rules
- If migration fails, roll back to the previous shard map and restore data invariants.
- Capture all errors, apply back-off, and revalidate before retrying.

Things Agents must not do
- Do not modify production data structures without approval.
- Do not bypass the design registry or consent gates.
- Do not drift from stated shard keys or governance constraints.
- Do not deploy without a backed-out rollback plan.

Recommended Agent Operating Model

The recommended operating model defines distinct roles with clear decision boundaries. Planner ideates shard strategies and risk budgets; Implementer turns plans into concrete schema and migrates data; Reviewer challenges assumptions and validates performance; Tester validates correctness and resilience; Researcher and Domain Specialist surface governance and regulatory constraints; Orchestrator coordinates handoffs and gates. Escalation paths exist for high-risk changes or uncertain outcomes.

Recommended Project Structure

Workflow-specific directory tree:

project-root/
  design-registry/
    shard_map.json
    migration_plan.md
    policy_matrix.md
  docs/
    architecture/
      sharding-design.svg
  src/
    plans/
    migrations/
  tests/
    unit/
    integration/
  infra/
    k8s/
    secrets/
  scripts/
    validate-sharding.py

Core Operating Principles

• Single source of truth for shard design.
• Explicit handoffs and gate checks between agents.
• Deterministic, auditable decision records.
• Security-first for all data and tooling access.
• Continuous improvement through formal post-mortems.

Agent Handoff and Collaboration Rules

Planners finalize shard keys and provide migration plan to Implementer. Implementers execute, with ongoing feedback from Domain Specialist. Reviewers validate decisions before tests. Testers execute test suites and report outcomes. Researchers provide governance input when needed. Orchestrator supervises all handoffs and maintains the decision log.

Tool Governance and Permission Rules

Case-by-case access requests to DB CLI, migration tools, and schema registries must be approved and logged. Secrets access is restricted to Authorized Roles via Vault. Changes must be tracked via PRs and require approvals before production use.

Code Construction Rules

Write idempotent migrations; avoid destructive ops without a rollback plan; favor declarative schemas; document all assumptions; include tests for migration idempotency and correctness.

Security and Production Rules

Encrypt data in transit and at rest; redact shard keys in logs; implement least privilege; verify compliance with governance requirements before production use.

Testing Checklist

• Unit tests for shard key calculation functions.
• Integration tests for migration scripts with staging data.
• End-to-end tests for cross-shard queries.
• Rollback verification tests and backups validation.

Common Mistakes to Avoid

• Ignoring hot shards and load distribution.
• Unclear rollback paths or missing verification gates.
• Skipping governance review for production migrations.
• Assuming cross-shard queries are always performant without testing.

Related implementation resources: AI Agent Use Case for Intermodal Transport Providers Using Rail Schedules To Coordinate Seamless Truck-To-Train Transfers and AI Use Case for Sales Pipeline Reviews and Deal Risk Scoring.

FAQ

What determines hash vs range partitioning in horizontal design?

Hash partitioning provides even load across shards; range partitioning is preferable for time-series data and predictable query patterns. Evaluate query types, hot keys, growth projections, and rebalancing costs to decide.

How do you ensure cross-shard queries stay performant?

Define shard keys to minimize cross-shard joins, implement explicit routing, and provide optimized distributed query plans. Benchmark with representative workloads and enforce SLAs on cross-shard latency.

What is the role of agent handoffs in this template?

Handoffs ensure specialized expertise at each stage. Planner hands off to Implementer with a complete shard map; Implementer hands off to Reviewer; Tester verifies and reports; Orchestrator logs decisions and gates progress.

How should failures be handled during partition design?

Abort changes, roll back to the previous shard map, validate data integrity, re-run tests, and re-plan with mitigations before reattempting migrations.

When should human review occur in the workflow?

For high-risk changes, policy violations, or when uncertainty exceeds predefined risk thresholds. No production migration without a documented human review.