AGENTS.md Template for Data Pipeline Reliability Reviews

Overview

Direct answer: This AGENTS.md template defines a compact operating manual for data pipeline reliability reviews. It governs how AI coding agents collaborate and how multi agent orchestration is managed, enabling auditable outcomes for both single agent and multi agent workflows.

The AGENTS.md Template for Data Pipeline Reliability Reviews provides a repeatable, auditable pattern that anchors data lineage, data quality, tool governance, and human review within a single project level operating context.

When to Use This AGENTS.md Template

Starting a new data pipeline reliability review for a data product or pipeline.
Establishing a repeatable operating model across teams for data quality and lineage checks.
Coordinating AI coding agents to audit data transforms and data contracts.
Enforcing tool governance and access controls in production data workflows.
Setting up hands off rules and escalation paths for data incidents.

Copyable AGENTS.md Template

Copy this AGENTS.md into your project directory to capture the operating context for data pipeline reliability reviews.

# AGENTS.md
Project Role: Data Platform Engineer
Agent Roster and Responsibilities:
  Planner: Defines the data pipeline reliability review plan and acceptance criteria
  Implementer: Builds data tests, instrumentation, and data quality checks
  Reviewer: Validates outputs against requirements
  Tester: Executes end to end data tests
  Researcher: Gathers data lineage and domain constraints
  Domain Specialist: Provides subject matter guidance for data domains
Supervisor or Orchestrator Behavior:
  Orchestrator coordinates the roster, triggers handoffs, and records decisions
Handoff Rules:
  Planner -> Implementer when plan complete
  Implementer -> Reviewer when artifacts ready for review
  Reviewer -> Tester when validation passes
  Researcher/Domain Specialist assist as needed
Context Memory and Source of Truth:
  Memory stored in project workspace; Source of Truth is the data catalog and lineage records
Tool Access and Permissions:
  Access to data sources via vault; production changes require approval
Architecture Rules:
  Idempotent data transforms; idempotent jobs; event driven triggers
File Structure Rules:
  data-pipeline-reliability/
    /agents
      /planner
      /implementer
      /reviewer
      /tester
      /researcher
      /domain-specialist
    /orchestrator
    /configs
    /data
    /docs
Data API or Integration Rules:
  Use stable APIs; avoid breaking changes; versioned interfaces
Validation Rules:
  Data quality checks; schema validation; lineage verification
Security Rules:
  Secrets encrypted at rest; least privilege; audit logging
Testing Rules:
  Unit tests for transforms; integration tests for data flows; end to end tests with synthetic data
Deployment Rules:
  Stage before prod; immutable deployments
Human Review and Escalation Rules:
  Escalate data quality anomalies to data governance
Failure Handling and Rollback Rules:
  Rollback to last good state; notify stakeholders
Things Agents Must Not Do:
  Do not bypass handoffs; do not modify data in place outside approved transforms

Recommended Agent Operating Model

Roles, responsibilities, and decision boundaries for data pipeline reliability reviews. The model emphasizes bounded autonomy with escalation paths to ensure safety and auditable decisions in a multi agent environment.

Recommended Project Structure

data-pipeline-reliability/
  /agents
    planner/
    implementer/
    reviewer/
    tester/
    researcher/
    domain-specialist/
  /orchestrator/
  /configs/
  /data/
  /docs/

Core Operating Principles

Clear ownership and auditable decisions
Bounded autonomy with escalation gates
Idempotent operations and traceable history
Single source of truth for data lineage
Security by design and least privilege

Agent Handoff and Collaboration Rules

Planner -> Implementer: deliver plan, acceptance criteria, and plan artifacts
Implementer -> Reviewer: deliver transformed artifacts and validation artifacts
Reviewer -> Tester: trigger end to end test plan and results
Researcher -> Planner: provide lineage context and data domain constraints
Domain Specialist -> any role: supply domain specific guidance as needed

Tool Governance and Permission Rules

Only approved tools and APIs; credentials managed in vault
Production changes require approval gates and audit trails
Secrets access minimized and rotated regularly

Code Construction Rules

Write modular, testable transforms; document inputs and outputs
Follow schema and contract tests for data models
Prefer idempotent operations and idempotent jobs

Security and Production Rules

Data at rest and in transit encrypted; access controlled
Production rollouts require safe deploys and rollbacks

Testing Checklist

Unit tests for transforms
Integration tests for data flows
End to end tests with synthetic data
Data quality checks on production data

Common Mistakes to Avoid

Skipping handoffs leading to ambiguous ownership
Allowing ad hoc changes without audits
Overreliance on a single agent without checks

FAQ

What is this AGENTS.md Template used for in data pipeline reliability reviews?

This template provides an operating manual to coordinate AI coding agents during data pipeline reliability reviews and ensure auditable results.

Which agent roles are defined in this template for data pipelines?

Planner, Implementer, Reviewer, Tester, Researcher, and Domain Specialist with defined responsibilities.

How are agent handoffs governed in this workflow?

Handoffs follow a defined sequence with artifacts and tests; the orchestrator records decisions and triggers follow-on steps.

What constitutes tool governance in this AGENTS.md Template?

Access to data sources and production systems is controlled with approvals, vaults, and auditable logs.

How is security and production risk addressed?

Security controls, data quality checks, rollback plans, and human review gates are integrated into the workflow.

Target User

Use Cases