AGENTS.md Template for Data Quality Monitoring Systems

Overview

The AGENTS.md Template for Data Quality Monitoring Systems defines a project-level operating manual to govern AI coding agents that perform data quality checks, anomaly detection, and remediation orchestration. It supports both single-agent execution and multi-agent orchestration with explicit handoffs, memory rules, and tool governance. Direct answer: This template provides roles, runbooks, and guardrails to run data quality checks reproducibly with AI agents and to coordinate across agents and tools.

It establishes a shared context, sources of truth, and escalation paths so data teams can audit, reproduce, and improve data quality workflows with confidence.

When to Use This AGENTS.md Template

• When bootstrapping a data quality monitoring system that relies on AI coding agents to validate upstream data quality checks.
• When implementing multi-agent orchestration for rule-based checks, anomaly detection, and remediation triggers.
• When you require a documented operating model with clear handoffs, access controls, and escalation paths.
• When you need reproducible runbooks for audits, governance, and onboarding of new team members.
• When integrating multiple data sources, pipelines, and tools under a single governance framework.

Copyable AGENTS.md Template

# AGENTS.md

# AGENTS.md Template for Data Quality Monitoring Systems

Overview
- Objective: Govern data quality checks executed by AI coding agents across data sources and pipelines.
- Scope: Single-agent checks and multi-agent orchestration with planner, implementer, reviewer, researcher, and domain-specialist roles.

Project role
- Owner: Data Platform Lead
- Objective: Provide a reproducible, auditable data quality monitoring workflow using AI agents.

Agent roster and responsibilities
- Planner: Designs runbooks, defines tasks, and sequences agent handoffs.
- Implementer: Executes quality checks (statistical tests, schema validations, schema drift checks) and generates artifacts.
- Reviewer: Verifies outputs, validates results against baselines, and approves or requests changes.
- Researcher: Gathers external rules, industry baselines, and data quality metrics; maintains rule catalog.
- Domain Specialist: Ensures integration with data sources, pipelines, and catalog metadata; handles domain-specific checks.

Supervisor or orchestrator behavior
- The Planner acts as the orchestrator, initiating tasks, dispatching to Implementer, and triggering Review when outputs are produced.
- The orchestrator must log task state, run IDs, and outcomes to a centralized truth store.

Handoff rules between agents
- Handoff 1: Planner → Implementer with task definition, data sources, and expected outputs.
- Handoff 2: Implementer → Reviewer with artifacts, results, and baselines.
- Handoff 3: Reviewer → Planner for remediation suggestions or approval to deploy.
- All handoffs must include context, evidence, and a memory pointer to the source-of-truth.

Context, memory, and source-of-truth rules
- Context: Dataset name, pipeline version, data sources, schemas, and current run ID.
- Memory: Long-lived memory stores for rules and baselines; ephemeral task context for each run.
- Source of truth: Canonical data warehouse, metadata catalogs, and policy definitions; all outputs reference the source-of-truth.

Tool access and permission rules
- Tools: Python runtime, SQL/query interface, REST APIs, Git, secrets store.
- Permissions: Least-privilege for each agent; secrets never embedded in code; access granted via role tokens.
- Secrets: Retrieve from secret store at runtime; rotate credentials per run when possible.

Architecture rules
- Use a modular, pluggable check architecture; each check is versioned and auditable.
- All checks must be deterministic and idempotent where possible.
- Runbooks must be stored under /checks/ and versioned via Git.

File structure rules
- data-quality-ops/
  - data_sources/
  - pipelines/
  - checks/
  - models/
  - rules/
  - memory/
  - configs/
  - docs/
  - tests/
- agents/
  - planner/
  - implementer/
  - reviewer/
  - researcher/
  - domain-specialist/
- tooling/
- deploy/

Data, API, or integration rules when relevant
- All data quality artifacts must reference upstream data sources and be traceable to metadata catalogs.
- Use API-based integrations with rate limits and retry policies; do not pollute data stores with transient artifacts.

Validation rules
- Each run must produce a result artifact with a run ID, pass/fail status, and a summary of findings.
- Validation against baselines and thresholds; abnormal deviations trigger alerts and human review.

Security rules
- Do not expose production secrets in code or logs.
- Enforce MFA for critical actions and require approvals for changes that affect production data.

Testing rules
- Unit tests for each quality check; integration tests to validate end-to-end data flow.
- End-to-end tests simulate data drift scenarios; tests must pass before deployment to production.

Deployment rules
- Changes require pull-request review and approvals.
- Runbooks deployed to a staging environment before production promotion.

Human review and escalation rules
- All failing runs require human review; escalate to data governance when data quality thresholds are breached.
- Provide a remediation plan and a rollback strategy.

Failure handling and rollback rules
- If a run fails, halt downstream pipelines and revert to the last known good state.
- Notify stakeholders and log the incident for postmortem.

Things Agents must not do
- Do not mutate production data without explicit approvals.
- Do not bypass runbook steps or skip human reviews.
- Do not leak secrets or credentials in logs or artifacts.

Recommended Agent Operating Model

Roles, responsibilities, decision boundaries, and escalation paths are defined to balance automation with governance. The Planner defines the runbook and gates; the Implementer executes checks and generates artifacts; the Reviewer approves or requests changes; the Researcher and Domain Specialist augment rules and integration fidelity; escalations route to human review when thresholds or governance policies are breached.

Recommended Project Structure

data-quality-ops/
  data_sources/
  pipelines/
  checks/
  models/
  rules/
  memory/
  configs/
  docs/
  tests/
  deployments/
  artifacts/

agents/
  planner/
  implementer/
  reviewer/
  researcher/
  domain-specialist/

Core Operating Principles

• Single source of truth and auditable run history
• Idempotent, deterministic checks
• Least privilege and secret hygiene
• End-to-end traceability from data source to remediation
• Governed by human-in-the-loop when thresholds are breached
• Documentation-driven, versioned checks and runbooks

Agent Handoff and Collaboration Rules

• Planner to Implementer: include task scope, data sources, schemas, baselines, and success criteria.
• Implementer to Reviewer: attach artifact, results, logs, and citations to rules.
• Reviewer to Planner: provide remediation actions or approval for deployment.
• Researcher and Domain Specialist assist at handoff points with updated rules and integration notes.
• Escalate any governance or data sensitivity issues to human review immediately.

Tool Governance and Permission Rules

• Command execution limited to approved runtimes; no destructive commands without approval.
• File edits must go through version control with PR approvals; secrets never stored in code or logs.
• API calls require rate limiting, retries, and access controls; sensitive endpoints require extra approvals.
• Production systems require change-management gates and observability before and after changes.
• Approval gates must be logged with rationale and the responsible owner.

Code Construction Rules

• Write modular, testable quality checks with clear inputs/outputs.
• Version all checks and memory models; link artifacts to run IDs.
• Avoid hard-coding credentials; rely on secret stores.
• Document assumptions and data schemas used by each check.

Security and Production Rules

• Do not bypass security controls; enforce data access restrictions and auditing.
• Run checks in isolated environments and promote changes via controlled pipelines.
• Maintain data privacy and comply with applicable regulations in all checks.

Testing Checklist

• Unit tests for each quality check with synthetic data.
• Integration tests for data source connectivity and rule execution.
• End-to-end tests simulating drift scenarios and remediation flows.
• Security tests for secret handling and access controls.
• Deployment tests in staging before production promotion.

Common Mistakes to Avoid

• Assuming a single check covers all data quality dimensions.
• Skipping documentation and versioning of checks.
• Hard-coding data paths or credentials in code or templates.
• Failing to establish governance and escalation paths for data quality issues.

Related implementation resources: AI Use Case for Nutritionists Using Myfitnesspal Data To Generate Customized Meal Plans Matching Specific Macro Goals and AI Use Case for Intern Coordinators Using Trello To Track and Automate Weekly Project Evaluations for Cohorts.

FAQ

What is the purpose of this AGENTS.md Template for Data Quality Monitoring Systems?

This template defines roles, runbooks, and guardrails to govern AI coding agents running data quality checks and coordinating multi-agent workflows for data quality governance.

Who should be in the agent roster and what are their responsibilities?

Planner designs runbooks, Implementer executes checks, Reviewer validates outputs, Researcher sources rules, and Domain Specialist ensures integration with data sources and pipelines.

How are handoffs between agents managed?

Handoffs follow a defined sequence with context and source-of-truth references: Planner > Implementer, Implementer > Reviewer, Reviewer > Planner for remediation or deployment decisions.

What security and governance rules apply?

Enforce least privilege, secret hygiene, gated approvals, audit logging, and non-production data handling policies for all agent activity.

How are failures handled and rollback performed?

On failure, revert to the last known good run, pause affected pipelines, notify stakeholders, and escalate to human review with a remediation plan.