AGENTS.md Template for Image Processing Pipelines

Overview

The AGENTS.md template for image processing pipelines provides a formal operating context for AI coding agents that orchestrate a sequence of image processing steps. This template governs a multi-agent workflow including ingestion, preprocessing, feature extraction, model inference, and post-processing, while also supporting single-agent operation. It defines roles, handoffs, memory rules, and governance to ensure predictable outputs.

Direct answer: This template establishes a repeatable, auditable operating manual for AI coding agents working on image processing pipelines from start to finish, with clear handoffs and governance for both single and multi-agent setups.

When to Use This AGENTS.md Template

• When building an image processing pipeline that relies on AI coding agents across multiple stages
• When explicit handoff rules, context sharing, and auditability are required
• When tool governance and secrets management must be enforced
• When human review is necessary for critical outputs and rollback safety is desired

Copyable AGENTS.md Template

# AGENTS.md

Project role: Image Processing Pipeline Lead and QA steward.

Agent roster and responsibilities:
- IngestAgent: pull images, normalize metadata, and feed to PreprocessAgent.
- PreprocessorAgent: apply normalization, resizing, color space conversion, and augmentation cues.
- FeatureExtractorAgent: compute features (e.g., SIFT/ORB) or embed images for downstream models.
- ModelInferenceAgent: run the model on preprocessed data; produce embeddings or predictions.
- PostprocessorAgent: convert model outputs into final results and metadata.
- QualityAssuranceAgent: run checks and flag anomalies; trigger human review when needed.
- OrchestratorAgent: coordinates all steps, enforces memory rules, and routes handoffs.
- DomainExpertAgent: provides domain-specific validation and corrections (e.g., medical or industrial imaging).
- AuditorAgent: records decisions, logs, and tool usage for audit trails.

Supervisor or orchestrator behavior:
- The OrchestratorAgent maintains a per-image context map and passes memory fragments between steps.
- Handoffs are triggered by defined success criteria or exception signals.
- All critical decisions require supervisor validation before external effects.

Handoff rules between agents:
- Ingest → Preprocessor: on ingest success and metadata normalization.
- Preprocessor → FeatureExtractor: after normalization and augmentation.
- FeatureExtractor → ModelInference: after feature computation and caching.
- ModelInference → Postprocessor: after inference completes.
- Postprocessor → QualityAssurance: after formatting results and adding provenance.
- If any step fails, Orchestrator aborts current path and triggers rollback or domain review.

Context, memory, and source-of-truth rules:
- Maintain a per-image memory segment with: image_id, source, timestamps, provenance, and model version.
- Source of Truth is the canonical data store housing the original image and the final output.
- All agents read from the source-of-truth store; writes are versioned and audited.

Tool access and permission rules:
- Only the Orchestrator and designated agents may call tools or services; every call is logged.
- Secrets are injected through a secure vault; direct secret access is forbidden for agents.
- API calls must be idempotent where possible; retries are bounded.

Architecture rules:
- Microservice-like boundaries between Ingest, Preprocess, Feature, Inference, and Postprocess stages.
- Stateless agents with a shared memory map managed by the Orchestrator.
- Clear contracts on input/output schemas for each handoff.

File structure rules:
- All workflow scripts sit under ./ai-skills/agents-md-templates/, with subfolders per role.
- All configuration under ./configs/ and data under ./data/.

Data, API, or integration rules:
- Use private endpoints for model inference; external calls require audit trails.
- Input images stored in the canonical data store; outputs saved with the same image_id.

Validation rules:
- Each step must emit a validation payload with success flag, provenance, and version.
- If validation fails, halt pipeline and route to human review.

Security rules:
- No plaintext secrets in code; secrets reside in a vault.
- Access must follow least privilege; rotate credentials periodically.

Testing rules:
- Unit tests exist for each agent's contract; integration tests cover end-to-end runs.
- Simulated failures included to verify rollback paths.

Deployment rules:
- Deploy agents and configs via CI/CD with feature flags; require approval for production shifts.

Human review and escalation rules:
- Human review is mandatory for anomalies, uncertain classifications, or critical outputs.
- Escalate to domain expert when domain-specific validation is inconclusive.

Failure handling and rollback rules:
- On failure, rollback to last known-good state and notify orchestrator for remediation.

Things Agents must not do:
- Do not bypass handoffs or memory contracts.
- Do not access or modify the canonical data store directly unless allowed by governance.

Recommended Agent Operating Model

Agents are organized into specialized roles with a central Orchestrator coordinating memory, handoffs, and the audit trail. The model supports a small set of domain specialists for imaging domains and strict tool governance. Decision boundaries are explicit, and escalation paths exist for human review and failure remediation.

Recommended Project Structure

./ai-skills/agents-md-templates/
├── planner
├── orchestrator
├── ingest_agent
├── preprocess_agent
├── feature_extractor_agent
├── model_inference_agent
├── postprocessor_agent
├── quality_assurance_agent
├── domain_expert_agent
├── auditor_agent

./configs/
└── image_processing_pipeline.yaml

./data/
└── raw_images/

./pipelines/image_processing/
├── ingest.yaml
├── preprocess.yaml
├── extract_features.yaml
├── inference.yaml
├── postprocess.yaml
└── test.yaml

Core Operating Principles

• Explicit agent roles with defined handoffs and memory contracts.
• Least privilege access to tools and secrets.
• Auditable actions with traceable decisions.
• Human-in-the-loop for uncertain or high-stakes outcomes.
• Idempotent operations and deterministic outputs where possible.

Agent Handoff and Collaboration Rules

• Planner ensures stage sequencing and feasibility before execution.
• Implementers must validate inputs against schemas before running tasks.
• Reviewers validate outputs and provenance before downstream use.
• Researchers and domain specialists provide validation for domain-specific checks.
• Handoffs require explicit success signals and context transfer payloads.

Tool Governance and Permission Rules

• All tool usage is mediated by the Orchestrator; direct tool calls by agents are forbidden.
• Secrets are retrieved from a secure vault; never embedded in code or memory.
• All API calls are logged and rate-limited; access controlled by least privilege.
• Production endpoints require approval gates and feature flags.

Code Construction Rules

• Follow schema-driven interfaces for all agent messages.
• Outputs must include provenance, version, timestamps, and validation status.
• Do not hard-code paths; read from canonical data store with versioning.
• Use idempotent, retry-safe operations for file writes and API calls.

Security and Production Rules

• All data in transit must be encrypted; at-rest encryption for stored artifacts.
• Access controls for agents, with role-based permissions and audit logging.
• Regular secret rotation and secret-scanning in CI pipelines.

Testing Checklist

• Unit tests for each agent contract and memory payload.
• Integration tests for end-to-end image processing pipelines.
• Failover and rollback tests across multiple failure modes.
• Security and permission tests for tool access and secrets handling.

Common Mistakes to Avoid

• Skipping explicit handoffs or memory contracts between agents.
• Over-permitting tools or ignoring audit trails.
• Skipping human review in high-stakes outputs.
• Drift between architectural contracts and actual implementation.

Related implementation resources: AI Use Case for Sales Pipeline Reviews and Deal Risk Scoring and AI Use Case for Content Marketers Using Wordpress To Auto-Translate Blog Posts Into Multiple Languages.

FAQ

What is this AGENTS.md Template for image processing pipelines?

This template provides a formal operating manual for AI coding agents orchestrating image processing stages—ingest, preprocess, feature extraction, model inference, and post-processing—covering both single-agent and multi-agent coordination.

How are agent handoffs defined and triggered?

Handoffs happen at defined stage boundaries with explicit trigger criteria and memory/context transfer rules to prevent drift.

How is tool governance enforced in this workflow?

Agents operate with minimal permissions, audit trails, and controlled tool access; all API calls and edits pass through a supervisor.

What are the failure-handling rules?

On failure, orchestrator rolls back to the last good state, records the incident, and notifies the domain expert for remediation.

Is human review required in the workflow?

Yes. Critical outputs and uncertain results require human review before downstream use.