Daily Pipeline Operations for an AI Agent Team — Lessons Learned from a Supervised Drafting Pipeline

Introduction

We operate an engineering team composed of 15 AI Agents, executing tasks such as code development, content creation, and deployment verification on a daily basis. The core model is as follows: the CEO Agent (Charmander) handles decision-making and task distribution, functional Agents perform their respective roles, and the CC Supervision Layer conducts audits. This model has been in operation for over two months, during which we encountered numerous pitfalls. The following are our real-world experiences.

Core Principle: Separation of Coding and Deployment

The most significant lesson learned is that the role responsible for writing code must be separate from the role responsible for deploying it. We mandate that all code be written via ACP (Claude Code), after which a dedicated Deployment Agent (Bee) executes SSH operations. Security audits (Falcon) and acceptance testing (Hedgehog) are inserted in between. All four layers are indispensable.

Why? Because Agents suffer from "hallucinations"—they may claim files exist when they do not, or assert that a task is complete when it has not actually been executed. If the same Agent is responsible for both coding and deployment, it may continue to hallucinate during the deployment phase, masking issues until the production environment crashes.

Anti-Hallucination Mechanisms

Our countermeasure: Every completion report must include actual output from commands like `ls`, `curl`, or `psql`; otherwise, the task is considered incomplete. This rule has saved us multiple times.

Specifically:

• **File Existence**: Use `ls -la <path>`. Do not rely on memory to claim "it should be there."
• **End-to-End Connectivity**: Use `curl -s -o /dev/null -w "%{http_code} %{size_download}" https://your-site.com/page`. Is the HTTP code 200? Is the body size greater than 100 bytes?
• **Service Status**: Use `ss -tlnp | grep :port`. Directory existence does not equal process execution.

Three Tracks of the Daily Content Pipeline

We maintain three parallel content tracks: Diary (daily lab records), Articles (long-form engineering experience summaries), and Skill Recommendations (sharing tools and workflow patterns). Each track has independent quality gates and publishing processes to ensure content does not interfere with one another.

Track 1: Diary

The Diary is a chronological record of daily laboratory activities. Requirements: It must be based on verifiable project activities; do not fabricate events or client names that did not occur. The Day count must be calculated precisely (Day 1 = 2026-03-07).

Track 2: Article

Articles are long-form summaries of engineering experiences. Requirements: Remove the "AI tone," be specific but anonymized. Do not mention specific clients, companies, or product names unless approved. Focus on methodologies and lessons learned.

Track 3: Skill-Market

Skill Recommendations feature one useful Agent skill or workflow pattern per day. Requirements: Explain when to use it, when not to use it, and provide a short checklist. Categorize these as skills, not client-specific content.

Key Metrics

• **Task Completion Rate**: Automatically tracked via the tracker.
• **Hallucination Rate**: Statistics on false reports compiled by the CC Supervision Layer.
• **Delivery Latency**: Average time from distribution to completion.

Conclusion

An AI Agent team is not magic; it is an engineering system requiring strict discipline. The core lies in clear role boundaries, robust verification mechanisms, and rapid error exposure. Without this infrastructure, adding more Agents merely amplifies chaos.

Two months of operations have taught us: Trust, but verify. Agents can handle a large volume of repetitive work, but there must be an independent verification layer. The core value of this model is not in the degree of automation, but in freeing human attention from trivial execution details, allowing focus on higher-level decision-making and quality control.