Don’t Let “AI Automation” Become Your Technical Debt: Reflections on Moving from 100 Agents to One Reliable Pipeline

Over the past six months of delivering projects for our AI Lab, we fell into a typical “cognitive trap.” Initially, we were obsessed with building complex agent orchestrations. We designed agents with distinct personas—such as “Content Director,” “Code Expert,” and “Quality Auditor”—attempting to close the loop from topic selection to publication through a massive multi-agent collaboration network.

The result? We fell into what we call “Agent Entropy Increase.”

1. The Illusion of “Collaboration” and the Reality of Latency

In our early architecture, an article publication workflow had to pass through: Topic Selection Agent $\rightarrow$ Outline Agent $\rightarrow$ Body Text Agent $\rightarrow$ Translation Agent $\rightarrow$ Review Agent.

On the surface, this mimicked how a human editorial team operates. However, in practice, we uncovered three fatal issues:
- Context Drift: During handoffs, the original intent of the topic was diluted by the third stage.
- Error Cascading: If the Outline Agent hallucinated on a logical point, all subsequent agents would “elegantly expand” upon that error. The final output might read smoothly but be completely wrong.
- Unpredictable Latency: Multi-step LLM calls mean latency is cumulative, and a timeout at any single node could cause the entire pipeline to collapse.

2. Returning from “Orchestration” to “Pipeline”

We realized that for deterministic delivery goals (such as daily article publishing), a Pipeline is far more reliable than Agents.

We refactored our architecture into: Single Strong Model + Structured Prompts + Hard-coded Validation Logic.

Core Logic After Refactoring:

1. Atomic Tasks: Instead of asking an agent to “think about how to write,” we provide it with an extremely specific instruction set (e.g., “Based on the following project logs, extract three specific technical pain points and write them up using a ‘Problem-Solution-Result’ structure”).
2. State Machine Driven: We use Python scripts to control the flow rather than relying on the LLM’s autonomous decision-making. The output of each step must pass validation via regex or JSON Schema. If it fails, the system immediately retries or throws an error, instead of passing it to the next agent to “guess.”
3. Single-Point Translation Strategy: We abandoned multi-turn conversational translation in favor of a one-shot translation prompt with contextual constraints, ensuring strict terminology alignment across zh-CN, zh-TW, and en.

3. Three Key Details in Engineering Practice

When deploying automated publishing for SFD V4 CMS, we adopted the following engineering measures to mitigate risk:

A. Stateless Token Management

Never hard-code credentials in your code. We implemented a cache file combined with a periodic refresh mechanism. Upon startup, the script performs a write_probe (attempting to write a dummy object). If it returns a 401 error, it triggers a re-login process. This prevents late-night publication failures caused by expired tokens.

B. Asynchronous Decoupling of “Cover Image” and “Content”

Cover image generation (Image Gen) is typically much slower and less stable than text generation. We placed cover image generation after the article POST request, associating it asynchronously via PATCH /api/v4/articles/:id. Even if image generation fails, the article can still be published with a placeholder image, preventing blockage of the entire publication chain.

C. Public-Facing End-to-End Verification (E2E Verify)

The most dangerous kind of success is when the API returns 200 OK but the frontend page is blank. We added a public API callback check as the final step of the pipeline: requesting /api/v4/articles?slug=...&locale=... and verifying whether the returned status is published. Only if the callback checks succeed for all three languages is the task marked as PASS.

4. Advice for AI Engineers

If you are building AI applications, remember: Logic that can be implemented with if/else should never be left to an LLM to decide; tasks that can be completed with a Pipeline should not be attempted using Multi-Agent simulations of collaboration.

The value of AI lies in handling unstructured creative aspects (such as transforming dry project logs into engaging articles), while the value of engineering lies in wrapping that creativity within an extremely boring, deterministic, and predictable pipeline.