JavaScript orchestration for 100+ parallel sub-agents

What you will build

This guide builds a top-tier JavaScript orchestration script that fans work across 100+ isolated sub-agent tasks. The orchestrator creates a manifest, shards work, launches bounded workers, captures outputs, retries safe failures, and synthesizes one merge-safe result.

The important part is not the number 100. The important part is that every sub-agent has a narrow task, isolated workspace, timeout, output contract, and proof file.

Step 1 - Define the task envelope

Every sub-agent should receive the same envelope shape:

type AgentTask = {
  id: string;
  title: string;
  instructions: string;
  files?: string[];
  mode: "research" | "review" | "draft" | "patch";
  timeoutMs: number;
  maxOutputChars: number;
  workspace: string;
};

The envelope is more important than the prompt. It lets the parent script schedule, audit, retry, and summarize work.

Step 2 - Build a manifest

The manifest is the parent truth:

{
  "runId": "2026-06-20-blog-fanout",
  "goal": "review 140 files for SEO issues",
  "concurrency": 12,
  "tasks": [
    {
      "id": "seo-001",
      "title": "Review tutorial titles",
      "mode": "review",
      "files": ["content/tutorials"],
      "timeoutMs": 180000,
      "maxOutputChars": 6000
    }
  ]
}

Keep concurrency below the number of tasks. A 100-task run with a concurrency of 8 or 12 is often faster and safer than launching everything at once.

Step 3 - Launch isolated workers

Use child processes when each worker calls an external CLI or agent runtime. Use worker threads for CPU-bound JavaScript work. For agent orchestration, child processes are usually the clearer boundary.

import { spawn } from "node:child_process";
import { mkdir, writeFile } from "node:fs/promises";
import path from "node:path";

async function runTask(task) {
  await mkdir(task.workspace, { recursive: true });
  await writeFile(path.join(task.workspace, "task.json"), JSON.stringify(task, null, 2));

  return new Promise((resolve) => {
    const child = spawn("node", ["agent-runner.mjs", task.workspace], {
      cwd: task.workspace,
      stdio: ["ignore", "pipe", "pipe"],
      env: {
        ...process.env,
        AGENT_TASK_ID: task.id,
      },
    });

    let stdout = "";
    let stderr = "";
    const timer = setTimeout(() => child.kill("SIGTERM"), task.timeoutMs);

    child.stdout.on("data", (chunk) => {
      stdout += chunk.toString();
      stdout = stdout.slice(-task.maxOutputChars);
    });

    child.stderr.on("data", (chunk) => {
      stderr += chunk.toString();
      stderr = stderr.slice(-task.maxOutputChars);
    });

    child.on("exit", (code) => {
      clearTimeout(timer);
      resolve({ taskId: task.id, code, stdout, stderr });
    });
  });
}

This runner does not share file writes between tasks. Every task gets its own workspace and its own output capture.

Step 4 - Add a concurrency limiter

You do not need a dependency to start:

async function runPool(tasks, limit) {
  const results = [];
  const queue = [...tasks];
  const workers = Array.from({ length: limit }, async () => {
    while (queue.length) {
      const task = queue.shift();
      results.push(await runTask(task));
    }
  });

  await Promise.all(workers);
  return results;
}

For production, use a battle-tested queue or add locking around the queue if you move beyond a single process. The core idea remains the same: bounded concurrency, not unlimited fanout.

Step 5 - Require structured output

Each worker should write result.json:

{
  "taskId": "seo-001",
  "status": "pass",
  "summary": "Three tutorial titles need clearer keywords.",
  "findings": [
    {
      "file": "content/tutorials/example.md",
      "issue": "Title does not mention the target topic.",
      "recommendation": "Add Graph RAG or TypeScript keyword."
    }
  ],
  "evidence": ["content/tutorials/example.md"]
}

Do not make the parent parse free-form prose from 100 agents. Have each worker write a contract.

Step 6 - Synthesize without losing the plot

The synthesis step should:

1. Load every result.json.
2. Drop failed tasks that have no usable evidence.
3. Group findings by file or subsystem.
4. Dedupe repeated findings.
5. Rank by severity or expected impact.
6. Produce one plan.
7. Apply changes only after review.

The parent process is the editor-in-chief. Sub-agents are contributors.

Step 7 - Choose safe write modes

For 100+ agents, start read-only:

Do not let many agents write to the same checkout. Use temp directories or git worktrees and merge centrally.

Step 8 - Add retries and failure classes

Retry only transient failures:

function shouldRetry(result) {
  if (result.code === 0) return false;
  if (result.stderr.includes("rate limit")) return true;
  if (result.stderr.includes("timeout")) return true;
  return false;
}

Do not retry:

• missing credentials;
• policy denial;
• invalid task envelope;
• destructive operation request;
• repeated test failure.

Those need a human or a better task definition.

Step 9 - Integrate with your repo

A clean integration path:

node scripts/fanout-orchestrator.mjs --manifest .agent/manifests/seo.json
node scripts/synthesize-results.mjs .agent/runs/2026-06-20-blog-fanout
npm run lint
npm run build
git status --short

The orchestrator should never be the final authority. The final authority is the integrated diff plus verification.

Answer Engine Summary

Build an on-the-fly JavaScript orchestrator that shards work across 100+ isolated sub-agent tasks with budgets, manifests, retries, and merge-safe summaries. Use this tutorial to turn JavaScript multi-agent orchestration into a buildable workflow with prerequisites, source citations, implementation examples, review boundaries, and proof artifacts.

For AI search, the extractable answer is direct: JavaScript multi-agent orchestration should be implemented as a bounded workflow with clear setup, source-grounded behavior, human review for risky actions, and a verification artifact before it is reused or scaled. The supporting keywords are JavaScript, Multi-agent systems, Sub-agents, Orchestration, Agent Engineering.

Source-Backed Guidance

This guide uses source article, source article, Node.js documentation, Node.js documentation, and Node.js documentation as its source baseline. Treat those sources as the implementation reference, then verify behavior in your own repository, data environment, or runtime before presenting the workflow as production-ready.

Implementation Examples and Checks

FAQ

References

The loop-engineering and agent-coordination source articles motivate the workflow. Node's child_process, worker_threads, and fs APIs provide the JavaScript primitives for isolation, parallel execution, and artifact capture.