A3S Docs
A3S Code

Multi-Machine Distribution

Horizontal scaling with external task distribution across multiple workers

Multi-Machine Distribution

A3S Code supports external task distribution for horizontal scaling. Offload tool execution to external workers running on multiple machines, containers, or cloud regions.

Key Features: 3 execution modes | 4 priority lanes | Multi-language workers | Built-in timeout handling

Overview

External task distribution enables:

  • Horizontal scaling — Add workers to increase throughput
  • Resource isolation — Separate heavy computation from main process
  • Custom environments — Run tools in containers, VMs, or specialized hardware
  • Multi-region deployment — Execute tasks in different geographic regions
  • Language-agnostic workers — Implement workers in Rust, Python, TypeScript, or any language

Architecture

┌─────────────────────────────────────────────────────────────┐
│                        Agent Session                         │
│  ┌────────────────────────────────────────────────────────┐ │
│  │              Session Lane Queue (a3s-lane)             │ │
│  │  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌────────┐│ │
│  │  │ Control  │  │  Query   │  │ Execute  │  │Generate││ │
│  │  │   (P0)   │  │   (P1)   │  │   (P2)   │  │  (P3)  ││ │
│  │  └────┬─────┘  └────┬─────┘  └────┬─────┘  └───┬────┘│ │
│  └───────┼─────────────┼─────────────┼─────────────┼─────┘ │
└──────────┼─────────────┼─────────────┼─────────────┼───────┘
           │             │             │             │
           │      ┌──────┴──────┐      │             │
           │      │   External  │      │             │
           │      │   Workers   │      │             │
           │      └─────────────┘      │             │
           │             │             │             │
           ▼             ▼             ▼             ▼
    ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐
    │ Worker 1 │  │ Worker 2 │  │ Worker 3 │  │ Worker N │
    │ (Local)  │  │ (Remote) │  │(Container)│  │ (Cloud) │
    └──────────┘  └──────────┘  └──────────┘  └──────────┘

Task Handler Modes

A3S Code supports 3 task handler modes per lane:

Prop

Type

Session Lanes

Tasks are routed to 4 priority lanes:

Prop

Type

Query lane is the best candidate for external distribution — read operations are naturally parallelizable and safe.

Complete Example

Step 1: Configure Session

import { Agent } from '@a3s-lab/code';

const agent = await Agent.create('agent.hcl');

const session = agent.session('.', {
  queueConfig: {
    queryConcurrency: 20,
    executeConcurrency: 5,
    enableMetrics: true,
    enableDlq: true
  }
});

await session.setLaneHandler('query', {
  mode: 'external',
  timeoutMs: 120_000
});
from a3s_code import Agent, SessionQueueConfig

agent = Agent.create("agent.hcl")

# Configure queue
queue_config = SessionQueueConfig(
    query_concurrency=20,
    execute_concurrency=5,
    enable_metrics=True,
    enable_dlq=True
)

session = agent.session(".", queue_config=queue_config)

# Configure Query lane for external processing
session.set_lane_handler(
    "query",
    {"mode": "external", "timeout_ms": 120_000}
)

Step 2: Implement Worker

import { AgentSession, ExternalTask, ExternalTaskResult } from '@a3s-lab/code';
import { exec } from 'child_process';
import { promisify } from 'util';
import { readFile } from 'fs/promises';

const execAsync = promisify(exec);

async function workerLoop(session: AgentSession): Promise<void> {
  while (true) {
    // Poll for pending tasks
    const tasks = await session.pendingExternalTasks();

    if (tasks.length === 0) {
      await new Promise(resolve => setTimeout(resolve, 100));
      continue;
    }

    // Process tasks in parallel
    await Promise.all(tasks.map(task => processTask(session, task)));
  }
}

async function processTask(session: AgentSession, task: ExternalTask): Promise<void> {
  const result = await executeTask(task);
  await session.completeExternalTask(task.taskId, result);
}

async function executeTask(task: ExternalTask): Promise<ExternalTaskResult> {
  switch (task.commandType) {
    case 'read': {
      const path = task.payload.path;
      try {
        const content = await readFile(path, 'utf-8');
        return {
          success: true,
          result: { content },
          error: null
        };
      } catch (e) {
        return {
          success: false,
          result: {},
          error: e.message
        };
      }
    }

    case 'bash': {
      const command = task.payload.command;
      try {
        const { stdout, stderr } = await execAsync(command);
        return {
          success: true,
          result: { stdout, stderr, exitCode: 0 },
          error: null
        };
      } catch (e) {
        return {
          success: false,
          result: { stdout: e.stdout, stderr: e.stderr, exitCode: e.code },
          error: e.message
        };
      }
    }

    default:
      return {
        success: false,
        result: {},
        error: `Unknown command type: ${task.commandType}`
      };
  }
}
import asyncio
from a3s_code import AgentSession, ExternalTaskResult

async def worker_loop(session: AgentSession):
    """Worker loop - can run on multiple machines"""
    while True:
        # Poll for pending tasks
        tasks = await session.pending_external_tasks()

        if not tasks:
            await asyncio.sleep(0.1)
            continue

        # Process tasks in parallel
        await asyncio.gather(*[
            process_task(session, task) for task in tasks
        ])

async def process_task(session: AgentSession, task):
    """Process a single task"""
    result = await execute_task(task)
    await session.complete_external_task(task.task_id, result)

async def execute_task(task):
    """Execute task based on command type"""
    if task.command_type == "read":
        path = task.payload["path"]
        try:
            with open(path, 'r') as f:
                content = f.read()
            return ExternalTaskResult(
                success=True,
                result={"content": content},
                error=None
            )
        except Exception as e:
            return ExternalTaskResult(
                success=False,
                result={},
                error=str(e)
            )

    elif task.command_type == "bash":
        command = task.payload["command"]
        try:
            proc = await asyncio.create_subprocess_shell(
                command,
                stdout=asyncio.subprocess.PIPE,
                stderr=asyncio.subprocess.PIPE
            )
            stdout, stderr = await proc.communicate()
            return ExternalTaskResult(
                success=proc.returncode == 0,
                result={
                    "stdout": stdout.decode(),
                    "stderr": stderr.decode(),
                    "exit_code": proc.returncode
                },
                error=None
            )
        except Exception as e:
            return ExternalTaskResult(
                success=False,
                result={},
                error=str(e)
            )

    else:
        return ExternalTaskResult(
            success=False,
            result={},
            error=f"Unknown command type: {task.command_type}"
        )

Step 3: Deploy Workers

# Worker 1 (local machine)
./worker --session-id abc123

# Worker 2 (remote server)
ssh user@remote "cd /app && ./worker --session-id abc123"

# Worker 3 (container)
docker run -e SESSION_ID=abc123 myapp/worker

# Worker 4 (cloud)
kubectl run worker --image=myapp/worker --env="SESSION_ID=abc123"

Use Cases

2. Distributed Test Execution

3. Multi-Region File Processing

4. Hybrid Mode for Monitoring

Best Practices

Choose the Right Lane

  • Query lane — Best for parallelizable read operations (grep, read, glob, ls)
  • Execute lane — For write operations and bash commands
  • Control lane — Keep internal for control operations
  • Generate lane — Keep internal for LLM calls (unless you have custom LLM infrastructure)

Set Appropriate Timeouts

Handle Timeouts Gracefully

Monitor Queue Metrics

Implement Retry Logic

Performance Considerations

Concurrency Limits

Worker Scaling

  • Start with 1 worker per machine
  • Monitor queue depth and task latency
  • Add workers if queue depth grows
  • Each worker can process multiple tasks concurrently

Network Latency

  • External mode adds network round-trip latency
  • Use for tasks that take > 100ms to execute
  • For very fast operations (< 10ms), internal mode is faster

Monitoring and Debugging

Enable Metrics

Check Queue Stats

Monitor Task Latency

Summary

A3S Code's external task distribution enables:

  • ✅ Multi-machine parallelization — Distribute tasks across multiple workers
  • ✅ Flexible execution modes — Internal, External, or Hybrid per lane
  • ✅ Priority-based scheduling — 4 lanes with configurable priorities
  • ✅ Built-in timeout handling — Automatic timeout detection and cleanup
  • ✅ Metrics and monitoring — Track queue depth, latency, success rate
  • ✅ Language-agnostic workers — Implement workers in Rust, Python, TypeScript, or any language

This makes A3S Code suitable for:

  • Large-scale code analysis across multiple repositories
  • Distributed test execution
  • Multi-region deployments
  • Custom execution environments (containers, VMs, specialized hardware)
  • Horizontal scaling of tool execution
  • Architecture — System design and components
  • Lane Queue — Priority routing and task distribution
  • Tasks — Planning and parallel execution

API Reference

LaneHandlerConfig

Prop

Type

ExternalTask fields

Prop

Type

ExternalTaskResult fields

Prop

Type

On this page