relay

Documentation

Workflows

Compose multiple agents. Two primitives: subagent() for LLM-driven composition, Graph for declarative pipelines.

Three primitives, on purpose

Relay gives you three layers to build multi-agent systems. Each one solves a different problem; pick the lowest-level that fits.

  • subagent() wraps an agent as a tool that another agent can call. The parent LLM decides when and how to delegate. Best when the flow is open-ended — "research this if you need to."
  • createOrchestrator() higher-level wrapper for the "supervisor + team" pattern. You hand it a team of named agents and one LLM coordinates them — auto-generates the system prompt from the team description. 80% of multi-agent use cases in 3 lines.
  • Graph wires steps with explicit edges and state passing. The developer decides the flow. Best when the steps are deterministic — "always research, then write, then review."

All three produce the same artifact server-side: a tree of runs linked by workflow_id. The dashboard renders the tree and the cost endpoint sums tokens across the whole workflow.

subagent() — LLM-driven composition

a writer agent calls a researcher and a critic as tools
rendering diagram…

Wrap any Agent with subagent({name, description, agent}) and pass it into another agent's tools. The LLM sees it as a normal function tool; the SDK runs the sub-agent linked to the parent run automatically.

import { createAgent, subagent, builtin } from "@relayhq/sdk";

const researcher = createAgent({
  model: "gpt-4o",
  system: "You research topics and return key facts.",
  tools: [builtin.calculator],
});

const writer = createAgent({
  model: "claude-sonnet-4-6",
  system: "You write concise posts grounded in the research you commission.",
  tools: [
    subagent({
      name: "research",
      description: "Research a topic and return key facts.",
      agent: researcher,
    }),
  ],
});

for await (const ev of writer.run("Write a 200-word post about pgvector")) {
  if (ev.type === "token") process.stdout.write(ev.text);
}

Python

from relayhq import create_agent, subagent, builtin

researcher = create_agent(model="gpt-4o", tools=[builtin.calculator])

writer = create_agent(
    model="claude-sonnet-4-6",
    tools=[
        subagent(
            name="research",
            description="Research a topic",
            agent=researcher,
        ),
    ],
)

Safety rails

  • Depth is capped at 5 by default (set maxDepth per subagent). Beyond that the handler returns an error to the parent LLM.
  • Input is validated against the JSON schema before invocation — an LLM that hallucinates args gets a structured error back instead of crashing the handler.
  • Each sub-run gets its own row in runs with parent_run_id set, so the cost and trace are attributable.

createOrchestrator() — supervisor + team

The fastest path to a multi-agent system. You declare a team of named specialists; one supervisor LLM decides which to call for each part of the user's request, calls them as tools, and synthesizes the final answer.

Under the hood, this is createAgent() with each teammate wrapped in subagent(), plus an auto-generated system prompt built from the team descriptions. You can do the same by hand — but you'll write the same boilerplate every time.

import { createAgent, createOrchestrator } from "@relayhq/sdk";

const researcher = createAgent({ model: "gpt-4o" });
const writer     = createAgent({ model: "claude-sonnet-4-6" });
const reviewer   = createAgent({ model: "claude-haiku-4-5" });

const team = createOrchestrator({
  model: "claude-sonnet-4-6",  // the supervisor's brain
  agents: {
    research: {
      agent: researcher,
      description: "Researches topics and returns key facts.",
    },
    write: {
      agent: writer,
      description: "Writes drafts from research notes.",
    },
    review: {
      agent: reviewer,
      description: "Reviews drafts; returns 'approved' or feedback.",
    },
  },
});

// Use it like any other agent
for await (const ev of team.run("Write a 200-word post about pgvector")) {
  if (ev.type === "token") process.stdout.write(ev.text);
}

Python

from relayhq import create_agent, create_orchestrator

researcher = create_agent(model="gpt-4o")
writer     = create_agent(model="claude-sonnet-4-6")
reviewer   = create_agent(model="claude-haiku-4-5")

team = create_orchestrator(
    model="claude-sonnet-4-6",
    agents={
        "research": {
            "agent": researcher,
            "description": "Researches topics and returns key facts.",
        },
        "write": {
            "agent": writer,
            "description": "Writes drafts from research notes.",
        },
        "review": {
            "agent": reviewer,
            "description": "Reviews drafts; returns approved or feedback.",
        },
    },
)

async for ev in team.run("Write a 200-word post about pgvector"):
    if ev.get("type") == "token":
        print(ev["text"], end="", flush=True)

Extending the supervisor

Two escape hatches when the auto-generated prompt isn't enough:

  • system: "..." — appended after the team description. Use for output format rules, safety constraints, tone.
  • extraTools: [...] — extra tools the supervisor can call directly, alongside teammates. Example: give the supervisor builtin.calculator so it can do simple math without delegating.
const team = createOrchestrator({
  model: "claude-sonnet-4-6",
  agents: { ... },
  system:
    "Always return the final answer as JSON: { summary, sources, confidence }.\n" +
    "If teammates disagree, set confidence to 'low' and list both views.",
  extraTools: [builtin.calculator, getCurrentTime],
});

When NOT to use the orchestrator

If you find yourself doing any of these, drop down to the lower-level primitives:

  • Need a strict sequence (always research → write → review, no skipping) → use Graph. The LLM will deviate eventually.
  • Need to pass typed state between agents → use Graph. The orchestrator passes a string prompt; not a struct.
  • Need parallel fan-out → use Graph with Promise.all inside a step. The supervisor calls tools sequentially by default.
  • You have one specialist agent → just use that agent directly. The supervisor layer is overhead.

Graph — declarative pipelines

research → write → review, with a feedback loop
rendering diagram…

When the flow is deterministic, the Graph API gives you nodes, edges, and state in a couple of chained calls. Every step becomes a Relay run linked under the same workflow.

import { createAgent, Graph, END } from "@relayhq/sdk";

const researcher = createAgent({ model: "gpt-4o" });
const writer     = createAgent({ model: "claude-sonnet-4-6" });
const reviewer   = createAgent({ model: "claude-haiku-4-5" });

type State = {
  topic: string;
  research?: string;
  draft?: string;
  verdict?: string;
};

const graph = new Graph<State>()
  .agent("research", researcher, { inputFrom: "topic",    outputTo: "research" })
  .agent("write",    writer,     { inputFrom: "research", outputTo: "draft" })
  .agent("review",   reviewer,   { inputFrom: "draft",    outputTo: "verdict" })
  .edge("research", "write")
  .edge("write", "review")
  .conditional("review", (state) =>
    state.verdict?.toLowerCase().includes("approved") ? END : "research",
  )
  .start("research");

const { state, path, workflowId } = await graph.run({ topic: "AI agents" });

console.log("path:",    path);        // ["research", "write", "review", ...]
console.log("draft:",   state.draft);
console.log("workflow:", workflowId);  // join all runs in the dashboard

The primitives

  • .step(name, fn) — a raw step. fn(state, ctx) returns a partial state that gets merged in.
  • .agent(name, agent, {inputFrom, outputTo}) — wrap a Relay agent as a step. Reads input from a state field, writes the final text to another state field.
  • .edge(from, to) — static transition. Pass END to terminate.
  • .conditional(from, fn) — dynamic transition. fn(state) returns the next step name (or END).
  • .start(name) — entry point. Defaults to the first step you registered.
  • .describe() — dump the graph shape. Used by the dashboard to render the flow before any run happens.

Safety: cycles and bounds

Cycles are allowed — feedback loops are useful — but every run() caps total step executions at maxSteps (default 30) and throws if exceeded. Override per call when you genuinely need more.

Python

from relayhq import create_agent, Graph, END

graph = (
    Graph()
      .agent("research", researcher, input_from="topic", output_to="research")
      .agent("write",    writer,     input_from="research", output_to="draft")
      .agent("review",   reviewer,   input_from="draft",    output_to="verdict")
      .edge("research", "write")
      .edge("write", "review")
      .conditional("review", lambda s: END if "approved" in s["verdict"].lower() else "research")
      .start("research")
)

result = await graph.run({"topic": "AI agents"})
print(result.path, result.workflow_id)

When to use which

  • subagent(): open-ended delegation, parent decides at runtime which sub-agent to call. Example: a support agent that calls a "refund_specialist" only when the user actually asks for a refund.
  • Graph: deterministic pipeline with state. Example: ETL-style "ingest → enrich → summarize → publish" where every record goes through the same steps.
  • Both, in the same workflow: a Graph step can itself be an agent with subagent() tools. They compose freely.

Inspecting a workflow

Every run in a workflow shares one workflow_id and links via parent_run_id. Two endpoints surface this:

# Tree of runs (depth-first, indented for rendering)
curl -H "Authorization: Bearer $RELAY_API_KEY" \
     https://api.relaygh.dev/v1/workflows/<workflow_id>

# → { runs: [...], cost: { runCount, inputTokens, outputTokens } }

# Only top-level runs (one row per workflow in your dashboard)
curl -H "Authorization: Bearer $RELAY_API_KEY" \
     "https://api.relaygh.dev/v1/runs?roots=true"

Recipes — patterns you'll actually build

The Graph and subagent primitives compose into the usual workflow shapes. Each recipe below is a complete, copy-pasteable example.

Parallel fan-out + aggregate

Run N agents on the same input concurrently, then aggregate the results. The Graph runner doesn't have built-in parallelism (steps are sequential), so we use the SDK directly inside a single step.

import { createAgent, Graph, END, collectFinalOutput } from "@relayhq/sdk";

const researcher = createAgent({ model: "gpt-4o" });
const critic     = createAgent({ model: "claude-sonnet-4-6" });
const summarizer = createAgent({ model: "claude-haiku-4-5" });

type State = { topic: string; research?: string; critique?: string; summary?: string };

const graph = new Graph<State>()
  // ─── Fan out: research + critique run in parallel ──────────────────
  .step("gather", async (state, ctx) => {
    const [research, critique] = await Promise.all([
      collectFinalOutput(
        researcher.run(`Research: ${state.topic}`, {
          workflowId: ctx.workflowId,
        }),
      ),
      collectFinalOutput(
        critic.run(`Identify counter-arguments to: ${state.topic}`, {
          workflowId: ctx.workflowId,
        }),
      ),
    ]);
    return { research, critique };
  })
  // ─── Reduce: summarize both into one output ────────────────────────
  .agent("summarize", summarizer, { inputFrom: "research", outputTo: "summary" })
  .edge("gather", "summarize")
  .edge("summarize", END);

const result = await graph.run({ topic: "Whether to adopt CRDTs" });

Retry loop with bounded attempts

Useful when an LLM step might produce invalid output (e.g. structured JSON that needs to validate against a schema, or code that needs to compile).

import { Graph, END } from "@relayhq/sdk";

type State = {
  prompt: string;
  attempt: number;
  output?: string;
  valid?: boolean;
  feedback?: string;
};

const graph = new Graph<State>()
  .agent("generate", coder, { inputFrom: "prompt", outputTo: "output" })
  .step("validate", async (state) => {
    const result = await runTests(state.output!);
    return {
      valid: result.ok,
      feedback: result.ok ? undefined : result.errors,
      attempt: (state.attempt ?? 0) + 1,
      // Feed the failure back into the prompt for the next attempt
      prompt: result.ok
        ? state.prompt
        : `${state.prompt}\n\nPrevious attempt failed: ${result.errors}\nFix and retry.`,
    };
  })
  .edge("generate", "validate")
  .conditional("validate", (state) => {
    if (state.valid) return END;
    if (state.attempt >= 3) return END; // give up after 3
    return "generate";
  })
  .start("generate");

const result = await graph.run({ prompt: "Write a binary search in TS", attempt: 0 });

Map-reduce over a list

Process N items individually with the same agent, then aggregate. Useful for batch summaries, per-document analysis, parallel extraction.

import asyncio
from relayhq import Graph, END, collect_final_output, create_agent

extractor = create_agent(model="gpt-4o-mini", system="Extract key facts as JSON.")
synthesizer = create_agent(model="claude-sonnet-4-6", system="Synthesize a coherent summary.")

graph = (
    Graph()
      # MAP: extract facts from each doc in parallel
      .step("map", lambda state, ctx: asyncio.gather(*[
          collect_final_output(
              extractor.run(doc, workflow_id=ctx.workflow_id or None)
          )
          for doc in state["docs"]
      ]).__await__() and {"facts": "..."})  # see TypeScript for cleaner version
      # REDUCE: synthesize all facts into one summary
      .agent("reduce", synthesizer, input_from="facts", output_to="summary")
      .edge("map", "reduce")
      .edge("reduce", END)
)

# In TypeScript the same pattern reads more naturally:
# .step("map", async (state, ctx) => ({
#   facts: (await Promise.all(state.docs.map(d =>
#     collectFinalOutput(extractor.run(d, { workflowId: ctx.workflowId }))
#   ))).join("\n"),
# }))

Branching by classification

First agent classifies the input; downstream agents handle each class. The conditional edge does the routing — no manual if/else chains needed.

import { createAgent, Graph, END } from "@relayhq/sdk";

const classifier = createAgent({
  model: "gpt-4o-mini",
  system: 'Reply with one word: "refund", "shipping", "technical", or "other".',
});
const refundAgent     = createAgent({ model: "claude-sonnet-4-6", system: "..." });
const shippingAgent   = createAgent({ model: "gpt-4o-mini",       system: "..." });
const technicalAgent  = createAgent({ model: "claude-sonnet-4-6", system: "..." });
const escalationAgent = createAgent({ model: "claude-sonnet-4-6", system: "..." });

type State = { ticket: string; category?: string; response?: string };

const graph = new Graph<State>()
  .agent("classify", classifier, { inputFrom: "ticket", outputTo: "category" })
  .agent("refund",     refundAgent,     { inputFrom: "ticket", outputTo: "response" })
  .agent("shipping",   shippingAgent,   { inputFrom: "ticket", outputTo: "response" })
  .agent("technical",  technicalAgent,  { inputFrom: "ticket", outputTo: "response" })
  .agent("escalate",   escalationAgent, { inputFrom: "ticket", outputTo: "response" })
  .conditional("classify", (state) => {
    const cat = state.category?.toLowerCase().trim() ?? "";
    if (cat.includes("refund"))    return "refund";
    if (cat.includes("shipping"))  return "shipping";
    if (cat.includes("technical")) return "technical";
    return "escalate";
  })
  .edge("refund",    END)
  .edge("shipping",  END)
  .edge("technical", END)
  .edge("escalate",  END)
  .start("classify");

const { state } = await graph.run({ ticket: "I never got my package" });

Multi-agent debate (until consensus or rounds)

Two agents argue opposite sides, a third judges. Cycles back if no consensus; bounded by total rounds.

import { createAgent, Graph, END } from "@relayhq/sdk";

const proposer = createAgent({ model: "claude-sonnet-4-6", system: "Argue FOR the proposal." });
const opposer  = createAgent({ model: "claude-sonnet-4-6", system: "Argue AGAINST the proposal." });
const judge    = createAgent({
  model: "gpt-4o",
  system: 'After reading both sides, output exactly: "CONSENSUS: <verdict>" or "MORE_DEBATE_NEEDED".',
});

type State = {
  proposal: string;
  transcript: string[];
  round: number;
  verdict?: string;
};

const graph = new Graph<State>()
  .step("pro", async (state, ctx) => {
    const out = await collectFinalOutput(
      proposer.run(
        `Proposal: ${state.proposal}\n\nTranscript so far:\n${state.transcript.join("\n")}`,
        { workflowId: ctx.workflowId },
      ),
    );
    return { transcript: [...state.transcript, `PRO: ${out}`] };
  })
  .step("con", async (state, ctx) => {
    const out = await collectFinalOutput(
      opposer.run(
        `Proposal: ${state.proposal}\n\nTranscript so far:\n${state.transcript.join("\n")}`,
        { workflowId: ctx.workflowId },
      ),
    );
    return {
      transcript: [...state.transcript, `CON: ${out}`],
      round: state.round + 1,
    };
  })
  .step("judge", async (state, ctx) => {
    const out = await collectFinalOutput(
      judge.run(state.transcript.join("\n"), { workflowId: ctx.workflowId }),
    );
    return { verdict: out };
  })
  .edge("pro", "con")
  .edge("con", "judge")
  .conditional("judge", (state) => {
    if (state.verdict?.startsWith("CONSENSUS:")) return END;
    if (state.round >= 5) return END; // hard cap
    return "pro"; // loop back
  })
  .start("pro");

const { state, path } = await graph.run({
  proposal: "Migrate from REST to GraphQL",
  transcript: [],
  round: 0,
});
console.log(`Verdict after ${path.length} steps:`, state.verdict);

Human-in-the-loop checkpoint

Workflow runs up to a step that requires human approval, persists state, exits. Resumed later via a separate run that picks up where it left off.

The simplest version: the "wait for approval" step polls your own DB / queue for the human decision before continuing. (Relay doesn't persist workflow state itself — that's intentional, keeps the runner stateless and composable.)

import { Graph, END } from "@relayhq/sdk";

const graph = new Graph()
  .agent("draft", drafter, { inputFrom: "brief", outputTo: "draft" })
  .step("await_approval", async (state) => {
    // Persist draft and an approval token to your own DB
    const token = await approvals.create({ draft: state.draft });

    // Notify a human (email, Slack, etc.)
    await slack.send(`Approval needed: https://your-app/approvals/${token}`);

    // Block until approved or rejected. In a serverless world, end the
    // workflow here and resume from a webhook in a follow-up run. In a
    // long-lived worker, poll:
    const decision = await approvals.waitForDecision(token, { timeoutMs: 3600_000 });
    return { approved: decision.approved, feedback: decision.feedback };
  })
  .agent("publish",   publisher,    { inputFrom: "draft", outputTo: "published_url" })
  .agent("revise",    drafter,      { inputFrom: "feedback", outputTo: "draft" })
  .conditional("await_approval", (state) =>
    state.approved ? "publish" : "revise",
  )
  .edge("publish", END)
  .edge("revise",  "await_approval")
  .start("draft");

Picking the right primitive

The rule of thumb, in increasing order of structure:

  • Single agent + tools: the model decides everything dynamically. Use when the flow is open-ended.
  • subagent(): another agent is a tool. Use when the parent should sometimes delegate, but not always.
  • createOrchestrator(): a supervisor LLM coordinating a team of specialists. Use when you want multi-agent without writing the routing logic. The fastest path from "I have 3 agents" to "working multi-agent system".
  • Graph: deterministic pipeline with steps and state. Use when you can describe the flow as a flowchart without thinking about the LLM.
  • Graph with agent steps: pipeline structure with LLMs filling in specific roles. The most common production shape.