Choreography Patterns

Common animation patterns for agents composing scenes via MCP. Each pattern shows the tool calls needed and when to use them.

Pattern 1: Arrive → Work → Leave

The most common pattern. An entity walks to a destination, does something, then returns.

create_choreography:
  on: "tool_call"
  defaultTargetEntityId: "agent"
  steps:
    - action: "setAnimation", params: { state: "walk" }
    - action: "move", target: "workstation", duration: 1200
    - action: "onArrive", steps:
        - action: "setAnimation", params: { state: "work" }
        - action: "flash", entity: "workstation", params: { color: "#E8A851" }
        - action: "wait", duration: 2000
        - action: "setAnimation", params: { state: "walk" }
        - action: "move", target: "idle-agent", duration: 1200
        - action: "onArrive", steps:
            - action: "setAnimation", params: { state: "idle" }

When: Agent processes a tool call, task, or request. The movement visualizes the work happening.

Pattern 2: Flash on event

Instant visual feedback without movement. Highlight an entity when an event occurs.

create_choreography:
  on: "tool_result"
  steps:
    - action: "flash", entity: "indicator-status", params: { color: "#4ade80", intensity: 0.8 }

When: Quick acknowledgment — a result arrived, a state changed, a condition was met. Combine with set_entity_state for a state change + flash.

Pattern 3: State toggle

Switch an entity between two states based on signal payload.

# Choreography A: open on task_dispatch
create_choreography:
  on: "task_dispatch"
  steps:
    - action: "setAnimation", entity: "door-main", params: { state: "open" }
    - action: "flash", entity: "door-main", params: { color: "#E8A851" }

# Choreography B: close on completion
create_choreography:
  on: "completion"
  steps:
    - action: "setAnimation", entity: "door-main", params: { state: "closed" }

When: Binary states — doors (open/closed), machines (on/off), indicators (active/inactive). Use two choreographies with different trigger signals, or one choreography with a when filter on payload values.

Pattern 4: Follow route

Move an entity along a predefined path.

create_choreography:
  on: "agent_state_change"
  when: { field: "state", operator: "eq", value: "thinking" }
  steps:
    - action: "setAnimation", entity: "agent", params: { state: "walk" }
    - action: "followRoute", entity: "agent", target: "patrol-loop", duration: 5000
    - action: "onArrive", steps:
        - action: "setAnimation", entity: "agent", params: { state: "idle" }

When: Patrolling, wandering, complex movement that isn't a straight line. Routes are designed in the scene-builder with smooth/sharp corners.

Pattern 5: Parallel coordination

Multiple things happen simultaneously — a common need when several actors react to the same event.

create_choreography:
  on: "task_dispatch"
  steps:
    - action: "parallel", steps:
        - action: "setAnimation", entity: "agent", params: { state: "walk" }
        - action: "move", entity: "agent", target: "forge", duration: 1500
        - action: "flash", entity: "forge", params: { color: "#E8A851" }
        - action: "setAnimation", entity: "worker-forge", params: { state: "prepare" }

When: Multiple actors need to react simultaneously. The parallel action runs all child steps at once instead of sequentially.

Pattern 6: Spawn → Animate → Destroy

Temporary entities that appear, do something, then vanish.

create_choreography:
  on: "tool_result"
  steps:
    - action: "spawn", params: { entityId: "fx-spark", x: 500, y: 300, semanticId: "result-spark" }
    - action: "flash", entity: "result-spark", params: { color: "#4ade80" }
    - action: "wait", duration: 800
    - action: "destroy", entity: "result-spark"

When: Visual effects, projectiles, temporary indicators, notifications that appear and disappear. The entity exists only for the choreography's lifetime.

Pattern 7: Conditional choreography (when filter)

Trigger different choreographies based on signal payload.

# Only for Read tool calls
create_choreography:
  on: "tool_call"
  when: { field: "toolName", operator: "eq", value: "Read" }
  steps:
    - action: "move", entity: "agent", target: "library", duration: 1000
    - action: "onArrive", steps:
        - action: "setAnimation", entity: "agent", params: { state: "read" }

# Only for Write tool calls
create_choreography:
  on: "tool_call"
  when: { field: "toolName", operator: "eq", value: "Write" }
  steps:
    - action: "move", entity: "agent", target: "workstation", duration: 1000
    - action: "onArrive", steps:
        - action: "setAnimation", entity: "agent", params: { state: "write" }

When: Different behaviors for the same signal type, differentiated by payload content. Avoids a single bloated choreography with conditional logic.

Choreography vs direct update

Need	Use	Why
Animated movement over time	`create_choreography` with `move`/`fly`	Choreographies handle animation frames
Instant teleport or snap	`update_entity` with `x`, `y`	No animation needed
State change as part of a sequence	`setAnimation` step in choreography	Part of a larger narrative
State change in isolation	`set_entity_state`	Quickest path, no choreography overhead
One-time setup (positioning, scaling)	`update_entity`	Scene composition, not animation
Reactive animation (on signal)	`create_choreography` + `create_wire`	Signal-driven, repeatable
Binding (continuous property link)	`create_binding`	Choreography output drives entity property over time

Complete scene composition recipe

A typical agent composes a scene in this order:

1. Discover the scene

describe_scene → understand current state
list_entities(filter: "actors") → see existing actors
get_scene_state → positions, routes, layers

2. Place actors

place_entity("peon", x: 200, y: 400, semanticId: "agent")
place_entity("building-forge", x: 500, y: 350, semanticId: "forge")
place_entity("indicator", x: 50, y: 50, semanticId: "indicator-status")

3. Create choreographies

create_choreography(on: "tool_call", steps: [...arrive-work-leave...])
create_choreography(on: "tool_result", steps: [...flash indicator...])
create_choreography(on: "error", steps: [...flash red...])

4. Wire signals to choreographies

create_wire(fromZone: "signal-type", fromId: "tool_call", toZone: "choreographer", toId: choreId1)
create_wire(fromZone: "signal-type", fromId: "tool_result", toZone: "choreographer", toId: choreId2)
create_wire(fromZone: "signal-type", fromId: "error", toZone: "choreographer", toId: choreId3)

5. Test

emit_signal(type: "tool_call", payload: { toolName: "Read" })
→ watch the agent walk to the forge, work, flash, and return

6. Refine

get_entity(semanticId: "agent") → check current state
update_entity(semanticId: "agent", scale: 0.8) → adjust size
set_entity_state(semanticId: "forge", state: "active") → test state

Key files

packages/mcp-server/src/tools/create-choreography.ts — choreography creation with all step types
packages/mcp-server/src/tools/create-wire.ts — signal-to-choreography wiring
packages/mcp-server/src/tools/create-binding.ts — continuous property bindings
docs/guide/choreographer-pipeline.md — how the choreographer processes steps
docs/specs/choreographer-signal-flow.md — entity resolution and fan-out behavior

Choreography Patterns ​

Pattern 1: Arrive → Work → Leave ​

Pattern 2: Flash on event ​

Pattern 3: State toggle ​

Pattern 4: Follow route ​

Pattern 5: Parallel coordination ​

Pattern 6: Spawn → Animate → Destroy ​

Pattern 7: Conditional choreography (when filter) ​

Choreography vs direct update ​

Complete scene composition recipe ​

1. Discover the scene ​

2. Place actors ​

3. Create choreographies ​

4. Wire signals to choreographies ​

5. Test ​

6. Refine ​

Key files ​