Skip to main content
The workflow engine provides automatic retries, checkpointing for recovery, and patterns for ensuring exactly-once execution of side effects.

​
Retries

​
Step-level retries

Service steps retry automatically on failure with exponential backoff: 
{
  service: "flaky-api",
  operation: "fetchData",
  params: { ... },
  outputTable: "results",
  outputSchema: { ... },
  maxRetries: 5,
  retryDelayMs: 2000,
}
FieldDefaultDescription
maxRetries3Number of retry attempts
retryDelayMs1000Base delay in milliseconds (doubles on each retry)

​
forEach item retries

When using forEach, retries apply per item: 
forEach: {
  source: "records",
  itemKey: "id",
  maxRetries: 3,
  retryDelayMs: 1000,
  onItemError: "skip",
}
With onItemError: "skip", failed items are logged and skipped after exhausting retries. With "fail" (default), the first item failure after retries stops the entire step.

​
Pagination retries

Each page request can retry independently: 
paginate: {
  type: "cursor",
  cursorField: "nextCursor",
  cursorParam: "cursor",
  maxRetries: 5,
  retryDelayMs: 2000,
}

​
Checkpointing

The workflow engine checkpoints progress at multiple levels. If a worker crashes or the job is interrupted, it resumes from where it left off:
  • Step-level — completed steps are not re-executed
  • Page-level — paginated steps resume after the last fully completed page
  • Item-level — forEach steps skip already-completed items
This means long-running workflows are resilient to transient failures — a paginated fetch through thousands of pages or a forEach over hundreds of items will pick up where it stopped, not start over.

​
Idempotency

Prevent duplicate active jobs for the same logical operation using an idempotency key: 
await client.submitJob({
  job: syncOrders,
  params: { orgId: "org_abc123" },
  idempotencyKey: "app--my-app:sync-orders",
});
If a job with the same idempotencyKey is already active (pending or running), the submission is rejected. This is especially useful for manual triggers and webhook-initiated workflows.

​
Exactly-once side effects

For operations with real-world consequences (payments, notifications), ensure they execute exactly once even across retries.

​
Native idempotency keys

If the target API supports idempotency keys, pass a deterministic key derived from the data: 
{
  service: "stripe",
  operation: "createCharge",
  params: {
    amount: "{{item.amount}}",
    currency: "usd",
    idempotency_key: "charge-{{item.order_id}}",
  },
  forEach: {
    source: "pending_charges",
    itemKey: "order_id",
  },
  outputTable: "charges",
  outputSchema: { ... },
}

​
preExecute checks

For APIs without native idempotency, use preExecute to check whether the operation has already been performed: 
{
  service: "legacy_payment",
  operation: "create_charge",
  params: {
    order_id: "{{item.order_id}}",
    amount: "{{item.amount}}",
  },
  preExecute: {
    service: "legacy_payment",
    operation: "list_charges",
    params: { order_id: "{{item.order_id}}" },
    condition: { path: "results.length", gt: 0 },
    resultMapping: {
      charge_id: "results[0].id",
      recovered: true,
    },
  },
  forEach: {
    source: "pending_orders",
    itemKey: "order_id",
  },
  outputTable: "charges",
  outputSchema: { ... },
}
If preExecute finds an existing result matching the condition, the main operation is skipped and the mapped values are used as the step output instead.