Skip to content

Design Choices

Why LGOS works the way it does. Add a row for each significant decision; when one changes, update its row.

Choice Why Cost Revisit when
Background runs live only in Hatchet; LGOS keeps no Response table. One source of truth: no outbox, resubmission, cleanup, or migrations. Polling depends on Hatchet; retention is Hatchet's (30 days by default when self-hosted), not store; an unknown ID takes about 17 seconds to return 404. Responses must outlive Hatchet or its retention.
The background worker runs the foreground Responses path, with no retries or checkpoint resume. One execution path; resuming needed a second runner and recovery machinery. A run that raises fails. A run reassigned after its worker dies starts over, and an interrupt answer that had already applied then fails as stale. Runs are too long or expensive to repeat.
Background creates accept an Idempotency-Key header. SDKs and gateways retry creates, which would otherwise start duplicate runs. It is not an OpenAI header, and gateways must forward it. OpenAI offers a retry-safe create.
A node calls interrupt() at most once per invocation. LangGraph's interrupt IDs then identify each pause without scanning checkpoints. A node cannot ask two questions. LangGraph gives each interrupt() call a unique ID.
A request may replay answered interrupts as history: the lgos_interrupt calls, then their outputs. Only an input that ends with interrupt calls or outputs is a resume, which requires previous_response_id. It is OpenAI's stateless function-calling transcript, so clients replay interrupts like any other call, and a graph sees what it asked and what the user answered, as chat apps keep their question tools in context. A stateless client cannot resume a pause by replaying it, as it would continue an ordinary function call: the paused checkpoint needs previous_response_id. Graph models receive calls to a tool they do not declare, which OpenAI's Responses and Chat Completions APIs accept as history. LangGraph can resume a paused run from a replayed transcript.
Paused interrupt runs expire through delete_expired_interrupt_runs, which the host schedules. The open-source Postgres checkpointer has no TTL; this mirrors LangGraph Agent Server's checkpointer TTL through the checkpointer API alone, so it works with any saver and needs no optional dependency such as Hatchet. Each sweep reads every checkpoint in the saver, and an answer arriving after expiry fails with 409. LangGraph checkpointers expire threads natively.
LGOS reports each graph execution through the OpenTelemetry API as a GenAI invoke_workflow span and gen_ai.invoke_workflow.duration histogram. Agent frameworks such as Google ADK, Microsoft Agent Framework, and Pydantic AI instrument their own runs through the API, and the GenAI conventions name LangGraph graphs as workflows. A core dependency on opentelemetry-api. The GenAI conventions are still in development and may rename these signals. LangGraph ships its own OpenTelemetry instrumentation, or the GenAI conventions stabilize.
A graph run that ends before its output exists fails, cancellations included: the workflow span is ERROR and both signals carry the cancellation's error.type. The conventions define no cancelled outcome, and widely used tracers record cancellation as a failure: OpenTelemetry's GenAI utilities, Logfire, LangSmith, ASP.NET Core, .NET's HttpClient, and Go's HTTP client. A stream closed mid-run fails too, so a client disconnect fails whether it arrives while the graph works or while it waits on the client. Cancellations count as failures in error rates and alerts; the error.type value, such as asyncio.exceptions.CancelledError or GeneratorExit, tells them apart. This departs from the HTTP conventions, which leave a cancellation the caller intended unset; from the official LangGraph server (langgraph-api 0.15.1 worker.py), which counts a requested cancellation apart from failures; and from OpenTelemetry Python's use_span, which leaves cancellations unset. The conventions add a cancelled outcome (semantic-conventions#560).
Graph steps are traced only in Langfuse, which records through a tracer provider of its own; OpenTelemetry exporters receive the workflow span and the calls a graph makes. The graph runs under the workflow span's context, not the span itself. Both share each trace ID. Langfuse observations carry prompts and outputs, and Langfuse joins the global provider by default, so every exporter received them and the trace backend's content changed with the Langfuse toggle; Langfuse recommends a provider of its own beside another backend. Its handler also writes trace attributes onto whatever span is current, so the workflow span would carry Langfuse's attributes. OpenTelemetry has no official LangChain instrumentation for graph steps. Trace backends show no graph steps; the shared trace ID links each run to its Langfuse trace. Graph code cannot add attributes to the workflow span, only to spans of its own. An application that creates its own Langfuse client first chooses its own provider. OpenTelemetry ships GenAI instrumentation for LangChain or LangGraph, or Langfuse stops writing onto spans it does not own.
lgos serve and lgos worker host a registry factory with logging, CORS, telemetry, persistence, interrupt expiry, and the Hatchet worker. lgos serve is Uvicorn's command with the registry in place of the app. Projects keep only their graphs, inside their own image and lockfile, as with Aegra. Reusing Uvicorn's command passes every option and UVICORN_* variable through unchanged. Routes, middleware, and authentication extend create_app with plain FastAPI. A heavy extra with opinionated defaults; options follow Uvicorn's releases; extended apps bring their own server command and logging. Each process, Uvicorn workers included, runs its own pool and expiry sweep unless the interval is 0. Uvicorn stops exposing its command, or langgraph.json becomes worth a second registration format.
The starter template is a Cookiecutter template generated from main; there is no lgos init. Cookiecutter already prompts, validates, and checks out releases, and template fixes need no LGOS release. Bundling the template for an lgos init would ship it, with .py files invalid before rendering, in every install. Generation needs git; main may briefly expect an unreleased LGOS (--checkout <tag> picks a release); generated projects get no template updates. The template must render from the installed LGOS version.
Model listing advertises tool names, not definitions: lgos.server_tools for tools a graph executes and lgos.client_tools for client-owned functions it can call. Clients reach graphs through gateways that copy this metadata, so they offer exactly the tools each graph accepts without hard-coding graph names, and still build OpenAI's native tool selectors without a tool discovery protocol. Clients label tools from their names. An advertised client tool works only in clients that implement that name, and LGOS does not enforce client_tools. OpenAI's model object describes the tools a model accepts.