Skip to content

Architecture

LangGraph OpenAI Serve mounts an OpenAI-compatible FastAPI sub-application on a host FastAPI app. It adapts OpenAI requests to application-owned LangGraph graphs and adapts their output back to OpenAI responses.

Request Path

flowchart LR
  client["OpenAI client<br/>owns conversation history"]
  host["Host FastAPI application"]

  subgraph lgos["langgraph-openai-serve"]
    direction TB
    api["Mounted /v1 API<br/>validate request"]
    config["GraphRegistry + GraphConfig<br/>resolve and adapt"]
    runner["Runner<br/>collect or stream events"]
    render["OpenAI response renderer<br/>Response or completion / SSE"]

    api --> config --> runner --> render
  end

  subgraph application["Application-owned code"]
    direction TB
    app_graph["Registered LangGraph graph"]
    dependencies["Models, tools, and data sources"]
    app_graph -->|"application calls"| dependencies
  end

  client -->|"OpenAI request"| api
  host -.->|"mounts"| api
  runner <-->|"graph.astream"| app_graph
  render -->|"OpenAI response"| client

The package owns the /v1 transport and adaptation boundary. The host application owns graph behavior and every model, tool, store, or data source used by that graph.

Polling-only background requests branch after validation. The API submits the validated request to a background engine and returns a queued Response. An independently deployed worker runs the same foreground Responses path, and the engine stores the resulting Response. Polling reads the engine; it never executes the graph in an HTTP request.

flowchart TB
  client["OpenAI client"]
  api["LGOS API"]
  engine["Background engine (Hatchet)"]
  worker["LGOS worker"]
  app_graph["Application graph"]

  client -->|"create, retrieve, cancel"| api
  api -->|"submit, read, cancel run"| engine
  engine -->|"deliver job"| worker
  worker -->|"foreground Responses path"| app_graph
  worker -->|"Response JSON"| engine

State Ownership

Ordinary conversations are stateless from LGOS's perspective: the client owns the transcript and sends the messages needed by each request. Optional graph features add narrowly scoped state without turning LGOS into a chat database.

flowchart LR
  client["UI or OpenAI client"]
  transcript[("Client-owned transcript")]

  subgraph runtime["Application runtime"]
    lgos["LGOS request handling"]
    app_graph["Application graph"]
  end

  checkpointer[("Async checkpointer<br/>paused execution")]
  coordinator["Run coordinator<br/>temporary lease"]
  store[("LangGraph Store<br/>explicit application data")]

  client -->|"resends input items"| lgos
  client -.->|"persists"| transcript
  lgos -->|"invokes"| app_graph
  lgos -.->|"interrupt resume and cleanup"| checkpointer
  lgos -.->|"serialize one interrupt run"| coordinator
  app_graph -.->|"save and load paused state"| checkpointer
  app_graph -.->|"read and write graph data"| store

Components

LanggraphOpenaiServe is the boundary between your FastAPI app and the OpenAI-compatible sub-application. It mounts the sub-application at the configured prefix. The host application owns middleware such as CORS, authentication, and telemetry. lgos serve is a ready-made host: it configures CORS, logging, telemetry, and persistence from LGOS_* settings, takes HTTP server options from Uvicorn, and leaves authentication to a gateway or to middleware added to its application.

The mounted OpenAI app owns the public HTTP surface: model listing, Responses, Chat Completions, health checks, request validation, response schemas, and OpenAI-shaped error handling.

GraphRegistry maps each OpenAI model value to a GraphConfig. GraphConfig then resolves the graph, applies custom input/context/output adapters when present, and tells the runner which optional GraphFeature values are enabled.

The runner is the only layer that calls LangGraph. It executes the prepared run and returns graph output or stream events for OpenAI response rendering.

Interrupt-enabled graphs add a narrow durable boundary: an asynchronous checkpointer stores paused workflow state. A run coordinator serializes one interrupt run across replicas. Application graphs may independently use a LangGraph Store for explicit data. PostgreSQL can provide all three roles; Redis is not required by this design. The demo shares one PostgreSQL pool per API or worker process among them.

Background Responses add no LGOS state: the background engine stores each run's request, status, and Response.

The Responses adapter owns interrupt function-call encoding and decodes previous_response_id plus function_call_output items into a resume request. Under the coordinator lease, preparation either validates that request into a native LangGraph Command(resume=...) or builds fresh graph input. Streaming and non-streaming requests share this prepared state, one runner, and the same interrupt validation. LangGraph checkpoints remain the source of truth for paused execution.

Chat Completions rejects interrupt-enabled models before preparing a graph run. A graph that calls interrupt() without declaring GraphFeature.INTERRUPTS fails with a configuration error on either API.

Endpoint paths and settings live in Reference.

Request Flow

  1. FastAPI validates the Responses or Chat Completions request, normalizes it, and resolves its model through GraphRegistry.
  2. GraphConfig converts messages and builds graph input, runtime context, and runnable configuration.
  3. For an interrupt graph, preparation derives the scoped operation key, acquires its coordinator lease, and validates any resume against durable state.
  4. The runner consumes graph.astream. A complete response requests only root values; a stream also forwards message and custom events to the SSE service.
  5. After execution quiesces, pending interrupts become one durable OpenAI tool-call batch; terminal or unsurfaced failed runs delete their checkpoint.
  6. LGOS releases any interrupt lease and renders a protocol-specific OpenAI object or SSE sequence.

For background creation, steps 2–6 happen in a worker after the API returns a queued Response. See Run Responses In The Background.

The paused Response ID and interrupt function-call items are client-owned. A UI that supports reconnectable interrupt input persists both before asking for a result; the backend does not become a general chat-history database.

See LangGraph Integration for adapter and runner details, OpenAI compatibility for the interrupt protocol, and Demo Architecture for the complete example deployment.