Reference¶
OpenAI-Compatible API¶
Default prefix: /v1. Change it with LGOS_OPENAI_API_PREFIX or
bind_openai_api(prefix=...). Generic access logs are emitted by the
deployment's ASGI server or ingress proxy.
| Method | Path | Purpose |
|---|---|---|
GET |
/v1/models |
List registered graph models with LGOS descriptions, features, and tool names. |
GET |
/v1/models/{model} |
Retrieve one model with the required LGOS metadata extension. |
POST |
/v1/responses |
Run a graph now or accept an opted-in background Response. |
GET |
/v1/responses/{response_id} |
Retrieve an authorized background Response, or stream its events with stream=true and starting_after. |
POST |
/v1/responses/{response_id}/cancel |
Cancel an active background Response; a finished one returns 400. |
POST |
/v1/chat/completions |
Run a graph through OpenAI chat completions. |
GET |
/v1/health |
Health check. |
GET |
/v1/version |
The installed LGOS version, as {"version": "..."}. |
FastAPI docs for the mounted OpenAI app are disabled by default. Set
LGOS_OPENAI_API_DOCS_ENABLED=True to expose {prefix}/docs, {prefix}/redoc,
and {prefix}/openapi.json.
Responses Request¶
The route accepts string or ordered message input, instructions, plain
input_text, input_file.file_id, string-valued metadata, safety_identifier
and its deprecated alias user, flat client
function tools, registered name-only custom-tool selectors, standard
web_search, their choices,
parallel_tool_calls, plain text output, and streaming. Replayed
assistant output messages preserve phase; complete
function_call items and matching string-valued function_call_output items
support ordinary client-tool continuation. Interrupt continuation sends only
matching function_call_output items with previous_response_id.
Server execution returns native custom_tool_call and
custom_tool_call_output pairs or a
web_search_call in the same response; complete output items can also be
replayed as history.
LGOS executes registered custom tools inside that response. This intentionally
differs from OpenAI's ordinary custom-tool flow, where caller code executes the
tool and supplies its output to a later model request; the wire items remain
standard Responses types.
The public web_search shape does not prescribe the graph's search backend;
the bundled demo chooses an HTTP or upstream provider backend.
Foreground LGOS Responses are not persisted for retrieval or deletion. Omitted,
null, and false store values are accepted, and the returned foreground
Response reports store=false; foreground store=true is rejected.
For a model declaring GraphFeature.BACKGROUND, background=true selects the
background path and permits either store=false or store=true; the
background engine keeps the result either way, and Hatchet keeps it for its
data retention period
(30 days by default when self-hosted). With stream=true the request returns
the Response's event stream, which retrieval can replay and resume;
retrieving a Response created without stream=true as a stream returns 400.
conversation is unsupported in both modes.
previous_response_id is supported for interruptible graphs to resume execution
(and rejected for non-interruptible graphs); new instructions are rejected on
those resumes. The route also rejects unregistered custom tools, client-supplied
custom descriptions or formats, other built-in tools,
structured output, image/audio input, URL or inline file input, function
result-content lists, reasoning and generation controls, include, stream options,
service tiers, reusable prompts,
prompt-cache controls, and truncation. Unknown fields are not silently ignored.
See the supported Responses subset
for the complete behavior and continuation rules.
Chat Completions Request¶
Chat messages accept string content, explicit text content parts, and native
file parts containing only file.file_id. The route supports modern function
tools, tool_choice, assistant tool_calls, matching tool messages,
streaming, and stream_options.include_usage. Image and audio parts, inline
file data or filenames, prompt-cache fields, deprecated function fields,
generation controls, and other unknown fields are rejected.
Settings¶
Package settings:
| Setting | Default | Notes |
|---|---|---|
LGOS_OPENAI_API_PREFIX |
/v1 |
Must start with /; trailing slash is normalized. |
LGOS_OPENAI_API_DOCS_ENABLED |
False |
Enables docs only for the mounted OpenAI app. |
LGOS_ENABLE_LANGFUSE |
False |
Lazily adds the package Langfuse callback to every graph run. |
The lgos command adds the server settings.
Settings prefixed with DEMO_ belong to the independent example applications
and are documented under Demo Settings and Commands.
Public API¶
Use LanggraphOpenaiServe to bind OpenAI-compatible routes to a FastAPI app.
After binding, server.openai_app exposes the mounted FastAPI application for
host integrations such as manual middleware or telemetry instrumentation.
Use GraphRegistry(graphs={...}, run_coordinator=...) to map OpenAI model
names to GraphConfig values; registry.graphs is a plain dict. The registry
must contain at least one graph. Model IDs must be non-empty, must not contain
/, and must not be . or ... registry.get_graph(model) returns a
GraphConfig or raises a 404 model_not_found error. run_coordinator is
shared by every interrupt-enabled graph and is required only when a graph
declares GraphFeature.INTERRUPTS.
LanggraphOpenaiServe(registry=registry, app=app) serves the registry.
LanggraphOpenaiServe(..., background=backend) accepts an optional
BackgroundBackend. LanggraphOpenaiServe(..., checkpoint_scope=resolver)
accepts an optional sync or async callable from FastAPI Request to a
non-empty, server-trusted string.
Interrupt checkpoint keys and background Response authorization include this
scope before model and run identity. Use
an authenticated tenant or principal identifier when runs must be isolated
between security domains; do not derive the scope from
untrusted OpenAI metadata or the end-user safety_identifier and user
fields. The default "default" scope is suitable only for a single-tenant or
shared-trust deployment. The resolver must return the same scope for the initial request and
its resume; changing tenant identity makes the other scope's checkpoint
deliberately unreachable.
RequestContextFilter adds the active request's LGOS fields, such as
request_id, to log records; install it on a host handler to enrich every
record. See Production Logging.
Responses input_file.file_id content and native Chat file parts normalize to
the same LangChain file block, so graphs receive native file_id values and
decide whether to download, parse, or forward them. File upload and storage
belong to an external OpenAI Files API, not the LGOS package. See
Accept And Display Files.
GraphConfig accepts:
graph: compiled graph, sync factory, or async factory.description: required human-readable model description advertised by model listing and retrieval.features:GraphFeaturevalues that enable optional server behavior or advertise graph input and client-tool capabilities.client_settings: explicit publicClientSettingsmodel class advertised by model retrieval.server_tools: allowlist of server-executed tool names, advertised aslgos.server_tools. A registered custom tool is selected with the Responsescustomtype and name;web_searchuses its built-in type. The graph owns tool definitions and execution. See Server Tools.client_tools: names of client-owned functions the graph can call, such asdisplay_file, advertised aslgos.client_toolsso clients offer only the functions a graph uses. LGOS does not enforce the list, and a name cannot also be a server tool.request_to_input(request, messages): custom normalized request and LangChain messages to graph input.context_factory(request, client_settings): compose the final typed LangGraph runtime context from normalized request values, server-owned values, and optional validated public settings.output_to_message(output): custom graph output to a durableAIMessage.
GraphConfig is immutable after construction. Pydantic snapshots features,
server_tools, and client_tools as frozen sets, so later mutations of the
input collections cannot change a registered model.
To run LangChain callbacks on every run of a graph, bind them with LangGraph's
graph.with_config(callbacks=[...]). LangGraph merges them with the callbacks
LGOS adds to each run, without changing the bound graph.
Streaming forwards non-empty text from every AIMessageChunk emitted by the
graph's messages stream. To keep a private model call out of the stream, tag
it with LangGraph's
nostream
tag (langgraph.constants.TAG_NOSTREAM), for example
ChatOpenAI(..., tags=[TAG_NOSTREAM]); the call still runs and
returns its output, but LangGraph emits none of its tokens.
disable_streaming=True
is for models that cannot stream: it makes the provider request non-streaming,
and LangGraph emits the call's complete message instead. With either option,
LangGraph also emits a complete message that a node returns into state. LGOS
forwards neither complete message, but other consumers of the graph's stream
see them.
A directly supplied compiled graph is reused and validated when its
GraphConfig is built. A sync or async graph factory is called for every request
and is never cached; LGOS validates each resolved value as a compiled state graph
and rechecks it before execution. Validation checks the context schema and
interrupt checkpointer capabilities, rejects static breakpoints
(interrupt_before or interrupt_after) in the graph or its subgraph nodes,
since LGOS pauses only at interrupt(), and rejects a checkpointer, including
checkpointer=True, on a graph without GraphFeature.INTERRUPTS, since clients
send the full conversation with every other request. A registry with an
interrupt-enabled graph and no run_coordinator fails during GraphRegistry
construction. A graph may declare both interrupts and background; a background
run that reaches an interrupt completes with lgos_interrupt function calls,
and an answer in either mode continues it.
When both are configured, LGOS validates the public settings first and passes
them to context_factory. Without a factory, the validated settings instance is
the runtime context, so the graph must use that settings model as its
context_schema. A factory may return None; every non-null result requires a
graph context schema. LGOS passes server-owned factory results to LangGraph
without rebuilding them. LangGraph's native
runtime-context handling
constructs mapping values through dataclass and Pydantic context schemas and
trusts existing instances. The factory owns the validity of instances it
creates. Graphs should access context from an injected Runtime[Context].
Graph adapters receive an immutable, protocol-neutral GraphRequest from either
API's decoder. It exposes
the shared model, metadata, user (the end user from safety_identifier, or
the deprecated user field), normalized client function tools,
selected server-tool names in server_tools, tool_choice, and parallel_tool_calls
values. NamedFunctionToolChoice identifies a required client function, while
NamedCustomToolChoice identifies a required registered custom tool. A
single web_search declaration with tool_choice="required" requires search.
Named built-in choices are outside the supported subset.
Raw OpenAI transport models are
not part of the graph-adapter interface.
Runtime context is separate from RunnableConfig:
| Value | LGOS/LangGraph path | Intended use |
|---|---|---|
| Graph input | graph.astream(input, ...) |
Messages and mutable workflow state. |
| Runtime context | public settings → optional context_factory → context= → Runtime.context |
Immutable per-run application values and dependencies. |
| Runnable config | config= |
Callbacks, tags, tracing, and other execution controls. |
| Interrupt run | server scope + model + server-generated run UUID → internal checkpoint key | Isolate, interrupt, and resume one operation. |
LGOS's runnable config carries its callbacks, run metadata, and, for an
interrupt-enabled run, a fixed-length SHA-256 checkpoint key derived from the
server-trusted scope, registered model, and run UUID. This is deliberately
not a UI chat or thread ID. There is intentionally no adapter for placing
arbitrary OpenAI request fields into config["configurable"]; use typed runtime
context for values consumed by nodes.
Langfuse Tracing¶
Langfuse is a first-class optional integration. Install it and enable the default callback through process environment settings:
uv add "langgraph-openai-serve[tracing]"
export LGOS_ENABLE_LANGFUSE=True
export LANGFUSE_PUBLIC_KEY=pk-lf-...
export LANGFUSE_SECRET_KEY=sk-lf-...
LANGFUSE_BASE_URL is optional; Langfuse Cloud is the default. Set it only for
a different cloud region or a self-hosted instance. A self-hosted Langfuse must
be a release that Langfuse's
compatibility matrix
lists for the Python SDK version that the extra installs. Langfuse's
CallbackHandler owns its standard SDK configuration and error behavior. LGOS
constructs it on the first graph run that needs runnable configuration, then
reuses that process-wide handler. Before that, LGOS creates the Langfuse
client with a tracer provider of its own, as Langfuse
recommends beside another
tracing backend. Langfuse's observations, which carry prompts and outputs, then
never reach the application's OpenTelemetry exporters, and the
application's spans never reach Langfuse; both share each trace ID. An
application that creates its own Langfuse client first keeps its own tracing
setup. When enabled, LGOS adds the Langfuse handler to every run, alongside
callbacks bound with graph.with_config(). To provide a custom Langfuse
handler, leave the toggle off and bind it with
graph.with_config(callbacks=[handler]);
langgraph_openai_serve.integrations.langfuse.get_langfuse_callback returns
LGOS's own handler for explicit use.
LGOS gives every graph run the stable name
lgos.graph_run for both endpoints and adds RunnableConfig.metadata fields for the
request ID, registered graph model, (for interrupt runs) operation ID, and
(when it names an end user with safety_identifier or the deprecated user)
that end user as lgos.user. Graph code can read them with LangGraph's
get_config(), with or without Langfuse. When the toggle is on, LGOS also sets
the end user, metadata.conversation_id, and the registered model as Langfuse's
native user, session, and trace name on every observation of the run; a custom
handler passed with the toggle off does not receive them. LangGraph also propagates primitive
configurable values
during execution, so callbacks on interrupt runs receive the derived checkpoint
thread_id. LGOS does not set LangChain's native tracer run_id or force a
custom Langfuse trace ID. See Production Logging and Request
Correlation.
The features set is returned in the lgos.features extension and
enables server behavior where applicable.
GraphFeature.MCP_TOOLS advertises that a client may attach and execute tools
from its configured MCP gateway; it does not publish tool definitions or grant
access to them.
GraphFeature.FILE_INPUTS advertises that the graph
resolves native file content parts. GraphFeature.INTERRUPTS enables and
advertises the interrupt/resume flow. GraphFeature.BACKGROUND enables and
advertises background Responses.
OpenTelemetry¶
LGOS reports every graph execution through the OpenTelemetry API, following the
GenAI workflow span
and workflow metric
conventions. The API records nothing until the host application configures an
SDK, for example with opentelemetry-instrument.
HTTP request telemetry belongs to the host app. The mounted OpenAI app turns off
FastAPI's native OpenTelemetry,
so the host's telemetry settings, including turning it off, cover LGOS routes,
and http.route includes the mount, such as /v1/responses.
| Signal | Name | Attributes |
|---|---|---|
Span, kind INTERNAL |
invoke_workflow {model} |
gen_ai.operation.name=invoke_workflow, gen_ai.workflow.name, gen_ai.conversation.id when the request supplies metadata.conversation_id, and error.type on failure |
Histogram, unit s |
gen_ai.invoke_workflow.duration |
gen_ai.workflow.name, and error.type on failure |
gen_ai.workflow.name is the registered model. Both signals cover graph
execution through the final output; they exclude lease waits, background queue
time, and checkpoint cleanup. While LGOS advances the graph, the span's context
is current: spans the graph creates, such as model calls, are its children,
while code that consumes a stream keeps its own parent span between events. The
span itself is not current, so code in the graph cannot change it, and it
carries only the attributes above; add attributes to spans of your own. A run
that ends before its output exists fails: one that raises an exception, one
cancelled by a disconnecting client or a cancelled background Response, and one
whose stream the consumer closes mid-run. The span status is then ERROR with
the exception message, if any, and error.type is the exception type, qualified
by its module unless it is built in, such as asyncio.exceptions.CancelledError
or GeneratorExit. Closing a stream after its final output is not a failure.
Design Choices lists the tracers whose
cancellation handling this follows. A raised Exception propagates to its
handler: LGOS routes and the background engines log it with its traceback, and
direct Python callers receive it; cancellations and closed streams are not
logged. The span does not repeat an exception as an event. The histogram uses
the conventions' bucket boundaries, which start at one second; configure an SDK
View for finer boundaries.
Runtime Settings¶
Subclass ClientSettings to publish only fields deliberately selected by the
server author. LGOS never inspects or publishes the LangGraph context schema:
from pydantic import Field
from langgraph_openai_serve import ClientSettings
class PublicSettings(ClientSettings):
use_history: bool = Field(default=True, title="Use conversation history")
Pass this model as GraphConfig.client_settings and use it as the graph's context
schema when it is the complete runtime context. Every public field must have a
deterministic default, because clients omit values equal to the advertised
defaults; a default_factory must return the same value on every call.
Registration fails when the model cannot build its defaults or schema.
All public fields travel together as compact JSON text in the
metadata.lgos_settings string. Clients omit values equal to the advertised
defaults. System instructions remain ordinary OpenAI messages and are
independent of ClientSettings; native OpenAI fields keep their standard
request semantics.
LGOS validates the settings with the model's strict JSON validation on every
request. Without
context_factory, the settings become Runtime.context. A factory can instead
combine them with server-derived identity, authorization, database clients, and
other dependencies.
The serialized descriptor appears only on model retrieval as
lgos.client_settings, with json_schema and defaults fields. All client
settings use the fixed metadata.lgos_settings key. Clients use the
descriptor's validated defaults object as the baseline; default keywords
within the generated JSON Schema are annotations, not the runtime baseline. The
schema's $schema keyword declares the JSON Schema 2020-12 dialect.
See Configure LangGraph Runtime Settings for the runtime settings flow, and Runtime Settings for the request lifecycle.
Interrupt-enabled graphs have additional registration requirements:
- compile the graph with an asynchronous checkpointer that supports
aget_tuple(),aput(),aput_writes(), andadelete_thread(); - configure an asynchronous
GraphRegistry.run_coordinator; and - use a durable checkpointer and cross-process coordinator in production.
The coordinator provides single-flight leases for interrupt runs: it returns an
async context manager and rejects an occupied checkpoint key with RunBusyError
instead of queueing it, which clients receive as HTTP 409 run_busy. Exiting
that context manager must release the lease even when the exit is cancelled.
The initial request does not require metadata. LGOS generates a UUID run ID for
every new run and embeds it in the paused Response ID. InMemoryRunCoordinator
is suitable only for tests and a single-process development server; it cannot
serialize requests across workers or hosts.
Expire Paused Runs¶
Pending checkpoints exist only to resume an interrupt batch returned to the
client. LGOS deletes isolated checkpoint state after terminal completion or when
execution fails or is cancelled before producing that batch. A run abandoned
after its batch is returned stays until deleted: call
delete_expired_interrupt_runs(checkpointer, run_coordinator, older_than=...)
from langgraph_openai_serve.graph.interrupt on a schedule, as LangGraph Agent
Server's checkpointer TTL
does. In production, prefer one scheduled job over a loop in every replica, for
example a Kubernetes CronJob with concurrencyPolicy: Forbid or your task
queue's scheduler. It deletes the runs whose latest pause is older than
older_than and returns their count. It reads every checkpoint through
alist(), leaves threads LGOS did not create alone, and skips runs whose lease
is held. Choose a TTL longer than the longest time a user may take to answer. To
write your own cleanup, select threads whose checkpoint metadata contains
OPERATION_ID_METADATA_KEY from the same module, then hold each run's lease,
confirm that its latest checkpoint in any namespace is still older than your
TTL, and delete it through the checkpointer. lgos serve runs this sweep in
each API process, configured by LGOS_INTERRUPT_TTL_MINUTES and
LGOS_INTERRUPT_SWEEP_INTERVAL_MINUTES; set the interval to 0 when one
scheduled job sweeps instead.
PostgreSQL Coordination¶
Install langgraph-openai-serve[postgres] to use the public
langgraph_openai_serve.integrations.postgres.PostgresRunCoordinator. Use
LangGraph's official
AsyncPostgresSaver
for checkpoints and
AsyncPostgresStore
for application data. The LGOS adapter supplies only the cross-worker
interrupt-run lease; it does not replace either storage primitive. Run each
configured storage adapter's setup() before serving requests, serializing
migration attempts when workers can start together. Their migrations use
CREATE INDEX CONCURRENTLY, which deadlocks with a worker blocked in
pg_advisory_lock; retry pg_try_advisory_lock instead. A shared
pool must follow the upstream connection requirements: autocommit=True,
prepare_threshold=0, and mapping rows.
PostgresRunCoordinator(pool, max_concurrent_leases=...) accepts an existing
psycopg_pool.AsyncConnectionPool configured with mapping rows and the default
close_returns=False; physical session closure is the safety fallback for an
indeterminate lock operation. When persistence adapters share that pool, set
the lease limit below the pool maximum so at least one connection remains
available for persistence I/O. Create one coordinator per process-owned pool
so that this capacity limit is not accidentally multiplied. Session advisory
locks require direct PostgreSQL connections or session-mode pooling;
transaction-mode poolers cannot preserve the lease. Lock contention itself
fails immediately through PostgreSQL's pg_try_advisory_lock; connection
checkout still follows the pool's configured timeout. The
demo deployment uses one pool for both
storage adapters and interrupt coordination. Each process applies pending
LangGraph migrations during startup under a separate schema advisory lock.
Busy interrupt leases fail before streaming begins with HTTP 409
and code: "run_busy".
Background Execution¶
LanggraphOpenaiServe(background=...) accepts any BackgroundBackend: an
engine that runs BackgroundJobs and stores their status and result.
| Method | Contract |
|---|---|
submit(job) |
Start the job, or return the run already holding job.idempotency_key. |
get(run_id) |
Return the run's job, status, and executed Response, or None. |
cancel(run_id) |
Stop the run; a finished run keeps its outcome. |
events(run_id) |
Yield the run's published events from the first, then those that follow. It may end early. |
The engine's worker calls
execute_background_job(job, engine_run_id, registry, publish) with its own run
ID, not job.run_id, which identifies the graph run. It runs the job through the
foreground Responses path and returns the Response JSON the engine stores. With
publish, which an engine passes for a job created with stream=true, the job also
hands it each event. Run IDs must be UUIDs because the public Response ID embeds
the run ID, so reading a Response needs no lookup table.
InMemoryBackgroundBackend(registry) runs jobs as tasks of one process for
development and tests; enter its lifespan in the application's lifespan.
Install langgraph-openai-serve[hatchet] for
langgraph_openai_serve.integrations.hatchet. create_hatchet_task(hatchet)
registers the task in the API and worker processes, with a 30-minute
schedule_timeout and a one-hour execution_timeout by default.
HatchetBackgroundBackend(task, hatchet.runs, hatchet.streams) submits, reads,
cancels, and follows its runs. The worker's Hatchet lifespan yields the
GraphRegistry, including its run_coordinator, that the task executes. The
task has no retries, and its 24-hour Hatchet idempotency key is the scoped
Idempotency-Key digest. The adapter is never imported by the core package.
Background create accepts an optional Idempotency-Key header of 1 to 255
characters. A retry returns the original Response without starting a second
run; reuse with different content returns 422 with
code="idempotency_key_reused". A background interrupt run chooses its run ID
at submission.
See Run Responses In The Background for the client contract, graph requirements, and deployment wiring.
Server¶
Install langgraph-openai-serve[server] for the lgos command and
langgraph_openai_serve.server. lgos serve module:attribute and
lgos worker module:attribute run a RegistryFactory: a callable that takes
ServerResources (checkpointer, store, run_coordinator) and returns a
GraphRegistry, or an async context manager that yields one. lgos serve
also accepts every Uvicorn option and UVICORN_* variable.
create_app(factory, settings=...) returns the FastAPI application that
lgos serve runs; its lifespan opens the resources and mounts the OpenAI
routes. ServerSettings reads the LGOS_* server settings. See
Run The LGOS Server.
Streaming Status¶
Inside a long-running graph node or tool, publish user-facing status with
status_event():
from langgraph.config import get_stream_writer
from langgraph_openai_serve import status_event
writer = get_stream_writer()
writer(status_event("Generating audio"))
# Perform the long-running work.
writer(status_event("Audio ready"))
The helper returns this custom stream value:
Status text is deliberately authored by the graph; LGOS does not infer it from internal node names or state. Graphs need no feature declaration to publish status.
Publish a status before slow work that streams nothing, such as a model call
whose output stays private: until a graph's first event, a client sees no
progress. Upstream LiteLLM and Bifrost also hold a stream's opening events,
such as response.created, until its first output, so a client behind them
receives nothing at all until then.
Status is streaming-only. Streaming Responses, in the foreground or in the
background, needs no metadata opt-in and emits each non-empty description as a
standard phase="commentary" message.
Non-streaming Responses and Chat Completions ignore statuses. Both APIs ignore
any other custom stream data. Use standard Responses function calls plus the
Files API for portable durable rich output.
See Streaming status for the wire contract and Stream final text and commentary for consumption.
Citations¶
Put citations on the final LangChain AIMessage:
from langchain_core.messages import AIMessage
from langchain_core.messages.content import create_citation, create_text_block
message = AIMessage(
content=[
create_text_block(
text="Read the source [1].",
annotations=[
create_citation(
url="https://example.com/source",
title="Example source",
start_index=9,
end_index=14,
cited_text="source",
)
],
)
]
)
Put visible inline citations in the assistant text. Structured annotations add machine-readable provenance; clients are not required to invent marker text from annotation indices.
LangChain citation indices refer to their containing text block. LGOS offsets
them into the final response text and preserves OpenAI's inclusive end_index.
Use citation_slice(start_index, end_index, content) to validate them and create a
Python slice. Responses maps citations to output_text.annotations and emits
the typed annotation event while streaming. Chat maps them to completed
message.annotations; its final streaming delta uses the compatibility
extension.
See Citation ownership for transport and client behavior.
The streaming graph runner preserves LangGraph's native CustomStreamPart
values, including their execution namespace. Non-streaming invocation does not
subscribe to or replay custom events.
Python API¶
The package root exports the API most applications use; these modules complete the public API.
langgraph_openai_serve ¶
Serve LangGraph graphs through an OpenAI-compatible API.
ClientToolChoice
module-attribute
¶
ClientToolChoice = (
Literal["none", "auto", "required"]
| NamedFunctionToolChoice
| NamedCustomToolChoice
)
A normalized tool_choice: a mode, or one named tool the model must call.
BackgroundBackend ¶
BackgroundJob ¶
Bases: BaseModel
A validated request and the server-owned context that executes it.
BackgroundRun ¶
Bases: BaseModel
An engine's current view of one submitted job.
queued_snapshot ¶
snapshot ¶
Return the public Response for the run's current state.
Source code in src/langgraph_openai_serve/background.py
ClientFunctionTool
dataclass
¶
A client-supplied function available to the graph.
ClientSettings ¶
Bases: BaseModel
Base class for settings that clients may configure for a graph.
Subclasses define the complete public contract. Every field needs a deterministic default so model discovery can advertise a complete, stable settings object.
default_values
classmethod
¶
Return the JSON defaults advertised by model discovery.
Source code in src/langgraph_openai_serve/graph/client_settings.py
json_schema
classmethod
¶
Return the JSON Schema advertised by model discovery.
validate_request
classmethod
¶
Read and validate this model's values from an OpenAI request.
Source code in src/langgraph_openai_serve/graph/client_settings.py
GraphConfig ¶
Bases: BaseModel
One graph served as an OpenAI model, with its features and adapters.
build_context
async
¶
Build the LangGraph runtime context for a request.
Source code in src/langgraph_openai_serve/graph/graph_registry.py
build_input
async
¶
Build the native graph input for a normalized request.
Source code in src/langgraph_openai_serve/graph/graph_registry.py
render_output
async
¶
Convert native graph output into the final assistant message.
Source code in src/langgraph_openai_serve/graph/graph_registry.py
resolve_graph
async
¶
Get the graph instance, resolving callable graph factories.
Source code in src/langgraph_openai_serve/graph/graph_registry.py
supports ¶
validate_client_settings
classmethod
¶
Fail at registration when a settings model cannot be advertised.
Source code in src/langgraph_openai_serve/graph/graph_registry.py
validate_compiled_graph ¶
Fail at registration when a directly supplied graph cannot be served.
Source code in src/langgraph_openai_serve/graph/graph_registry.py
validate_tool_ownership ¶
Keep the side that executes each advertised tool unambiguous.
Source code in src/langgraph_openai_serve/graph/graph_registry.py
GraphError ¶
Bases: RuntimeError
Raised when a graph's configuration or output cannot be served.
GraphFeature ¶
Bases: StrEnum
Features supported by a registered graph.
GraphRegistry
dataclass
¶
The graphs served as OpenAI models, keyed by model ID.
run_coordinator leases the interrupt runs of every interrupt-enabled
graph; use a shared coordinator when several processes serve these graphs.
__post_init__ ¶
Fail at startup for a registry the OpenAI routes cannot serve.
Source code in src/langgraph_openai_serve/graph/graph_registry.py
get_graph ¶
Return the graph served as model.
Source code in src/langgraph_openai_serve/graph/graph_registry.py
GraphRequest
dataclass
¶
Request data shared by protocol decoders and graph execution.
InMemoryBackgroundBackend ¶
Run background jobs as tasks of this application process.
For development and tests: runs stay in memory until the process exits.
Source code in src/langgraph_openai_serve/background.py
cancel
async
¶
Mark an active run cancelled and cancel its task.
Source code in src/langgraph_openai_serve/background.py
events
async
¶
Yield the run's published events, waiting for new ones.
Yields:
| Type | Description |
|---|---|
AsyncGenerator[str, None]
|
Each encoded event, from the first. |
Source code in src/langgraph_openai_serve/background.py
get
async
¶
lifespan
async
¶
Own the execution tasks for one ASGI application lifespan.
Source code in src/langgraph_openai_serve/background.py
submit
async
¶
Start one task, or return the run holding the job's key.
Source code in src/langgraph_openai_serve/background.py
InvalidRequestError ¶
Bases: Exception
A request rejected with an OpenAI invalid_request_error envelope.
Source code in src/langgraph_openai_serve/core/errors.py
LanggraphOpenaiServe ¶
Mount a graph registry's OpenAI-compatible API on a FastAPI host app.
Initialize the server with a FastAPI app and a populated graph registry.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
registry
|
GraphRegistry
|
The graphs to serve and their interrupt run coordinator. |
required |
app
|
FastAPI | None
|
The host FastAPI application to mount the OpenAI API on. If None, a new FastAPI app will be created. |
None
|
checkpoint_scope
|
Callable[[Request], str | Awaitable[str]] | None
|
Optional server-trusted resolver used to isolate interrupt checkpoints by deployment or authenticated principal. |
None
|
background
|
BackgroundBackend | None
|
Optional background backend. |
None
|
Source code in src/langgraph_openai_serve/openai_server.py
bind_openai_api ¶
Mount OpenAI-compatible endpoints on the host FastAPI app.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
prefix
|
str | None
|
URL prefix for the OpenAI routes; defaults to the
|
None
|
Source code in src/langgraph_openai_serve/openai_server.py
RequestContextFilter ¶
Bases: Filter
Add the active LGOS request fields to log records.
LGOS installs it on its own loggers. A host that installs it on a handler also enriches records from graph nodes, dependencies, and its own code.
filter ¶
Enrich a record while preserving fields supplied by the caller.
Source code in src/langgraph_openai_serve/core/logging.py
citation_slice ¶
Convert an inclusive citation span to a validated Python slice.
Source code in src/langgraph_openai_serve/graph/citations.py
execute_background_job
async
¶
Execute one job through the foreground Responses path.
With publish, which an engine passes for a job created with stream=true,
the job runs like a foreground stream and hands each event to it. Without, it
produces only its final Response.
Source code in src/langgraph_openai_serve/background.py
status_event ¶
Build a status update for a node to write with get_stream_writer().
langgraph_openai_serve.graph.runner ¶
Run LangGraph workflows from protocol-neutral requests and messages.
LangGraphOutput
module-attribute
¶
A run's result: its final message, or the interrupts that paused it.
LangGraphStreamEvent
module-attribute
¶
One streamed item: a text delta, a native LangGraph part, or the run's result.
run_langgraph
async
¶
Prepare, execute, and close one graph run for direct Python callers.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
request
|
GraphRequest
|
Normalized graph selection, metadata, user, and client tools. |
required |
messages
|
list[BaseMessage]
|
Decoded LangChain messages to process through the graph. |
required |
graph_registry
|
GraphRegistry
|
The GraphRegistry instance containing registered graphs. |
required |
resume
|
InterruptResume | None
|
A decoded, complete interrupt answer batch, when resuming. |
None
|
checkpoint_scope
|
str
|
Server-trusted scope used to isolate checkpoint state. |
'default'
|
Returns:
| Type | Description |
|---|---|
LangGraphOutput
|
The final assistant message or the pending interrupt batch. |
Source code in src/langgraph_openai_serve/graph/runner.py
run_langgraph_stream
async
¶
run_langgraph_stream(
request, messages, graph_registry, *, resume=None, checkpoint_scope="default"
)
Prepare, stream, and close one graph run for direct Python callers.
Yields:
| Type | Description |
|---|---|
AsyncGenerator[LangGraphStreamEvent, None]
|
Assistant text chunks, custom events, then the final output. |
Source code in src/langgraph_openai_serve/graph/runner.py
langgraph_openai_serve.graph.interrupt ¶
Durable LangGraph interrupts: run leases, run identity, and resume state.
OPERATION_ID_METADATA_KEY
module-attribute
¶
The run metadata key that LangGraph copies into checkpoints, marking LGOS runs.
InMemoryRunCoordinator ¶
Coordinate interrupt runs within one event loop.
Source code in src/langgraph_openai_serve/graph/interrupt.py
__call__
async
¶
Hold the lease for key or raise RunBusyError.
Source code in src/langgraph_openai_serve/graph/interrupt.py
InterruptResume
dataclass
¶
A complete set of interrupt answers for one run.
LangGraphInterruptBatch
dataclass
¶
The durable interrupts awaiting answers for one graph run.
RunBusyError ¶
Bases: InvalidRequestError
Raised when another request holds an interrupt run's lease.
Source code in src/langgraph_openai_serve/graph/interrupt.py
RunCoordinator ¶
Bases: Protocol
Acquire a lease that rejects, rather than queues, an occupied run.
Exiting the lease must release it even when the exit is cancelled, because run cleanup is abandoned after a deadline; shield an asynchronous release.
delete_expired_interrupt_runs
async
¶
Delete paused interrupt runs whose latest pause is older than older_than.
Other threads in checkpointer are left alone, and a run whose lease is
held, such as one being resumed, is skipped until the next call.
Returns:
| Type | Description |
|---|---|
int
|
The number of deleted runs. |
Source code in src/langgraph_openai_serve/graph/interrupt.py
langgraph_openai_serve.server ¶
Serve a graph registry as a standalone application: lgos serve and lgos worker.
RegistryFactory
module-attribute
¶
RegistryFactory = Callable[
[ServerResources], GraphRegistry | AbstractAsyncContextManager[GraphRegistry]
]
Build the served graphs, or open them for the process lifetime.
ServerResources
dataclass
¶
Persistence shared by every graph one server process runs.
ServerSettings ¶
Bases: BaseSettings
Server settings read from explicit values and the process environment.
create_app ¶
Serve the factory's registry under the OpenAI prefix once the app starts.
Source code in src/langgraph_openai_serve/server/app.py
langgraph_openai_serve.integrations.hatchet ¶
Hatchet engine for background Responses.
HatchetBackgroundTask
module-attribute
¶
The Hatchet task that create_hatchet_task registers.
HatchetBackgroundBackend ¶
Submit, read, cancel, and follow background Responses as Hatchet task runs.
Source code in src/langgraph_openai_serve/integrations/hatchet.py
cancel
async
¶
events
async
¶
Yield the run's durable stream; a failure only ends it.
The SDK reconnects by itself after a lost connection. The engine also hangs up a subscription that stayed idle for 30 minutes, which a queued or quiet run can outlast, so that resumes from the last event here.
Yields:
| Type | Description |
|---|---|
AsyncGenerator[str, None]
|
Each encoded event, from the first. |
Source code in src/langgraph_openai_serve/integrations/hatchet.py
get
async
¶
Read the run's input, status, and output from Hatchet.
Source code in src/langgraph_openai_serve/integrations/hatchet.py
submit
async
¶
Trigger the task, or return the run holding the job's key.
Source code in src/langgraph_openai_serve/integrations/hatchet.py
create_hatchet_task ¶
create_hatchet_task(
hatchet,
*,
name="lgos-background-response",
schedule_timeout=timedelta(minutes=30),
execution_timeout=timedelta(hours=1),
)
Register the background Response task in the API and worker processes.
The worker lifespan must yield the GraphRegistry the task executes.
The task is not retried: a failed Response is final, and the client may
create a new one.
Source code in src/langgraph_openai_serve/integrations/hatchet.py
langgraph_openai_serve.integrations.postgres ¶
PostgreSQL coordination for interrupt-enabled graph runs.
PostgresRunCoordinator ¶
Coordinate interrupt runs with PostgreSQL session advisory locks.
The pool must return mapping rows (row_factory=dict_row), which
AsyncPostgresSaver also requires when it shares the pool.
max_concurrent_leases limits how many pool connections coordination may
hold at once. When the checkpointer shares this pool, reserve at least one
connection for checkpoint I/O to avoid exhausting the pool with leases.
Source code in src/langgraph_openai_serve/integrations/postgres.py
__call__
async
¶
Hold a PostgreSQL advisory lock for one interrupt run.
Source code in src/langgraph_openai_serve/integrations/postgres.py
langgraph_openai_serve.integrations.langfuse ¶
Lazy construction for the optional Langfuse tracing integration.
get_langfuse_callback
cached
¶
Return the process-wide Langfuse callback, constructing it lazily.
Source code in src/langgraph_openai_serve/integrations/langfuse.py
langgraph_openai_serve.protocol ¶
Stable names used by the public LGOS protocol extensions.
CONVERSATION_METADATA_KEY
module-attribute
¶
The request metadata key that groups a client's runs into one conversation.
INTERRUPT_TOOL_NAME
module-attribute
¶
The function-call name under which a paused Response exposes each interrupt.
JSON_SCHEMA_DIALECT
module-attribute
¶
The $schema of a graph's published client settings.
MODEL_EXTENSION_KEY
module-attribute
¶
The OpenAI model field that carries a graph's LGOS metadata.
SETTINGS_METADATA_KEY
module-attribute
¶
The request metadata key whose JSON object sets a graph's client settings.
STATUS_EVENT_TYPE
module-attribute
¶
The type of a custom stream event that reports graph progress.