Demo Design Choices¶

Why the demo works the way it does. Each section owns its component's decisions. Add a row for each significant decision; when one changes, update its row. Package decisions live in Design Choices.

OpenTelemetry¶

Choice	Why	Cost	Revisit when
Dashboards read graph runs, failures, and duration from LGOS's `gen_ai.invoke_workflow.duration` metric and conversations from its `invoke_workflow` spans.	Both cover foreground and background execution, and their `error.type` tells an exception from a cancellation. Background submission finishes before execution and has no graph conversation attributes on its HTTP span. Only graph services emit them, so a new graph service appears without Collector or dashboard changes.	Durations exclude Hatchet queue time, and the conventions' bucket boundaries make percentiles below one second coarse. Conversation rows need a finished workflow span.	Dashboards need queue and submission latency, or graphs need sub-second latency percentiles.
Each demo FastAPI app records its own requests with FastAPI's native OpenTelemetry and sets `auto_configure` to `False`; `opentelemetry-instrument` owns every process's providers, OTLP export, logging, and client instrumentation.	FastAPI's telemetry follows its own routing, including mounts, without monkeypatching or per-chunk transport spans. One SDK owner keeps every process, including the worker, on the same OTLP pipeline.	Two mechanisms to understand. Each app disables FastAPI's environment export in code. `http.route` includes the mount, and FastAPI records no `user_agent.original`.	FastAPI's environment export can own the whole SDK, or the deployment needs request attributes FastAPI does not record.
The API and background worker use Hatchet's native instrumentor with the provider configured by `opentelemetry-instrument`; direct Hatchet collector export is disabled.	Native spans and trace propagation join background execution to the request through the existing Collector. SDK exclusions omit payloads and caller metadata.	Traces go to the configured observability backend; deployments must keep the native exclusions configured.	The deployment needs application spans in Hatchet's own trace viewer.

Gateways¶

Choice	Why	Cost	Revisit when
Both UIs reach LGOS only through the gateway selected by `OPENAI_GATEWAY_TYPE`; neither connects to an API container or imports LGOS.	The demo exercises a real OpenAI-compatible edge, and UI inference cannot bypass the gateway's data plane.	LiteLLM rewrites error `type`, `param`, and `code`; Bifrost's normalized model detail omits LGOS extensions.	A gateway preserves LGOS metadata and errors unchanged.

LiteLLM¶

Choice	Why	Cost	Revisit when
Run the `homeserver-litellm` image and sync LGOS catalogs into native `model_info.lgos`.	The image preserves native Responses streaming and background lifecycles; the sync gives both UIs LGOS metadata through `/model/info`.	A custom image to maintain, and a sync to rerun after graph changes.	Upstream LiteLLM preserves streaming and background Responses.

Bifrost¶

Choice	Why	Cost	Revisit when
One custom provider per API and a dedicated `lgos-files` provider.	Separate identities show independent APIs behind one endpoint; normalized Files need their own provider.	Clients keep the provider prefix in Responses model IDs because Bifrost ignores `x-model-provider` there, and send it as the `provider` query parameter on background retrieve and cancel.	Bifrost honors `x-model-provider` on Responses.
Publish LGOS metadata as native `additional_attributes` on zero-priced pricing rows, written by one catalog job after Bifrost and the graph APIs are up.	Both UIs read complete metadata from one native `/v1/models` call, and the graph providers enable only model listing and native Responses. Bifrost stores attributes only on pricing rows, which only its datasheet creates.	A generated datasheet without Bifrost's public prices, an empty datasheet seeded before Bifrost's first start, a sync to rerun after graph changes or gateway restarts, and an undocumented management endpoint behind the admin account.	Bifrost attaches attributes without pricing rows or loads them from `config.json`.

Chainlit¶

Choice	Why	Cost	Revisit when
Human review is a persisted message whose `chainlit-utils` form element submits through `callAction`.	Reload and navigation restore reviews from history, with no waiter, resume task, or session cache. `Ask*Message` blocks on its WebSocket, and Chainlit does not persist plain `cl.Action` controls.	Custom JSX that `chainlit-utils` serves ahead of Chainlit's public-file route; answers are recorded before the continuation, so a failed or stopped run leaves the review answered; a running response is not rebound to a new WebSocket, so returning early may need a refresh.	Chainlit persists actions or resumable asks, or the demo runs several Chainlit workers.
`HitlWorkflow` normalizes `createdAt` before updating a restored ledger.	Chainlit's PostgreSQL layer rejects the timestamp format it hydrates; without this, a ghost `pending` ledger blocks the next turn.	A workaround coupled to Chainlit internals.	After every Chainlit upgrade.
Startup calls `keep_restored_sessions()`, so a reconnected session keeps its state.	Chainlit resumes a restored session again on every reconnect, including the one a resumed thread's chat profile triggers: the model receives the history twice, `user_session` resets, and `on_chat_resume` reruns.	Replaces Chainlit's `connection_successful` handler; a browser that was offline while a response finished is not redrawn until reload.	Chainlit stops resuming restored sessions, and after every Chainlit upgrade.
Chainlit keeps its own element copy in its storage provider; the central Files API owns the inference copy, whose file IDs the user message's metadata keeps for edits and resumed threads.	Chainlit's storage provider chooses object keys and gives the browser a direct read URL; the Files API assigns file IDs and serves content only through the authenticated gateway. Bridging them needs a key mapping and an authorized read route.	Every upload is stored twice, and displayed generated files are copied into Chainlit storage.	Chainlit stores elements under provider-returned keys and serves reads through an authenticated route.

Open WebUI¶

Choice	Why	Cost	Revisit when
One manifold Pipe serves every graph and publishes each model's description, capabilities, MCP tool, and Chat Variables form from the gateway catalog whenever Open WebUI lists models; the `homeserver-open-webui` image keeps that info from `pipes()`, and the app factory hides the `lgos/` models of Open WebUI's own gateway connection, which serves every other gateway model.	Open WebUI keeps only `id` and `name` from `pipes()` and otherwise reads model info from stored model records; upstream declined Pipe-provided metadata (open-webui#23047). Without stored copies, graph changes appear on the next model-list load, as in Chainlit.	An image patch that upstream declined, so it stays, and a wrapper of an Open WebUI internal; both are re-checked on every image upgrade. Models without stored records need `BYPASS_MODEL_ACCESS_CONTROL`, so Open WebUI has no per-model access control: with the shared key every user can use every model, and only the SSO overlay limits models per user. Open WebUI's single model registry can fail a per-user chat with "Model not found" after another user's listing. A model saved in Open WebUI's editor replaces the runtime info.	Open WebUI accepts model info from Pipes.
Small patches for upstream gaps ship in the `homeserver-open-webui` image, built from the pinned official image: `pipes()` receives the user's OAuth token, and OpenAI speech and images authenticate like the OpenAI connection with the same base URL. The Generic Pipe follows Open WebUI's connection to the gateway too, so that connection's `auth_type` is the one credential switch.	Both gaps would otherwise keep a shared key in SSO mode, and upstream has no option for either (open-webui#21282 closed unmerged). The image build applies the patches with zero fuzz and tests them, and the same image serves other Open WebUI deployments.	Every Open WebUI upgrade needs a rebuilt image from the separate image repository, and `ENABLE_BASE_MODELS_CACHE` must stay off because the cache lists models without a user.	Upstream closes both gaps.
The app factory installs the Functions at Open WebUI startup through Open WebUI's Functions API, called in-process with a short-lived admin token built like Open WebUI's own automation tokens; the Pipe reads its gateway from the container environment.	Open WebUI loads Functions only from its database. Installing at startup needs no separate command, and the public API keeps Open WebUI's own validation instead of writing its tables. The token needs neither a password, which the SSO overlay disables, nor an API key.	Every start compares the stored Functions with the mounted source, a Function change needs a container restart, and the token's claims follow an Open WebUI internal, re-checked on every image upgrade.	Open WebUI can load Functions from files or configuration.
Later requests replay an answered `ask_user` card as `lgos_interrupt` calls rebuilt from the card: its question, choices, and answer, after the text shown above it.	Open WebUI stores the card and its answer natively but rewrites `ask_user` arguments, dropping keys it does not know, so the original interrupt payload is not kept.	A replayed call differs from the one LGOS sent: payload fields besides `question`, `choices`, and `allow_other` are lost, and the details shown above the card replay as assistant text. Chainlit replays its exact calls from the review ledger.	Open WebUI keeps unknown `ask_user` arguments, or a Pipe can store data with a tool call.
Open WebUI keeps its own raw upload copy; the central Files API owns the inference copy.	Open WebUI's native attachment UI requires its own file record.	Every upload is stored twice.	Open WebUI can attach an external file ID.

Coding Agent¶

Choice	Why	Cost	Revisit when
Codex owns a conversation's memory: a request carrying `safety_identifier` and `metadata.conversation_id` resumes that conversation's Codex thread and sends only the latest message. The UI's history only starts a thread.	Codex keeps the commands it ran and their output, which no UI replays. Without its own thread it cannot answer questions about its earlier work.	Editing or regenerating an earlier UI message does not change what Codex remembers. Threads are files in `demo/docker/volumes/lgos-codex` that nothing deletes, and any caller sending the same pair continues the thread. A conversation whose thread is gone restarts from the UI's history with a status notice, without Codex's record of earlier commands.	Retained graph state gets a suite-wide deletion and expiry contract.