Managed compose services

Container types

Managed compose types inherit the service category capability api_manage_services: true.

Add types in etc/containers/types.json or GET /v1/container-types. Only types with api_manage_services accept POST /v1/container-services and per-service start/stop.

Service record shape

Each etc/services/<id>.json file:

{
  "version": 1,
  "id": "market-studio",
  "type_id": "forge_market_studio",
  "label": "Granite Market Studio",
  "compose_root": "/path/to/deploy/forge-market-studio",
  "compose_files": ["compose.granite.yaml"]
}

compose_files lists overlay filenames (not compose.yaml). Allowed overlays are validated in fleet_server/managed_compose_service.py (includes compose.granite.yaml, compose.market.yaml, and forge-llm overlays).

HTTP API

Legacy forge_llm-only:

Rollout helpers

Infra flows via POST /v1/flows/submit (e.g. market-studio-rollout) use the same per-service_id slot; parallel rollouts across different envs (prod vs dev) are allowed.

Body {"sync": true} runs synchronously; default schedules background rollout and returns log_path.

Market Studio rollout does not use SSH — it builds from a local forge-market checkout, starts compose, registers the managed service, and curls /health.

Use rollout, not start, after a gateway code change

The gateway image bakes apps/forge-gateway in at build time and compose.yaml mounts no source volume. POST /v1/container-services/<id>/start runs docker compose up -d without --build, so it recreates the container from the existing forge-gateway:control-plane image and silently keeps the old code. After changing gateway source, use POST /v1/admin/forge-llm-control-plane-rollout, which runs compose build forge-gateway before up -d.

LLM request queue (bounce vs hold)

control_plane/queue_manager.py serialises inference behind one concurrent slot. DEFAULT_WAIT_BY_MODE in control_plane/modes.py decides how a caller waits when that slot is taken:

Queue depth is capped by FORGE_LLM_MAX_QUEUE_DEPTH (default 64). Only hold callers occupy a slot — a bounced caller stops waiting the moment it receives its 503, so enqueuing one would leak its slot forever.

Diagnosing a wedged queue

A saturated queue and a wedged queue look identical from the client (503 / queue_busy / code: queue_full). Tell them apart by whether queue_position moves:

• Real load — queue_position fluctuates between calls.
• Wedged — queue_position is pinned at exactly FORGE_LLM_MAX_QUEUE_DEPTH across repeated calls and never drains, because every slot is held by a ticket whose waiter is gone.

Confirm against the gateway rather than the client: GET /v1/active reports queue_depth, and 0 there while clients still see queue_full means the queue is fine and the problem is elsewhere. Recover with POST /v1/admin/forge-llm-control-plane-rollout — queue state is in-process, so a restart clears it.

Two accounting rules keep the queue from wedging, both covered by apps/forge-gateway/tests/test_queue_manager.py:

1. A bounce caller is never enqueued.
2. When release() hands the active slot to a ticket whose waiter has already timed out, the timed-out caller returns that slot instead of stranding it.

Tickets older than twice the hold timeout are reaped on the next acquire, so a missed path degrades to a self-healing delay rather than an outage that needs a restart.

Future migration jobs may use template build_market_image (see forge-migrator recipe) to build on Granite without manual SSH.

Example — register Market Studio manually

export FLEET=http://127.0.0.1:18766
export TOKEN=…

curl -fsS -X POST "$FLEET/v1/container-services" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "id": "market-studio",
    "type_id": "forge_market_studio",
    "compose_root": "/home/you/forge-fleet/deploy/forge-market-studio",
    "compose_files": ["compose.granite.yaml"],
    "label": "Market Studio"
  }'

curl -fsS -X POST "$FLEET/v1/container-services/market-studio/start" \
  -H "Authorization: Bearer $TOKEN"

Implementation map

Before: Container templates · After: Operate architecture · Lookup: HTTP API