Deploy to a server¶

One host, Docker Compose, a reverse proxy in front. That is the whole shipped path, and it is what runs the deployments this project is used in.

There are no Kubernetes manifests, and no one-click quickstarts for the platform-as-a-service providers. That is not modesty about scale. The stack is six containers, two of which hold state, one of which can start containers of its own, and one of which is a Postgres that must have pgvector - which is already more than a git push deploy target models, and a guide that pretended otherwise would be describing a deployment nobody has run.

Past one host - a managed Postgres, several API hosts, several deployments - is Scale beyond one host.

Read the production checklist first

Nine settings ship with defaults that are fine on a laptop and wrong on a host somebody else can reach. scripts/server-init.sh below generates all nine, so the checklist is what you check afterwards rather than what you type.

What you need¶


A host	4 vCPU and 8 GB of RAM runs it. See sizing
Container engine	Engine 24+ with the Compose plugin (2.24 or later), and your user in the `docker` group. Nothing is built on the host: the images are pulled from GHCR. Alternatively, use the manual Podman path.
Two hostnames	one for the site, one for the API — see why two
A reverse proxy	Traefik or Nginx. It terminates TLS
No provider key	chat models and embeddings are both keyed per organization, in the product's vault. The environment holds no credential for either — see Data protection

The host also needs ports 80 and 443 open, and nothing else. Postgres and Redis are published on no interface at all, and the worker serves nothing.

Why two hostnames¶

The browser talks to both. Most calls go through the frontend's own server-side routes, but the chat WebSocket connects to the API directly, so the API needs a name a browser can resolve and a certificate of its own.

app.example.com and api.example.com is the shape. They can be any two names; what they must not be is one name with a path prefix, because the API's cookies and the site's are scoped to the host.

Podman support¶

Podman is supported through the manual deployment path with Docker Compose as its provider. The runtime checks used Podman 6.1.4 and Docker Compose v2.40.3 on Fedora CoreOS 44 (a macOS Podman machine, ARM64, cgroups v2, SELinux enforcing). The shipped file needs Compose 2.24 or later.

Do not use podman-compose 1.6.0. It started the stack and an agent answered, but a controlled failed migration still started the dependent app. Docker Compose correctly refused that start. The providers' version numbers are independent; podman-compose 1.6.0 is not Docker Compose 1.6.0.

The quickstart remains Docker-only and directs detected Podman to this manual path, including with --no-sandbox. From the configured repository root, choose the actual Docker Compose executable (a standalone binary or CLI plugin):

export PODMAN_COMPOSE_PROVIDER=/absolute/path/to/docker-compose
podman compose version
podman compose -f docker-compose.yml up -d

Check that podman compose version identifies Docker Compose. Using -f explicitly keeps the clone's source-build override out of this published-image installation. Follow Install to bootstrap the organization and model after the services start.

Optional sandbox. pydantic-ai-backend 0.2.39 worked against both rootful and rootless Podman sockets. It does not need a new backend or release. Use docker-compose-podman.yml to adapt the socket and workspace mounts; prepare their permissions before enabling the profile.

On native Linux with rootless Podman:

systemctl --user enable --now podman.socket
export SANDBOX_DOCKER_SOCKET="$XDG_RUNTIME_DIR/podman/podman.sock"
export SANDBOX_WORKSPACE_ROOT="$HOME/.local/share/agenticos/sandbox-workspaces"
mkdir -p "$SANDBOX_WORKSPACE_ROOT"
podman unshare chown 10001:10001 "$SANDBOX_WORKSPACE_ROOT"
export DOCKER_GID=0

For rootful Linux, enable the system podman.socket instead, set SANDBOX_DOCKER_SOCKET=/run/podman/podman.sock, and prepare the workspace with sudo install -d -o 10001 -g 10001 "$SANDBOX_WORKSPACE_ROOT". Run Compose against that rootful engine. Its socket group is normally 0; a customized rootful socket needs its actual group in DOCKER_GID.

On a rootless macOS Podman machine, both bind sources belong to the Linux VM, not to macOS. Provision them inside that VM:

export SANDBOX_DOCKER_SOCKET="$(podman machine ssh 'printf "%s/podman/podman.sock" "$XDG_RUNTIME_DIR"')"
export SANDBOX_WORKSPACE_ROOT=/tmp/agenticos-sandbox-workspaces
podman machine ssh 'mkdir -p /tmp/agenticos-sandbox-workspaces && podman unshare chown 10001:10001 /tmp/agenticos-sandbox-workspaces'
export DOCKER_GID=0

Keep the deployment's existing keys. Set a nonempty SANDBOXD_TOKEN and SANDBOX_URL=http://sandboxd:8080 in the env file the stack reads, as described under sandbox configuration. Then start with both files:

podman compose -f docker-compose.yml -f docker-compose-podman.yml \
  --profile sandbox up -d

The override disables SELinux labeling only for sandboxd, which already holds the engine socket. It shares the workspace label with :z because sibling sandboxes also mount it. Sandbox containers retain their own labels and no-new-privileges; do not relabel the socket or disable SELinux globally. Rootless socket ownership maps to group 0 inside the user namespace.

Use a persistent workspace path on a server: /tmp is only a local example. Persist the socket, workspace and group settings in the operator's environment for subsequent Compose commands. A host DOCKER_HOST does not change the socket mounted inside sandboxd.

Verified paths include an HTTP agent run, persisted completion, workspace files, nonzero exits, timeouts, enforced memory/CPU/PID limits and runtime image builds. The shipped workbench image also read/wrote XLSX, rendered a plot and ran Node and LibreOffice. The model was a deterministic OpenAI-compatible test server; the desktop runtime and every Linux distribution were not tested. See the recorded checks for versions, scope and the migration failure reproduction.

Sizing the host¶

Measured on an idle deployment, not estimated:

	at rest	ceiling
`app` (2 uvicorn workers)	~1.0 GB	2.5 GB at the default 4 workers
`db`	~1.3 GB with the tuning below	2 GB
`worker` (one ingest child)	387 MiB	2.5 GB (`WORKER_MEMORY_LIMIT`)
`frontend`	~300 MB	1 GB
`redis`	9 MiB	512 MB

The number that decides the host is UVICORN_WORKERS. Each worker is a separate process that imports the whole application — 460 MiB, spawned rather than forked, so nothing is shared. Four of them are 1.9 GB before a request arrives.

Two workers suit a team of ten and still leave one serving while the watchdog replaces a wedged sibling. One worker is the setting to avoid: a blocked event loop is then the whole deployment, until it kills itself.

A host short of CPU shows a different symptom: Child process [N] died lines in the app log with no watchdog verdict above them, while every page is slow. Those are workers uvicorn's supervisor killed for answering its ping late, not workers that failed. It waits UVICORN_WORKER_HEALTHCHECK_TIMEOUT seconds, 60 by default. Look at the host's CPU steal before you raise it.

Two settings keep a slow deployment diagnosable:

Every log is capped. Each service in the production compose files writes at most five files of 20 MB. Docker's default keeps a container's log until the container is removed, and nothing on the host rotates it.
Postgres records what each statement costs. The db service preloads pg_stat_statements and a migration creates the extension. Where the database role may not create it, the migration logs a notice and goes on. agenticos cmd doctor prints a query statistics line either way.

The database is tuned against its own limit

docker-compose-prod.yml runs Postgres with shared_buffers=512MB against a 2 GB limit, and gives it 512 MB of /dev/shm — Docker's default is 64 MB, which a parallel scan over a collection's vectors exhausts, reporting could not resize shared memory segment. Move the limit and move the tuning with it; they are written next to each other for that reason.

It also sets max_connections=200, because the pools can open more than Postgres' stock 100: two pools of 15 per uvicorn worker, so 120 at the default four, and 30 more for the background worker. Raise it with UVICORN_WORKERS or DB_POOL_SIZE, or Postgres refuses connections with "too many clients" before a pool can queue them or shed the load.

Point the names at the host¶

Two A records, before anything else. Let's Encrypt proves you control a name by fetching a file over HTTP from wherever it resolves, so a certificate is not issued until this is true and propagated.

app.example.com   A   203.0.113.10
api.example.com   A   203.0.113.10

A wildcard does not do this for you

Where *.example.com already points somewhere — a marketing site, usually — both names resolve to it. A record for the specific name beats a wildcard, so the fix is to add the two above rather than to remove the wildcard.

Check from somewhere that is not the host, because the host may have its own answer:

dig +short app.example.com api.example.com

Get it onto the host¶

sudo install -d -o "$USER" -g "$USER" /opt/agenticos
git clone https://github.com/vstorm-co/agenticos.git /opt/agenticos
cd /opt/agenticos
bash scripts/server-init.sh

server-init.sh writes backend/.env: it generates the five secrets, asks for the two hostnames and an address for Let's Encrypt, and derives the public URLs and the CORS origin from what you gave it. It asks for no provider key: those live in each organization's vault. It refuses to overwrite an existing file.

The clone is where the compose files and that env file live; no code runs from it. What runs is the two images the repository publishes.

The images¶


`ghcr.io/vstorm-co/agenticos-backend`	The API, the worker and the migrations - one image, three commands
`ghcr.io/vstorm-co/agenticos-frontend`	The console

Both are built for linux/amd64 and linux/arm64 by .github/workflows/images.yml. A release (v0.0.380) publishes 0.0.380 and moves latest; every commit on main publishes edge and sha-<short>. The compose files read the tag from AGENTICOS_VERSION in backend/.env and default to latest.

Three rules the workflow keeps, each worth knowing before relying on a tag:

Only a commit on main is ever published. A v* tag pushed from a branch, or a run dispatched on one, is refused before anything is built - so latest cannot get past the pull-request boundary.
A release on a commit main already built is not rebuilt. Its sha-<short> manifest gets the version and latest as extra names, so the digests a host pinned to are exactly the ones the release names.
A commit with no images can be given them. Run the workflow by hand with its sha input - gh workflow run images.yml --ref main -f sha=<commit> - and it publishes that commit's sha-<short> tag and nothing that moves. That is the path for a commit older than the workflow, and for one whose run was lost.

Pin a release on a host you care about

AGENTICOS_VERSION=0.0.380 in backend/.env, so that make prod on a bad day pulls what ran yesterday rather than whatever was released this morning. scripts/deploy.sh pins for you - to the sha- tag of the commit it deploys - for exactly as long as the deploy runs.

Pin it from that release's checkout: git checkout v0.0.380 beside the variable. The sandbox service's image is pinned in the compose file, at the version the backend of the same commit speaks, so an older backend run from a newer checkout meets a newer sandboxd - and latest from an old checkout is the same mismatch the other way round. scripts/deploy.sh checks out the commit it deploys, so a scripted deploy or rollback keeps the two together.

Both packages pull anonymously. If a pull answers unauthorized, the package has been made private or a stale docker login ghcr.io is in the way; neither is something a host can fix by itself.

backend/.env holds the key that unwraps every stored credential

VAULT_MASTER_KEY is what makes an organization's provider keys, bot tokens and MCP credentials readable. Losing it does not lock you out of the product; it makes every secret in it unrecoverable. Back the file up somewhere a lost disk cannot take with it, and rotate with agenticos cmd vault-rotate rather than by editing.

Two optional things it does not ask about, both in that file: SMTP_*, without which invitations and password resets cannot be sent, and LOGFIRE_TOKEN, which is where traces of agent runs go.

Choose a reverse proxy¶

Something has to terminate TLS and route the two names. Both options below reach the same containers; pick on whether you already run one.

Leave compression to the application

The API gzips its own answers, the console's /api/* proxy compresses again for the browser whatever the API compressed, and Next compresses its pages. So the proxy needs no gzip on, and it must not blank the client's Accept-Encoding: that only turns the compression off. A response already carrying Content-Encoding passes through Nginx and Traefik unchanged.

Option A: Traefik¶

The shorter path, and the one to pick on a host that already has Traefik: the containers carry labels, Traefik discovers them, asks for the certificate and renews it. Nothing to reload and no second config file to keep in step.

If Traefik is not there yet, the repository ships one: traefik/traefik.yml and docker-compose-traefik.yml, which is an entrypoint on 443 with a Let's Encrypt resolver and 80 redirecting to it.

docker network create traefik_webgateway
docker compose --env-file backend/.env -f docker-compose-traefik.yml up -d

Where Traefik is already running, leave those alone and point TRAEFIK_NETWORK at the network it watches. The overlays read that name, so nothing about the existing proxy has to change.

Then bring the stack up with PROXY=traefik, which adds the two overlay files that carry the labels:

make prod PROXY=traefik
make prod-frontend PROXY=traefik

server-init.sh has already written PROXY=traefik into backend/.env, which is where scripts/deploy.sh reads it from — so later deploys keep the proxy this host was set up with rather than the one a script assumed.

exposedByDefault: false is doing real work

It is the one setting in traefik/traefik.yml worth reading before you run it. Only app and frontend carry traefik.enable=true, so Postgres, Redis, the worker and the sandbox daemon are reachable from nothing outside the host — and that is a property of not being labelled, so it survives somebody adding a service without thinking about the proxy.

Some request paths and query strings are credentials

These path segments open something to whoever reads them:

a public link's key, in /a/<key>, /api/v1/public/artifacts/<key> and /api/v1/artifact-embed/<key>, until the link is rotated;
a signed content address, /api/v1/artifact-content/<token>, for its lifetime;
a shared conversation's token, in /shared/<token> and /api/v1/conversations/shared/<token>;
a workflow form's token, in /f/<token> and /api/v1/public/workflow-forms/<token>;
a webhook test's token, in /api/v1/workflow-webhook-tests/<token> and /api/v1/workflows/<id>/webhook-test/<token>;
a channel link's token, in /link/<token> and /api/v1/me/channel-link/<token>;
an invitation's token, in /invitations/<token>, /api/v1/invitations/<token> and /api/v1/invitations/<token>/accept;
a workflow run's resume token, in /api/v1/workflow-resume/<run_id>/<token>.

So do these query parameters:

visitor and token on the agent widget's socket, /api/v1/embed/<key>/ws: a hosted page's key, which resumes the visitor's conversation, and the customer's signed visitor token;
code and state on a sign-in provider's return, /api/v1/oauth/<provider>/callback, and code on the console's /auth/callback;
invitation_handle on /api/v1/oauth/<provider>/login and /api/v1/auth/kerberos/login, which redeems the invitation it stands for;
token on the console's /reset-password, /auth/magic-link and /auth/email-change, the links that a password reset, a sign-in link and an email change send.

A request made from one of those pages can carry its address in the Referer header. The API replaces the API segments and the values of those parameters with [REDACTED] in its own access log, in uvicorn's line for each accepted WebSocket, in every log record and in Logfire spans, the console output, the endpoint's arguments and the copy Logfire makes when a span starts included. Other query parameters, such as q, skip, limit and ids, are kept.

Traefik can keep, drop or redact a whole field but cannot rewrite part of one, so traefik/traefik.yml drops RequestPath, which includes the query string, from Traefik's access log. Nginx can rewrite part of a value, but nginx/nginx.conf drops the path, the query string and the Referer the same way, with the agenticos_no_path log format each server block names, so the list of secret paths is kept in one place. Both logs still have the client, the status, the size and the duration. Nginx's error log writes the request line and the Referer of a failed request, and no setting changes that, so keep the error log on the host. If you need the paths in the access log, remove that field rule or log format and handle the log as a secret: keep it on the host and do not ship it to a log service.

Option B: Nginx¶

For a host where Nginx already terminates TLS, or where the proxy is not in Docker at all. The stack publishes both ports on 127.0.0.1 and Nginx reaches them there:

make prod
make prod-frontend

nginx/nginx.conf is the template. Two substitutions before it serves anything: the server_name in each block is ${DOMAIN:-localhost}, and Nginx does not expand that — put the two hostnames in by hand. Certificates are yours to obtain and renew, and so is the Strict-Transport-Security header, which the backend deliberately leaves to whatever terminates TLS. The template's access log leaves out the request path, the query string and the Referer; Some request paths and query strings are credentials, above, says why and what the error log still writes.

BIND_HOST is a security setting, not a convenience

The loopback default is what makes the auth rate limit mean anything. RATE_LIMIT_TRUST_FORWARDED_FOR tells the API to count an attempt against the address the proxy forwards, so whatever can reach the API past the proxy chooses the address its attempts are counted against. Set BIND_HOST=0.0.0.0 only for a proxy on a different machine, and firewall the port to it.

The frontend is a compose project of its own

Compose names a project after the directory, so both stacks were agenticos — and bringing the frontend up then reported the five backend containers as orphans, with compose's own suggestion to run the command again with --remove-orphans. Taking that advice stops the API, the database, Redis and the worker. The make targets and scripts/deploy.sh pass -p agenticos-frontend, so the warning is gone. The compose files fix no container names either - each project names its own, so two stacks on one host cannot take each other's containers over, and deploy.sh waits on the app and frontend services rather than on a name. A deployment that predates both fixes is recreated under the new names on its next up; nothing has to be removed by hand.

The two projects still meet on a network with a fixed name - agenticos_edge in production, agenticos_backend on the dev server - because the frontend joins it as an external network. A host running two AgenticOS stacks sets AGENTICOS_EDGE_NETWORK and AGENTICOS_DATA_NETWORK (or AGENTICOS_NETWORK) apart in each stack's backend/.env; otherwise both stacks' db, redis and app resolve on one bridge, and a request can reach the neighbour's database.

Start it, and create the first account¶

make prod pulls the images, starts the stack and runs the migrations - the last as a migrate service the API waits on, so a docker compose up -d by hand on the same files does the same. The first pull is about 2 GB; a later one is the layers that changed.

Then create an organization, an owner and a working agent:

docker compose --env-file backend/.env -f docker-compose-prod.yml \
  exec -T app agenticos cmd bootstrap \
  --email you@example.com --password 'a real password' \
  --org 'Your Company' --provider anthropic --api-key sk-ant-...

The provider key here is what the two starting agents run on. Without it they are created and cannot answer; every other provider is added in the product, per organization, from the vault.

Check it from outside, not from the host

curl -fsS https://api.example.com/api/v1/health
curl -fsSo /dev/null -w '%{http_code}\n' https://app.example.com

A stack that is healthy on the host and unreachable from the internet is DNS, the firewall or the certificate — three things a health check inside the host cannot see.

Then, once, by hand: sign in, invite somebody (which proves SMTP_*), and send AgenticOS a message (which proves the provider key and the WebSocket). Each exercises a path nothing else here checks.

The security headers come from the backend, so any proxy is covered

A Content-Security-Policy, X-Frame-Options: DENY, X-Content-Type-Options: nosniff, Referrer-Policy and Permissions-Policy are set on every response — including the 500 for an unhandled exception, which is built outside the middleware stack and stamps them itself. The interactive API docs drop the CSP only, because Swagger loads assets a strict policy forbids.

HSTS is deliberately left to the proxy, which is where TLS terminates. A proxy that sets its own CSP should be at least as strict as this one.

Services on your own network¶

A request a member configures - a workflow HTTP step, a Git, website or S3 source, an A2A agent or an MCP connection - is refused when its host resolves to a private address, so a wiki, a Git server or an MCP server inside your network is out of reach by default. To open one, list it in OUTBOUND_PRIVATE_ALLOWLIST in backend/.env, as a host, a .suffix, an address or a CIDR range, and restart the API and the worker. See the setting's row in the configuration reference for the forms, what is refused whatever you list and what every account can then reach. List only services that check their own credential.

agenticos cmd doctor reports the list as its private network line. It fails when a listed range covers the database or Redis, and warns about a range wider than /16 (IPv4) or /48 (IPv6).

Turning the sandbox on¶

The service that runs an agent's code is behind a compose profile, because it is the one container holding the Docker socket and mounting that on a shared host should be a decision rather than a default. Three things, once:

make sandbox-token                       # writes SANDBOXD_TOKEN to backend/.env
sudo mkdir -p /var/lib/agenticos/sandbox-workspaces
sudo chown 10001:10001 /var/lib/agenticos/sandbox-workspaces

Then a deploy brings it up: scripts/deploy.sh passes --profile sandbox when SANDBOXD_TOKEN in backend/.env has a value, so the host itself says whether it runs one. Like make and quickstart, it exports DOCKER_GID into the sandbox's group_add: the socket's group on Linux (following symlinks), or 0 on Docker Desktop, where the mounted socket belongs to the VM. An explicit DOCKER_GID in the environment or on the make command line takes precedence. Nothing else in .env is needed: the backend reaches the daemon through a sandbox connection somebody creates in the console, and http://sandboxd:8080 is recognised as this deployment's own.

A profile compose is not told about is a service compose stops

up -d on the same project without --profile sandbox does not leave the sandbox alone — it stops it. So a host that started it by hand had it taken away by its next deploy, with an agent's code execution failing for reasons nowhere near the deploy that caused it (#1506). That is why the script reads the token rather than taking a flag.

Deploying a change¶

Native memory and routine continuity¶

Before this upgrade, take the database, media and vault-key backups and coordinate the API and worker rollout. Use the deployment's normal migration step; do not copy rows or create replacement schedules manually.

Before applying 0285_member_proactivity.py, stop API/sweep producers and drain active proactive jobs, then stop workers. The migration refuses while a watch or nightly model job is running and backfills queued work with a shared per-member concurrency key. Update API and workers together so new jobs use the same key; a watch and a nightly look for one person must not independently spend the same remaining allowance.

0248_drop_memory_mem0.py refuses the upgrade while memory_mem0 remains in any stored draft or published spec. Before retrying, inventory every endpoint and vault credential mapping from the backup, erase the external memories and verify completion at each provider. Only then remove those bindings explicitly from agents.draft_spec and agent_versions.spec. An export alone is not an erasure. Keep the cleanup evidence, review unused vault keys and revoke retired credentials. The migration makes no remote request and never assumes a purge succeeded. If an earlier development build already stripped the bindings, recover the inventory from its pre-upgrade backup and perform the same purge; native person/agent erasure cannot reach that external data.
0251_primary_conversations.py and 0258_primary_channel_history.py establish primary private chats. An existing mapping wins. Only demonstrably private, linked channel history is adopted once; project/group and ambiguous histories remain separate. Do not rerun the backfill after somebody clears a primary chat.
Existing routine IDs, schedules and logs remain in place. The delivery migration 0255_trigger_delivery.py gives old routines run_log; new routines default to a primary chat or their trusted source conversation. Review the destination, acting person, end and budget under Routines, rather than creating copies.
0259_trigger_version_pin.py supports immutable version pins. A pin and a named environment are exclusive. Clear a pin before deleting its version. A parked run retains its admitted version; 0260_routine_run_origin.py retains its origin so deleting a routine does not release its authorization guard.
0261_watch_attempt_status.py starts explicit attempt tracking without guessing older success/failure history. An unavailable or incomplete check does not advance the success timestamp. Review existing watches and their account access: historical rows do not prove how consent was originally given.

Choose a model explicitly before enabling memory maintenance; the daily briefing keeps the existing organization model/budget settings. With 0262_watch_agent_sources.py, watches use their selected agent and retain approved sources on runs. Existing watches require the owner to select an agent and approve its sources before running again; IDs and findings are preserved. Check governance and the channel setup, including PUBLIC_BASE_URL and expired SMS identity links. In a test organization, verify one private chat, one routine's Activity/delivery, pause/expiry and one unavailable-source watch before enabling a wider rollout.

The retry-history boundary in 1.25.0¶

Version 1.25.0 adopts Pydantic AI 2.61's retried tool outcomes and separate retry-feedback messages. Existing histories and run checkpoints load without a database migration or republishing agents. Histories containing these parts need the new reader: an older backend cannot replay these message shapes.

Upgrade all API and worker processes together, draining or stopping the old processes before the new ones accept work. Do not mix older and newer backend replicas against the same database. After either new retry form has been written, a code-only rollback across this boundary is unsupported; retaining the new reader or restoring a pre-upgrade database snapshot is required.

By hand¶

remote=$(ssh you@your-host 'mktemp -t agenticos-deploy.XXXXXX')
ssh you@your-host "cat > $remote" < scripts/deploy.sh
ssh you@your-host "trap 'rm -f $remote' EXIT; bash $remote <commit-sha>"

scripts/deploy.sh fetches that commit, waits for the images CI published for it (sha-<short>, usually already there), pulls them, restarts, and waits for both containers to report healthy before it returns non-zero or not. It takes a commit rather than a branch, so what is deployed is what was reviewed, not whatever main has moved to since - and what runs is byte-for-byte what CI built, on a host that never needs the toolchain.

Copy it to the host, then run it — do not pipe it into bash -s

Under bash -s the script is the shell's own standard input, and the first command in it that reads stdin consumes the rest. docker compose exec forwards stdin to the container even with -T, so the migration ate everything below itself, bash reached EOF, and the deploy exited 0 having never built the frontend or waited for any container. The site was down and the deploy was green (#1488).

Two connections instead of one is what stops the procedure being able to truncate itself.

It is not zero-downtime. Compose recreates the containers whose image changed, so the site is unavailable for the few seconds that takes.

From GitHub, with an approval¶

.github/workflows/deploy.yml offers each published stable release for deployment and waits for somebody to approve it. That gate is a repository setting, not a step in the file — without it, the workflow deploys each published stable release unattended.

Set it up once:

Settings → Environments → New environment, named production.
Tick Required reviewers and add whoever may approve. This is the gate.
Add the environment's variables: SITE_URL, API_URL, and APP_DIR if the checkout is not at /opt/agenticos.
Add the secrets below.

Cancel a deploy you do not intend to approve

Every run shares the deploy-production concurrency group, and a run sitting at the approval gate holds it. It does not expire on its own — GitHub cancels an unactioned one after 30 days — so until somebody approves or cancels it, later releases queue behind a decision nobody is going to make, and the server keeps running whatever was deployed last.

So a deploy you have decided against is cancelled, not left. One left waiting on a superseded commit blocked three later runs here before anybody noticed the queue rather than the runs.

deploy-queue.yml is what happens when it is left anyway: every six hours it cancels a Deploy run that has waited more than twelve for an approval, and opens an issue naming the run and how far the latest stable release has advanced from the last successful deploy. It approves nothing and deploys nothing — it drains the queue so the next published release reaches the gate, and says that it did. Run it by hand with gh workflow run deploy-queue.yml.

Approving from the environment's queue approves the oldest run

Which is the stale one. A late approval given there deploys the commit from whenever the queue jammed, not the commit that was just pushed — that is how a fortnight-old commit reached this server. Approve from the run page of the commit you mean, or start a fresh one with gh workflow run deploy.yml --ref main -f ref=v1.34.3 and approve that.

Secret	What
`DEPLOY_HOST`	The host's address
`DEPLOY_USER`	The account the checkout belongs to
`DEPLOY_SSH_KEY`	A private key whose public half is in that account's `authorized_keys`
`DEPLOY_KNOWN_HOSTS`	`ssh-keyscan your-host`, run from somewhere you trust

Generate the key for this and nothing else:

ssh-keygen -t ed25519 -N '' -C 'github-actions-deploy' -f deploy_key
ssh-copy-id -f -i deploy_key.pub you@your-host
ssh-keyscan your-host                    # → DEPLOY_KNOWN_HOSTS
cat deploy_key                           # → DEPLOY_SSH_KEY, then delete it locally

The host key is a secret rather than a ssh-keyscan at deploy time

Scanning at deploy time trusts whatever answers on that address, which is the thing a host key exists to prevent. Scan once, from somewhere you trust, and store the answer.

A run then appears with Review deployments; approving it starts the job. workflow_dispatch runs the same job against a ref you name, which is how a rollback is done, and it passes through the same approval.

Backups¶

One volume matters, and it is not obvious which:

Volume	Holds	Backup
`postgres_data`	everything — agents, conversations, sealed credentials	yes
`media_data`	uploaded files, before ingestion	yes

The cache has no volume at all. Rate-limit buckets, channel dedupe claims and membership answers all carry a TTL and all rebuild themselves, so Valkey runs with --save '' and starts empty after a restart. A host that ran an earlier release still carries an agenticos_redis_data volume that nothing mounts any more; docker volume rm agenticos_redis_data removes it.

docker compose --env-file backend/.env -f docker-compose-prod.yml exec -T db \
  sh -c 'pg_dump -U "$POSTGRES_USER" -Fc "$POSTGRES_DB"' > "agenticos-$(date +%F).dump"

The identifiers come from the container's own environment rather than being written out, because both are settings: a deployment that changed either would otherwise get an empty file and an error nobody reads on the way past.

A database backup without backend/.env is not a backup

The credentials in it are sealed with VAULT_MASTER_KEY. Restored next to a different key, every provider key, bot token and MCP credential in the dump is unreadable — and the product will tell you so one refusal at a time.

Rolling back¶

	How
Code	Deploy the previous commit: `workflow_dispatch` with its sha, or `scripts/deploy.sh`. The images are still in the registry, so this is a pull, not a build. A commit with no `sha-<short>` images - older than `images.yml`, or its run lost - is published first with `gh workflow run images.yml --ref main -f sha=<commit>`; the deploy names that command when it gives up waiting
Schema	`agenticos db downgrade --revision=-1`, then deploy the code that matches
Data	`pg_restore` the dump, then check the migration the code expects

Rolling code back across a migration is a decision, not a command. The old code meets a schema it has never seen; whether that works depends on the migration. Read it before assuming.

Recap¶

One host, Compose, a proxy in front. Seven containers - one of them runs the migrations and exits - two of them stateful, every one pulled - nothing is built on the host. Pin AGENTICOS_VERSION.
UVICORN_WORKERS decides what the host costs. 460 MiB per worker, nothing shared. Two for a team, four for real traffic.
DNS before everything. No certificate is issued until the names resolve to the host.
The approval is a repository setting, not a line in the workflow. Without required reviewers on the production environment, every merge deploys itself.
Back up postgres_data and backend/.env together. Either without the other is not a restore.