Configuration

Every service reads a HOCON config file baked into its JAR, and every value you are meant to change is overridable by an environment variable. The pattern is always the same:

database {
    url = "jdbc:postgresql://localhost:5432/tracedown"
    url = ${?DATABASE_URL}
}

The second line only takes effect when the variable is set, so the file default stands unless you override it. This page lists what is actually wired up. If a setting is not on this page, it is a HOCON value with no environment override, and changing it means rebuilding — the notable cases are called out in place.

The defaults are tuned for a single-host development stack: Postgres on localhost, an empty password, and secrets that are placeholders. They are not production values. See Secrets & Encryption for what has to change before you expose the stack.

Common to most services

These are read by nearly every JVM service under the same names, which is what lets the Compose stack define them once and share them across containers.

Variable	Purpose	Default	Required
PORT	Ktor listen port	Per service — see below	No
DATABASE_URL	JDBC URL	jdbc:postgresql://localhost:5432/tracedown	No — except realtime-service and schema-migrator
DATABASE_USER	Database user	tracedown	Same as above
DATABASE_PASSWORD	Database password	(empty)	Same as above
REDIS_A_URL	Operational Redis (AOF) — outbox, sessions, queues	redis://localhost:6379	No — except realtime-service
REDIS_B_URL	Ephemeral cache Redis — metrics, rate limits	redis://localhost:6380	No
DB_POOL_SIZE	HikariCP maximum pool size	10	No
JAVA_TOOL_OPTIONS	JVM heap sizing in constrained containers	(unset)	No

Each service has its own default port, so a stock stack has no collisions:

Service	Default port
api-gateway	20714
probe-scheduler	20810
result-ingestor	20820
notification-dispatcher	20830
email-service	20840
metrics-service	20850
aggregate-worker	20860
realtime-service	20870

Not every service uses every variable. result-ingestor connects to Redis A only. email-service needs no Postgres at all — it is a pure queue consumer.

DB_POOL_SIZE does not apply everywhere

DB_POOL_SIZE is read directly via System.getenv, not through HOCON. Services that pass an explicit pool size in code ignore it entirely: aggregate-worker, metrics-service and realtime-service each pin 5 connections. The resource-limits overlay sets DB_POOL_SIZE for them anyway, which is misleading — changing it there has no effect.

It is the pool size for api-gateway, result-ingestor and notification-dispatcher. probe-scheduler is the odd one: its pool is derived from its dispatch concurrency, and DB_POOL_SIZE (or SCHEDULER_DB_POOL_SIZE) only overrides that derivation — see probe-scheduler below. Setting DB_POOL_SIZE once for the whole stack therefore silently caps the scheduler at a value its workers cannot live on.

Connection budget — 103 idle, against a default of 100

HikariCP fills to its maximum eagerly and holds the connections idle for the life of the process, so a pool size is a reservation, not a ceiling you might one day reach. Budget it as spent:

Service	Pool
api-gateway	10
result-ingestor	10
notification-dispatcher	10
probe-scheduler	58 (50 dispatch workers + 8 headroom)
metrics-service	5
aggregate-worker	5
realtime-service	5
email-service	0 — holds no database
Total	103

103 is above PostgreSQL's default max_connections of 100, so a stock database cannot start this stack: some services take their pools and the rest fail to acquire and exit. The bundled Compose file runs Postgres with postgres -c max_connections=160 for exactly that reason — keep it if you substitute your own database.

The scheduler is over half the budget on its own, and it is the pool most often forgotten because it appears in no DB_POOL_SIZE setting. 160 leaves ~57 spare for everything else that connects — a psql session, the pg_dump in Backup & Restore, and above all replicas. A second gateway plus a second scheduler adds 68, more than the spare, for 171 against 160. Raise max_connections before scaling out, or lower SCHEDULER_DISPATCH_WORKERS and let the pool follow it. The full arithmetic is in Scaling.

The production guard

Variable	Purpose	Default	Required
DEPLOYMENT_ENV	Deployment environment name; production arms the startup guard	dev	No — but set it in production
ALLOW_INSECURE_DEV_KEYS	Disables the guard even in production	(unset)	No

With DEPLOYMENT_ENV=production, four services refuse to start on published development secrets. This is the single most important production setting on this page: it converts "you forgot to change a secret" from a silent liability into a startup failure.

Service	What it refuses in production
api-gateway	The all-zero PLATFORM_AES_KEY, the default JWT_SECRET
probe-scheduler	The all-zero PLATFORM_AES_KEY
notification-dispatcher	The all-zero PLATFORM_AES_KEY
email-service	EMAIL_PROVIDER unset or console — the dev transport logs reset links and invite tokens instead of sending them

ALLOW_INSECURE_DEV_KEYS=true switches those guards off; it exists for test rigs, and it logs a warning naming itself on every start.

Only the exact string production arms the guard

The value is trimmed and lowercased before the comparison, so PRODUCTION and production are fine. Everything else is not: prod, prod-eu, staging, a typo, or an unset variable all leave every guard disarmed and the service running on whatever secrets it was given.

Because that failure is silent by construction, every service that comes up unguarded says so at startup — including the ones with no insecure defaults of their own, so the report is uniform across the fleet:

• A value that is not production logs a WARN naming the value it read and stating that the guards are not armed.
• A value that was clearly reaching for production — anything beginning prod or prd, plus exactly live — logs an ERROR instead, naming the literal you have to set. A near-miss is a typo with security consequences, not an ordinary dev run.
• production with ALLOW_INSECURE_DEV_KEYS set logs an ERROR saying the guards are disabled and which dev defaults are still in place.

A correctly guarded service says nothing, which is the one case that needs no attention. Grep your boot logs for DEPLOYMENT_ENV after any deployment change: silence across the fleet is the answer you want.

The bootstrap credentials are guarded separately and more strictly — see the first account below. ALLOW_INSECURE_DEV_KEYS does not lift that one.

api-gateway

The gateway is the only service you expose. It terminates the API, issues sessions, and owns organisations, workspaces, projects, services, variables, webhooks, invites and agent registration.

Connectivity

Variable	Purpose	Default	Required
PORT	Listen port	20714	No
DATABASE_URL / DATABASE_USER / DATABASE_PASSWORD	Postgres connection	See common table	No
REDIS_A_URL	Operational Redis	redis://localhost:6379	No
REDIS_B_URL	Cache Redis — backs rate limiting	redis://localhost:6380	No
REDIS_C_URL	Resource hierarchy cache	(empty — disabled)	No
REDIS_C_TTL_SECONDS	Hierarchy cache entry TTL	3600	No

Redis C is an optional third role that caches the org → workspace → project → service hierarchy. Leaving REDIS_C_URL empty disables the cache and the gateway resolves the hierarchy from Postgres — correct, just less cached. Small deployments should leave it off.

Security

Variable	Purpose	Default	Required
JWT_SECRET	Reserved token-signing secret — see Secrets & Encryption	default-dev-secret-change-in-production	No — but change it
JWT_TTL_MINUTES	Session lifetime	43200 (30 days)	No
PASSWORD_MIN_LENGTH	Minimum password length	8	No
PASSWORD_MIN_UPPERCASE	Minimum uppercase characters	1	No
PASSWORD_MIN_DIGITS	Minimum digits	1	No
PASSWORD_MIN_SPECIAL	Minimum special characters	1	No
TOTP_ISSUER	Name shown in authenticator apps	Tracedown	No
RATE_LIMIT_ENABLED	Per-IP sliding-window limiting via Redis B	true	No
RATE_LIMIT_GENERAL_MAX	Requests per window, general endpoints	120	No
RATE_LIMIT_GENERAL_WINDOW	General window, seconds	60	No
RATE_LIMIT_AUTH_MAX	Requests per window, auth endpoints	15	No
RATE_LIMIT_AUTH_WINDOW	Auth window, seconds	60	No
TRUSTED_PROXIES	Trusted proxy hops when deriving the client IP for rate limiting	1	No
API_CORS_ORIGINS	Browser origins allowed to call the API, comma-separated	(unset — no CORS headers)	No — unless the dashboard is on another origin

The password minimums compose rather than replace: the length floor is 8 and within it at least one uppercase, one digit and one special character must appear. Auth endpoints get a tighter budget than general traffic because they are the ones worth brute-forcing.

TRUSTED_PROXIES is how the gateway decides which X-Forwarded-For hop is the real client. The default of 1 matches the single host web server the deploy stack expects in front of the gateway; set it to your actual proxy depth, because a wrong value makes rate limiting either spoofable or keyed to your proxy's address.

Cross-origin access (API_CORS_ORIGINS)

Unset is the default, and it means no CORS headers are emitted at all. The gateway starts normally and logs one line saying so. That is the correct answer for every same-origin deployment — the bundled Compose stack, the single-process edition, the Vite dev server's /api proxy — where the app and the API are served from one origin and no request from the dashboard is cross-origin. It is also the safe reading of silence: no origin gains credentialed access because a variable was forgotten.

Set it only when the dashboard is served from a different origin than the API. It is a comma-separated list, each entry exactly scheme://host[:port]:

API_CORS_ORIGINS=https://app.example.com,https://ops.example.com

There is no wildcard, and there cannot be one: the dashboard sends credentials with every request, and a credentialed response may not answer Access-Control-Allow-Origin: *. It has to name the exact origin, which is why the origins are listed rather than inferred.

A configured value is validated at startup and a malformed entry is a boot failure naming the offending string — a trailing slash, a path, a query, a userinfo part or a scheme other than http/https. Only a configured value can fail this way; setting nothing is never an error.

The one line that tells you the variable is missing

A cross-origin deployment that never set the variable would otherwise learn about it only from a browser console. So the gateway watches for it: the first request that arrives carrying an Origin header pointing somewhere other than the host it was sent to logs a WARN naming both, and naming API_CORS_ORIGINS. It is said once per process — a latch, not a per-request warning — so grep the boot logs early. Authorities are compared, not schemes, because a TLS-terminating proxy forwards plain HTTP and the request's own scheme says nothing about the browser's.

Sessions are not JWTs

Despite the name, JWT_SECRET does not sign session tokens — sessions are opaque random tokens stored hashed in the database, and rotating this value does not log anyone out. It is guarded against its dev default in production and reserved for future signing use. See Secrets & Encryption.

Platform

Variable	Purpose	Default	Required
APP_URL	Frontend base URL used in emails	http://localhost:5173	No
URI_INVITE	Frontend invite route, appended to APP_URL	/invite	No
URI_PASSWORD_RESET	Frontend reset route	/reset-password	No
PLATFORM_AES_KEY	64 hex chars — encrypts secrets, signs challenges	64 zeros	No — but change it
SINGLE_ORG_MODE	Bootstrap a default org and user on first start — see The first account	true	No
DEMO_USER_EMAIL	Bootstrap user email	[email protected]	No
DEMO_USER_PASSWORD	Bootstrap user password	Down2trace!	No
INVITE_TTL_DAYS	Invite token lifetime	7	No
INVITE_RESEND_COOLDOWN_MINUTES	Minimum gap between invite resends	5	No
TRUSTED_DOMAIN_MODE	Skip domain-ownership checks (auto-verify all)	false	No
ALLOW_PROFILE_EDIT	Let users edit their display name	true	No
METRICS_PUBLIC_URL	Public metrics base URL shown in Grafana integrations	(empty)	No
GATEWAY_PUBLIC_URL	Public base URL of the gateway as probe agents reach it, e.g. https://tracedown.example.com; printed as PROBE_AGENT_SCHEDULER_URL next to every bootstrap token the dashboard or --agent-bootstrap issues. Unset, the dashboard prints the Compose stack's internal address with a warning	(empty)	No
AUDIT_LOG_RETENTION_DAYS	Days to keep audit log entries (enforced by the aggregate-worker; set identically in both)	90	No

APP_URL is what users click. It is the base for invite and reset links in outgoing email, so it must be the URL a browser can reach — not an internal container hostname — or your invites will land as dead links.

The first account

Tracedown is invite-only: every user after the first is invited from inside the app, and --create-org assigns an organization to a user who already exists. So SINGLE_ORG_MODE is the only path in Tracedown that creates a user at all. It is the first-account flow, and nothing else is.

It is on by default (platform.conf, singleOrgMode = true), because without it a fresh self-hosted install has no way to reach a first account at all. Against an empty user table the gateway creates a default organization and its owner from DEMO_USER_EMAIL and DEMO_USER_PASSWORD on start. Once that account exists the bootstrap is a no-op, so leaving the flag on does nothing further — but set SINGLE_ORG_MODE=false once you have your account, and add further organizations with the CLI:

java -jar api-gateway.jar --create-org <name> --owner <email>

The defaults [email protected] / Down2trace! ship in platform.conf on purpose: they are what makes a fresh checkout or a demo instance run with no setup at all. They are also, being committed, known to everyone.

Production refuses to bootstrap on the published credentials

With DEPLOYMENT_ENV=production and SINGLE_ORG_MODE=true, the gateway refuses to start unless both DEMO_USER_EMAIL and DEMO_USER_PASSWORD have been moved off their published values and the password passes the password policy above. The startup error names each thing that is wrong.

There is no override. ALLOW_INSECURE_DEV_KEYS lifts the production guard; it does not lift this. A weak key is a risk assessment — a password printed in a public repository is not.

The flag is not banned outright in production because it is the only way to create the first account; binding the refusal to the credentials closes the hole the flag actually opens, and leaves the account you do need reachable. When the credentials are real, the gateway logs a warning that the owner is about to be created and starts normally.

Domain trust

TRUSTED_DOMAIN_MODE is the single switch that decides whether Tracedown cares who owns the target of a probe. It defaults to false — verified mode — as a signal of good-faith use of the platform. Domains must be verified before they are probed freely, and probes against unverified domains are constrained to:

• a maximum of 3 calls per script,
• no body saving,
• a minimum 5-minute interval,
• no includes() — it would otherwise let a script scrape third-party response content.

Verified mode also reveals the Domains UI and enables the worker's DomainReverifyJob. The point is to stop the platform being pointed at infrastructure you do not control.

Set TRUSTED_DOMAIN_MODE=true to skip ownership checks and auto-verify every domain — convenient for a self-hosted install probing only infrastructure you own, at the cost of those protections. Set the same value on api-gateway, probe-scheduler and aggregate-worker — each reads it independently, and a split setting produces a stack that enforces the limits in one place and not another.

Automatic DNS setup

A user proving ownership has to put a TXT record in their zone. In the domains UI they can paste an API token for a supported DNS provider (Cloudflare today) and have the gateway write the record for them. The token is used for that one request — the zone lookup and the write — and is never stored, logged, or reused. There is nothing to configure: the option appears wherever the provider is reachable, and the record can always be added by hand instead.

The gateway also recognises the domain's DNS provider from its name-server delegation (DnsProviderProfiles — Cloudflare, Route 53, GoDaddy, Namecheap and a dozen more), walking up from api.example.com to the zone actually delegated. That costs one DNS lookup and needs no credential, so it works for providers we have no API client for: it names the provider and, where one exists, the page that edits that zone's records.

Where a recognised provider has an addressable DNS page, the domains UI offers an "Open DNS in " button that goes straight to it — the record is still pasted by hand, and no credential is involved.

A host application can replace that with something richer through the frontend's domain-dns-setup slot; when it does, the built-in button stands down rather than offering the same thing twice.

Retention

Variable	Purpose	Default	Required
PURGE_RETENTION_DAYS	Days after soft-delete before hard purge	0 (immediate)	No
RESULT_RETENTION_DAYS	Probe-result retention — caps the usage window	90	No

RESULT_RETENTION_DAYS is set in two places

The gateway uses it to bound the usage window; aggregate-worker uses it to decide what to actually delete. They are separate variables in separate services with the same name. If the gateway's value exceeds the worker's, the UI offers a window whose data has already been deleted. Keep them identical. See Retention & Aggregation.

Variables

Variable	Purpose	Default	Required
MAX_VARS_PER_RESOURCE	Most variables one resource may hold	100	No

Counted separately per resource — per organization, workspace, project, service and webhook — so a project at the cap does not stop its services having their own. Variables are read on every probe dispatch, so an unbounded set costs both storage and hot-path time; the cap bounds runaway or automated creation, and is the same number for every organization.

System-managed variables are not counted against it: the defaults seeded at organization creation, and the companion variables a config toggle creates. Enabling a feature never fails for want of room.

Deleting a variable frees its slot — the count is of live variables, not of everything ever created. A create beyond the cap is refused with variable_limit_reached.

Probe request limits

Variable	Purpose	Default	Required
REQUEST_TIMEOUT_MS	Outbound probe request timeout	30000	No
MAX_RETRIES	Maximum retries	10	No
PROBE_MAX_REDIRECTS	Maximum redirect hops	10	No

PROBE_MAX_REDIRECTS is one variable, not two: the gateway applies it when a script is saved and the probe-scheduler applies it when the script runs. Set it once for the deployment and both halves agree.

Seed data

Seeding only runs as part of the SINGLE_ORG_MODE bootstrap, so it is a first-boot-only affair.

Variable	Purpose	Default	Required
SEED_ENABLED	Seed demo data during bootstrap	false	No
SEED_PROJECT_NAME	Seeded project name	Default	No
SEED_SERVICE_NAME	Seeded service name	httpbin	No
SEED_TARGET_URL	Seeded probe target	https://httpbin.org/get	No
SEED_SCHEDULE	Seeded probe cron	*/5 * * * *	No

Default groups are not configurable by environment

Each new organisation gets four groups — Admins, Users, Viewers and DevOps — with per-section access levels (0 none, 1 read, 2 write). This is a HOCON list with no environment override; changing the defaults means editing platform.conf and rebuilding. Group membership and permissions are editable per-org in the UI afterwards, which is the intended path.

Body stores

The gateway owns body stores — the alternative locations agents may keep saved response bodies in — and reads a body back out of a store that keeps it in place.

Variable	Purpose	Default	Required
BODY_STORE_AES_KEY	64 hex chars — encrypts body-store secret keys at rest. Must be identical on result-ingestor	(unset)	Once a store exists
BODY_STORE_FILESYSTEM_BASES	Comma-separated directories a filesystem store's root may sit beneath	(unset)	No
BODY_STORE_PRIVATE_ENDPOINTS	Allow http:// and private, CGNAT, loopback, internal-suffix and single-label hosts as store endpoints	false	No

BODY_STORE_AES_KEY is a separate key from PLATFORM_AES_KEY, with the same 64-hex-character format, and it is shared with result-ingestor and nothing else. Without it the stores' credentials cannot be decrypted and every store call fails; the gateway and the ingestor log a WARN at startup when stores exist and the key is missing. See Secrets & Encryption.

BODY_STORE_FILESYSTEM_BASES has no default. Left unset, filesystem stores cannot be created at all — which is what you want unless you have a directory shared by the gateway, the ingestor and the agents. A store's root must be a directory beneath one of the bases; a base itself is refused. Set the same value on the ingestor.

BODY_STORE_PRIVATE_ENDPOINTS lifts the endpoint guard that otherwise refuses anything but a public https host. It exists for an object store on the same private network as the platform, it is a deployment-wide decision rather than a per-store one, and it must be set on the ingestor too. The shipped Compose stack sets it, because everything in it is on one Docker network. See Private endpoints.

Email

The gateway does not send mail itself. Invites and password resets are published onto the email_queue in Redis A, and email-service — the only process that opens a connection to a mail provider — renders and delivers them. So there are no provider settings here: configure mail once, on email-service, and every sender on the platform uses it.

With no provider configured, email-service defaults to console, which prints mail to its log instead of sending it. That is fine until you invite someone — the invite link then exists only in that log.

probe-scheduler

The scheduler owns cron evaluation and dispatch to agents. It talks to Postgres, Redis A, and the agents over mutual TLS.

Variable	Purpose	Default	Required
PORT	Listen port	20810	No
PLATFORM_AES_KEY	Decrypts variables and the CA key for its client cert	64 zeros	No — but change it
GATEWAY_URL	Gateway base URL, for health-challenge tokens	http://localhost:8080	No
SCHEDULER_SWEEP_INTERVAL	Consistency sweep interval, seconds	300	No
SCHEDULER_THREAD_POOL_SIZE	Quartz thread pool size	10	No
SCHEDULER_DISPATCH_QUEUE_SIZE	Dispatch queue depth	100000	No
SCHEDULER_DISPATCH_WORKERS	Concurrent in-flight dispatches	50	No
SCHEDULER_DB_POOL_SIZE	Overrides the derived JDBC pool size	(unset — derived, 58)	No
TRUSTED_DOMAIN_MODE	Skip domain-ownership checks (auto-verify all)	false	No
PROBE_DEFAULT_TIMEOUT_MS	Per-request timeout when a service has no override	30000	No
PROBE_MAX_TIMEOUT_MS	System-wide maximum timeout	300000	No
PROBE_MAX_REDIRECTS	Maximum redirect hops	10	No
PROBE_PAYLOAD_ENCRYPTION_ENABLED	Fleet-wide kill switch for per-agent payload sealing	true	No

PROBE_MAX_TIMEOUT_MS is a clamp, not a default — per-service overrides are capped at it, so it is the real ceiling on how long one probe can occupy a dispatch worker.

The scheduler's database pool is derived, not defaulted

Alone in the fleet, probe-scheduler sizes its JDBC pool from its own concurrency: SCHEDULER_DISPATCH_WORKERS + 8, so the stock 58. Every dispatch worker can want a connection at the same instant — that is what "concurrent dispatches" means — and a worker that cannot get one loses the probe outright rather than waiting for a slot.

DB_POOL_SIZE, or SCHEDULER_DB_POOL_SIZE, overrides the derivation. Doing that alone reintroduces the defect: pin the pool below the worker count and contention becomes 30-second connection timeouts and lost probes. If the scheduler must be smaller, lower SCHEDULER_DISPATCH_WORKERS and let the pool follow. It is also why a stack-wide DB_POOL_SIZE=10 is not the safe economy it looks like — see the connection budget.

PROBE_PAYLOAD_ENCRYPTION_ENABLED turns nothing on. Whether a dispatch is sealed to the agent's certificate on top of mutual TLS is a per-agent setting — see Probe Agents. This variable exists only so the mechanism can be disabled fleet-wide from the environment, without editing rows; set it false and every dispatch travels as it did before, inside mutual TLS and nothing more.

The GATEWAY_URL default points at localhost:8080, which is not the gateway's own default port. Set it explicitly; in Compose it is the internal service URL.

The dispatch queue holds service IDs (~50 bytes each), so 100000 is cheap. It needs to exceed your largest per-tick fleet: when it fills, the overflow is shed as skipped runs even if the agents were idle.

Raising SCHEDULER_DISPATCH_WORKERS is rarely the fix

Each worker blocks on an agent's probe round-trip, so this value is global backpressure — it caps how many connections the platform opens against targets at once. It is 50 deliberately. Raising it lets the scheduler overwhelm a slow target; congestion collapse and runaway latency have been observed against a single internet-hosted endpoint. If you need more throughput, add agents — see Probe Agents. Raise this only when you know the targets absorb the extra concurrency.

result-ingestor

Consumes probe results from Redis A and persists them. Postgres and Redis A — no Redis B — plus the body-storage settings below.

Variable	Purpose	Default	Required
PORT	Listen port	20820	No
DATABASE_URL / DATABASE_USER / DATABASE_PASSWORD	Postgres connection	See common table	No
REDIS_A_URL	Queue to consume from	redis://localhost:6379	No
STORAGE_FILESYSTEM_ROOT	Root under which agent-written bodies are relocated — must match the agent's PROBE_AGENT_STORAGE_DIR	/data/bodies	No
STORAGE_S3_ENDPOINT	S3-compatible endpoint — presence enables S3 body relocation	(unset)	No
STORAGE_S3_ACCESS_KEY / STORAGE_S3_SECRET_KEY	S3 credentials	(empty)	Yes, once the endpoint is set
STORAGE_S3_BUCKET	Bucket for relocated bodies	(unset)	Yes, once the endpoint is set
STORAGE_S3_PREFIX	Key prefix within the bucket	(empty)	No
STORAGE_S3_REGION	Signing region: auto for R2, the bucket's region for AWS S3; MinIO ignores it	auto	No
STORAGE_S3_TIMEOUT_SECONDS	Connect/read/write timeout per store call; a store that stops answering fails the call instead of parking the thread	30	No
BODY_STORE_AES_KEY	64 hex chars — decrypts body-store secret keys. Must be identical to the gateway's	(unset)	Once a store exists
BODY_STORE_FILESYSTEM_BASES	Comma-separated directories a filesystem store's root may sit beneath	(unset)	No
BODY_STORE_PRIVATE_ENDPOINTS	Allow http:// and private, CGNAT, loopback, internal-suffix and single-label hosts as store endpoints	false	No

The storage settings mirror where the agents put saved response bodies: the ingestor relocates bodies as results land, so its view of the store has to match the agents' — same filesystem root when bodies are on a shared volume, same S3 endpoint when they are in a bucket.

The three BODY_STORE_* variables are the ingestor's half of body stores, and all three must match the gateway's exactly. The ingestor is the service that imports a body out of an import store, so an import store's endpoint has to be reachable from here — the dashboard's Test button probes it from the gateway instead, which is not the same network on every deployment.

The ingestor is not given PLATFORM_AES_KEY

It does not need it. Probe variables, TOTP secrets and the CA key are decrypted elsewhere; the only credentials the ingestor decrypts are body stores', and those are under BODY_STORE_AES_KEY. Setting the platform key here does nothing. See Secrets & Encryption.

Queue pop timeout is fixed

The ingestor's blocking-pop timeout (5 seconds) is a code default with no HOCON entry and no environment override. It is not tunable without a rebuild, and there is little reason to want it to be.

notification-dispatcher

Consumes outbox events and delivers email and webhook notifications.

Variable	Purpose	Default	Required
PORT	Listen port	20830	No
DATABASE_URL / DATABASE_USER / DATABASE_PASSWORD	Postgres connection	See common table	No
REDIS_A_URL	Operational Redis	redis://localhost:6379	No
PLATFORM_AES_KEY	Decrypts org variables referenced from webhook URLs	64 zeros	No — but change it
DISPATCHER_POLL_INTERVAL_MS	Outbox poll interval	5000	No
DISPATCHER_BATCH_SIZE	Events per batch	50	No
DISPATCHER_STATUS_POP_TIMEOUT	Status queue pop timeout, seconds	5	No

PLATFORM_AES_KEY must match the gateway's

Webhook URLs can reference org variables (for example $o.telegramToken), which the gateway encrypted with its key. The dispatcher decrypts them with its own. If the two differ, decryption fails and those webhooks never deliver. The same applies to the scheduler, which decrypts probe variables. One key, every service. See Secrets & Encryption.

Webhook retry behaviour is not configurable by environment

Retry backoff (base 2 seconds, growing 4x) and the per-recipient cooldown (300 seconds) are code defaults with no application.conf entries. The attempt ceiling itself is per webhook rather than configuration: each webhook carries an attempt count (1–10, default 1 — no retry), set when the webhook is created or edited.

email-service

A queue-based email dispatcher. It consumes from Redis and sends — it needs no Postgres connection at all.

Variable	Purpose	Default	Required
PORT	Listen port	20840	No
REDIS_A_URL	Queue to consume from	redis://localhost:6379	No
EMAIL_PROVIDER	One of smtp, resend, mailgun, console, file	console	No
EMAIL_FROM_ADDRESS	Envelope from address	[email protected]	No
EMAIL_FROM_NAME	Display name	Tracedown	No
EMAIL_SMTP_HOST	SMTP host	(empty)	Yes, for smtp
EMAIL_SMTP_PORT	SMTP port	587	No
EMAIL_SMTP_USERNAME	SMTP username	(empty)	No
EMAIL_SMTP_PASSWORD	SMTP password	(empty)	No
EMAIL_SMTP_TLS_MODE	One of STARTTLS, SMTPS, PLAIN	STARTTLS	No
EMAIL_RESEND_API_KEY	Resend API key	(empty)	Yes, for resend
EMAIL_MAILGUN_API_KEY	Mailgun API key	(empty)	Yes, for mailgun
EMAIL_MAILGUN_DOMAIN	Mailgun sending domain	(empty)	Yes, for mailgun
EMAIL_MAILGUN_REGION	us or eu	us	No
EMAIL_FILE_PATH	Output path for the file provider — a single file, overwritten each send	./emails	No
EMAIL_CONSOLE_ATTACHMENT_DIR	Where the console provider writes attachments	build/email-attachments	No
EMAIL_SERVICE_POP_TIMEOUT	Queue pop timeout, seconds	5	No
EMAIL_TEMPLATE_DIR	Directory of mail templates consulted before the packaged ones — adds types and overrides wording	(unset)	No
EMAIL_LOGO_URL	Absolute URL of a small PNG shown in the header of every mail (28×28); unset shows the wordmark alone	(unset)	No
EMAIL_PRODUCT_URL	Where the mail header links to	(unset)	No
EMAIL_FOOTER_HTML	Small print under every mail, as HTML — an imprint line, links to your own pages	(unset)	No

This is the only place mail is configured. Every other service that sends — the gateway's invites and password resets, the notification-dispatcher's alerts — hands the envelope to this one over Redis A, so a provider set here covers all of them and there is no second set of names to keep in step.

EMAIL_FILE_PATH is a single file overwritten on each send (the default yields a file literally named emails); it exists for tests, not for archiving.

metrics-service

Exposes the Prometheus scrape endpoint and backs Grafana integration.

Variable	Purpose	Default	Required
PORT	Listen port	20850	No
DATABASE_URL / DATABASE_USER / DATABASE_PASSWORD	Postgres connection	See common table	No
REDIS_A_URL	Operational Redis	redis://localhost:6379	No
REDIS_B_URL	Cache Redis — holds the metric buckets	redis://localhost:6380	No
METRICS_TTL_SECONDS	Metric entry TTL	86400 (1 day)	No
METRICS_HOURLY_BUCKET_TTL_SECONDS	Hourly bucket TTL	90000 (25 hours)	No

The hourly bucket TTL exceeds the metric TTL by an hour on purpose: a bucket must outlive the window it summarises, or the final scrape of an hour reads an expired key.

This service pins a pool of 5 connections and ignores DB_POOL_SIZE.

The usage bucket TTL is fixed

metrics.usageBucketTtlSeconds (604800 — 7 days) has no HOCON entry and so no environment override.

aggregate-worker

Rolls raw results into hourly and daily aggregates, enforces retention, purges soft-deleted rows, and cleans up sessions. What each job does is covered in Retention & Aggregation; this is the configuration surface.

Variable	Purpose	Default	Required
PORT	Listen port	20860	No
DATABASE_URL / DATABASE_USER / DATABASE_PASSWORD	Postgres connection	See common table	No
REDIS_A_URL	Operational Redis	redis://localhost:6379	No
REDIS_B_URL	Cache Redis	redis://localhost:6380	No
RESULT_RETENTION_DAYS	Raw probe result retention; -1 keeps forever	90	No
BODY_RETENTION_DAYS	Saved response body retention; -1 leaves bodies to go with their results	-1	No
HOURLY_AGGREGATE_RETENTION_DAYS	Hourly aggregate retention; -1 keeps forever	365	No
AGENT_HEALTH_RETENTION_DAYS	Agent health record retention; -1 keeps forever	90	No
AUDIT_LOG_RETENTION_DAYS	Audit log retention; -1 keeps forever	90	No
NOTIFICATION_LOG_RETENTION_DAYS	Notification delivery history retention; -1 keeps forever	90	No
TRUSTED_DOMAIN_MODE	When true, DomainReverifyJob is disabled	false	No

Raw results are the bulk of the database and hourly aggregates are cheap, which is why they default to 90 and 365 days respectively — you keep a year of trend after the detail ages out. Setting RESULT_RETENTION_DAYS=-1 stops raw results being deleted by age at all and the table grows without bound; if you do, plan disk accordingly. It no longer stops bodies expiring — that is BODY_RETENTION_DAYS's own window, below.

Keep RESULT_RETENTION_DAYS identical to the gateway's value.

BODY_RETENTION_DAYS (worker.bodyRetentionDays) is the worker's alone — the gateway does not read it, and the usage window it offers is still capped by RESULT_RETENTION_DAYS. It defaults to -1, meaning the body window is off: the worker's body pass does not run and every body simply goes out with its result, which is what every release before 0.4.35 did. Set it to a number of days to shed the bodies, which are most of the bytes, while keeping the result history; setting it higher than the result window does nothing, because a body never outlives its result. See Results and bodies age separately.

RESULT_RETENTION_DAYS and BODY_RETENTION_DAYS both read a positive value as a number of days and any negative value as "never expire by age". 0 is not a value for either: the worker refuses to start, failing configuration load with BODY_RETENTION_DAYS must not be 0 — use -1 to never expire by age, or a positive number of days. If an older install set one of them to 0 to mean "keep forever", change it to -1 before upgrading to 0.4.35. See Keeping data forever.

Job intervals

Variable	Purpose	Default	Required
WORKER_INTERVAL_HOURLY_AGGREGATION	Hourly aggregation interval, seconds	900	No
WORKER_INTERVAL_DAILY_AGGREGATION	Daily aggregation interval, seconds	3600	No
WORKER_INTERVAL_RETENTION	Retention interval, seconds	3600	No
WORKER_INTERVAL_PURGE	Three-tier deletion purge interval, seconds	300	No
WORKER_INTERVAL_SESSION_CLEANUP	Expired session cleanup interval, seconds	900	No

WORKER_INTERVAL_RETENTION drives more than retention: the outbox purge, agent health cleanup, audit-log and notification-log trims and expired-token cleanup all run on the same tick.

Body storage

At retention time the worker deletes saved response bodies from S3-compatible storage alongside the database rows.

Variable	Purpose	Default	Required
STORAGE_S3_ENDPOINT	S3-compatible endpoint — presence enables deletion	(unset)	No
STORAGE_S3_ACCESS_KEY	Access key	(empty)	Yes, once the endpoint is set
STORAGE_S3_SECRET_KEY	Secret key	(empty)	Yes, once the endpoint is set
STORAGE_S3_BUCKET	Bucket the worker may delete in	(unset)	Yes, once the endpoint is set
STORAGE_S3_PREFIX	Key prefix within that bucket the worker may delete under	(empty)	No
STORAGE_S3_REGION	Signing region (auto for R2)	auto	No
STORAGE_S3_TIMEOUT_SECONDS	Per-call timeout; a hung delete is recorded in pending_body_deletions and retried later instead of stalling retention	30	No
STORAGE_FILESYSTEM_ROOT	Root the worker may delete under on disk	/data/bodies	No

The worker deletes only inside the location you give it

STORAGE_S3_BUCKET and STORAGE_S3_PREFIX (for S3) and STORAGE_FILESYSTEM_ROOT (for disk) do not merely describe where bodies are — they fence the worker. A stored body whose URI falls outside them is skipped rather than deleted, and logged at WARN with a count for the run.

That fence is what keeps a misconfigured worker from deleting inside somebody's body store, so set all three to the same values the result-ingestor has. If STORAGE_S3_ENDPOINT is set with no STORAGE_S3_BUCKET, the worker falls back to deleting wherever a body's URI points and says so at startup with a WARN — the pre-0.4.33 behaviour, kept so that an upgrade does not silently stop deleting, but not a configuration to stay on.

The endpoint variable is the on/off switch

STORAGE_S3_ENDPOINT has no default. Its presence enables S3 body deletion; leaving it unset disables it. Setting it to an empty string is not the same as leaving it out. If bodies are stored in S3 and the endpoint is unset here, retention deletes the database rows and the objects are orphaned — they accrue cost forever with nothing referencing them.

Any S3-compatible store works: Cloudflare R2, MinIO, Backblaze B2, Spaces.

This service pins a pool of 5 connections and ignores DB_POOL_SIZE.

realtime-service

A WebSocket server that bridges Redis pub/sub to connected browsers.

Variable	Purpose	Default	Required
PORT	Listen port	20870	No
DATABASE_URL	JDBC URL	(none)	Yes
DATABASE_USER	Database user	(none)	Yes
DATABASE_PASSWORD	Database password	(none)	Yes
REDIS_A_URL	Redis to subscribe to	(none)	Yes

realtime-service has no fallback defaults

Alone among the services, its application.conf uses mandatory HOCON substitutions — ${DATABASE_URL}, not ${?DATABASE_URL}. There are no baked-in defaults, so an unset variable is a startup failure, not a silent fallback to localhost. All four must be set. This is why it is the service that most often fails to boot when you run it outside Compose.

Note DATABASE_PASSWORD must be set, but may be empty — an empty value satisfies the substitution.

This service pins a pool of 5 connections and ignores DB_POOL_SIZE.

Ping intervals are fixed

realtime.pingIntervalMs (5000) and realtime.pingTimeoutMs (10000) have no environment overrides. Clients that need different keepalive timing cannot get it by configuration.

Compose environment

The Docker stack reads docker/.env, which you create by copying the shipped docker/.env.example. It is the one file most installs need to edit, and every value the template ships is a development default.

Variable	Purpose	Default	Required
DB_NAME	Database name	tracedown	No
DB_USER	Database user	tracedown	No
DB_PASSWORD	Database password	tracedown	No
PLATFORM_AES_KEY	Shared 64-hex-char encryption key	Dev placeholder	No — but change it
JWT_SECRET	Session signing secret	Dev placeholder	No — but change it
GATEWAY_PORT	Host (127.0.0.1) port for the API gateway	20714	No
REALTIME_PORT	Host (127.0.0.1) port for the WebSocket	20870	No
METRICS_PORT	Host (127.0.0.1) port for the metrics endpoint	20850	No
REDIS_A_URL	Operational Redis URL	Container-internal	No
REDIS_B_URL	Cache Redis URL	Container-internal	No
REDIS_C_URL	Hierarchy cache Redis URL	Container-internal	No

Every shipped value is a development default

The password is the username, and the AES key and JWT secret are placeholders committed to the repository. They exist so docker compose up works with no setup — see Quickstart. All of them must change before the stack is reachable by anyone but you. Secrets & Encryption covers generating real values and the consequences of rotating each one.

The dev stack points all three Redis URLs at a single Redis container serving every role. That is fine for one host and wrong under load: Redis A is AOF-persisted operational state, Redis B is a throwaway cache, and mixing them means cache churn competes with the outbox. The Compose file ships commented-out redis-b and redis-c services; uncomment them and repoint the URLs when you scale out. See Scaling.

Probe agents

Probe agents are configured separately. They are a Python service using pydantic-settings, and every variable is prefixed PROBE_AGENT_. Agents hold their own keypair and are dialled by the scheduler over mutual TLS, so their configuration is mostly about identity and reachability rather than the shared database and Redis above.

See Probe Agents for the full reference.