EN | JA

Installation and Deployment

The cross-platform kanade deploy design and script retirement gates are in SPEC §2.13. Typed schema/validation and plan persistence are implemented, but production typed dispatch and the deploy CLI are unavailable; keep using the current procedures below during migration. It targets enrolled agents only and leaves the Agent Installation screen one-liners unchanged.

This section details how to bootstrap kanade components as native Windows services in production or staging environments.

Deployment Model

Production hosts and target endpoints run kanade components as background Windows services, providing automatic startup and configured service-manager recovery.

Service NameTriple / BinaryConfig SourceTypical Target
KanadeNatsnats-server.exeConfig file with token substitution / ACLsCentral server
KanadeBackendkanade-backend.exeHardened Registry / Config fileCentral server
KanadeAgentkanade-agent.exeConfig file / hardened Registry / local state DBManaged endpoints

1. Prerequisites

  • Host OS: Windows 10/11 or Windows Server 2016+.
  • gsudo: Required to perform elevated installations from standard user shells (or run commands from an Administrator-level PowerShell prompt).
  • Network Routing: Managed endpoints must be able to reach the NATS server port (default 4222) over TCP.

2. Setting Up the NATS Server (Broker)

The NATS server acts as the messaging core.

  1. Stage the deployment bundle using scripts/build-release.ps1 -Roles nats.
  2. Deploy the service with elevation:
    # Elevated PowerShell prompt
    & "dist\nats\deploy-nats.ps1" -NatsToken "your-secure-nats-token" -Recreate
    

This installs the KanadeNats service, configures it to run under the local system account, sets up JetStream data directories, and substitutes the token into the installed NATS configuration file, and restricts that file to SYSTEM and Administrators.

Optional: per-role users instead of the shared token

By default the broker runs on the single shared token above. To run the three role users (agent, backend, breakglass) instead, deliberately:

# Default cost 8 requires a long random password (32 random alphanumeric characters recommended).
# Hashes only, minted with scripts/ops/nats-password-hash.ps1 (or .sh); never plaintext.
$env:KANADE_NATS_AGENT_PASSWORD_HASH      = '<hash>'
$env:KANADE_NATS_BACKEND_PASSWORD_HASH    = '<hash>'
$env:KANADE_NATS_BREAKGLASS_PASSWORD_HASH = '<hash>'
& "dist\nats\deploy-nats.ps1" -UseNatsUsers      # switch
& "dist\nats\deploy-nats.ps1" -UseNatsToken -NatsToken "your-secure-nats-token"   # revert

-UseNatsUsers replaces the authorization block of the installed config with an include of nats-server.users.conf (the template with the hashes substituted) and gives both files the SYSTEM + Administrators ACL; -UseNatsToken restores the shipped token block (hand edits inside it are not restored) and removes the users file. The service is stopped and started by the script as always (-NoStart defers the start). On Linux the equivalent is KANADE_NATS_AUTH_MODE=users|token for setup.sh, recorded so re-runs keep it, with a manual systemctl restart nats-server to apply it (see deploy/linux/README.md).

For the recommended bcrypt cost 8 and broker burst measurements, see Choosing the bcrypt cost.

The switch is atomic: a config cannot hold both a token and users, and a client presenting a token is rejected once users exist. Every agent, backend and CLI host must already hold a user pair before the broker flips; follow the separately documented readiness procedure first.

Expected authentication mode and connection findings

The backend watches how every connection to the broker authenticated and raises a warning when that is not what you intended, so that a switch to per-role NATS users that failed or silently reverted, or a broker that authenticates nobody, does not go unnoticed. The broker's own report is the source (its monitoring endpoint, polled every minute), not anything an agent says about itself.

Setting. Settings → Server → Expected NATS authentication (the nats_auth_mode server setting) is either token (the default, today's shared token) or users (per-role users). It is an expectation only: it changes nothing about how any process connects, it just defines what counts as unexpected. Set it to users when you switch the broker to the users block, and back to token if you switch back.

Findings. Each connection is judged by the mode you expect, the role it announces in its connection name (kanade-<role>[/identity]) and the user the broker reports it authenticated as:

findingwhenseverity
broker_openpositive broker evidence of access without an authenticated identity (no-auth), in either modecritical
token_revertedusers expected, but the connection used the shared tokencritical
role_user_mismatchusers expected, but the user is not the announced role's (agent, backend, breakglass for the CLI), e.g. the backend user on an agent connectionwarning
credential_unnameableusers expected, but the credential could not be namedinfo

Under token, the shared token is expected and only broker_open is reported. The announced role is chosen by the client, so a mismatch is evidence of a misconfigured or misused host, while a match proves nothing: it is only the connection's claim, compared against the broker's word.

Findings are grouped by kind and announced role, so a whole fleet reverting at once is a handful of entries, each with a connection count and at most five registered host names. A finding is raised when it first appears, shows as a banner on the Dashboard, and is marked resolved when it is gone (a later recurrence is raised again). While the backend's own broker connection cannot prove the mode, user names read as unknown, so expect credential_unnameable entries until it reconnects.

Acceptance evidence and uncertainty. A TCP connection is already visible in /connz?auth=1 after INFO, before CONNECT or successful authentication. A server-issued cid, an empty/missing authorized_user, a client name, language, version, RTT, or elapsed time alone does not prove access without authentication. The projector brackets its paged connz walk with /varz, checking server id, startup/configuration timestamps and response/page metadata. A restart, reload, identity change, malformed/missing response or timeout invalidates the poll. Pages are sorted by ascending CID and overlap at the preceding boundary CID. New connections append and do not by themselves prevent resolution; a missing boundary, duplicate/reordered CID or incomplete walk holds resolutions, while usable positive observations can still raise findings. The cumulative total_connections counter is not a completeness condition. Offset pagination is not an atomic snapshot: the boundary checks detect skipped prefixes, not every possible race or a change immediately after the final response.

With no reported identity on TCP, application message traffic (in/out messages) or a live subscription proves the client was allowed to use the protocol. Positive traffic must also be consistent with broker policy: explicit auth_required: true plus a missing identity stays unproven. Counters can predate a second CONNECT, whose options are overwritten before re-authentication. All three counters and the client kind/transport must be present; missing is not zero. An explicit TCP auth_required: false also proves that listener is open. The normally omitted false field is conservatively unknown, not inferred false: a silent open connection can remain unproven until application activity is seen. The backend normally has subscriptions, so an active open broker remains visible without waiting for a grace period. An idle authentication-required TCP connection within the broker's own auth_timeout is auth-pending; after that budget, or without enough monitoring evidence, it is auth-unproven. Expiry never proves acceptance. These are different from unknown, which still means a reported credential cannot safely be named and retains credential_unnameable handling. Pending/unproven connections do not replace an agent's last observed credential.

TCP auth policy and timeouts are never applied to WebSocket connections. WS can have separate authorization, credentials from cookies, or no_auth_user. The current monitoring fields do not expose its effective authorization policy. Without a reported identity, even positive traffic remains auth-unproven with reason listener_auth_unproven: counters may predate a second CONNECT whose replacement credentials will be rejected. This also limits detection of truly open WS listeners; TCP acceptance evidence remains available. A reported identity takes precedence over activity. Cookie/token auth retains redaction. WS user values stay unknown because a live TCP user login cannot prove that another listener's value is a username rather than a secret. no_auth_user assigns a broker identity, so this audit cannot distinguish that fallback from a credentialed login as the same user: it is not proof of a fully open broker. WS authentication without an identity remains unproven. MQTT/leaf/unknown transports without a reported identity are also unproven. This audit trusts the configured monitoring endpoint to be the broker and to honor auth=1; it does not test listener reachability or actively probe production with empty credentials.

Any pending/unproven observation holds existing finding resolutions; it does not block a new finding proven by another connection. Health reports authentication_uncertain, with pending_connections and unproven_connections counts (including partial-poll observations for incomplete). The optional uncertainty_reasons map breaks those observations down using fixed, secret-free EvidenceReason codes: pre_connect_pending, incomplete_monitoring, listener_auth_unproven, invalid_connection_start, auth_policy_conflict, and acceptance_unproven. Codes describe evidence limitations, not a proven cause. The notice ledger's hold_resolve_kinds policy preserves existing notices while allowing positive findings from other connections, including new critical alerts. Counts and reason maps are null when unavailable or stale. The aggregate metadata uses a JSON envelope in the existing audit-state TEXT field; legacy plain statuses remain readable. Connection evidence uses the existing retention ledger described below; Observed.connection retains broker identifiers and timestamps alongside the classified label. auth-pending/auth-unproven never contribute finding evidence. External consumers should use the health API rather than interpret that internal field.

The unnamed critical observed on 2026-10-18 from 02:24:58.636897559 UTC to 02:25:58.230333505 UTC after an OOM-triggered broker restart is a likely false positive. The pre-CONNECT window is reproducible on authentication-required NATS 2.14.3, but retained historical connection evidence cannot establish that this was the exact cause of that incident.

The evidence rules follow NATS 2.14.3's monitoring implementation and monitoring documentation. Its protocol parser gates application operations while awaiting authentication; its CONNECT handler sets metadata before checking authentication, and authentication implementation handles listener overrides and fallback identities.

Regression checks use fixtures, local HTTP mocks and disposable brokers only:

cargo test --locked -p kanade-backend nats_auth
cargo test --locked -p kanade-backend nats_conns
cargo test --locked -p kanade-backend live_connz -- --ignored
cargo make check

The live tests need nats-server on PATH and permission to bind loopback sockets. They exercise INFO-before-CONNECT, rejected credentialless CONNECT, anonymous authenticated token/user connections, reconnects, an open broker, WS token overrides and WS no_auth_user. Run against the supported broker version when upgrading; fixtures do not substitute for checking changed semantics.

Grace after a switch. When you change the expected mode, new findings are held back for 5 minutes. Switching re-authenticates every client, and each reconnects with backoff while the poll only looks once a minute, so for a few polls the broker's view is a mix of old and new; the window keeps those stragglers from being reported against the mode you just chose. Re-saving the same mode does not start it again, and findings already open still update and resolve during it. The Dashboard says when the window is open.

Reading it from outside. GET /api/health/fleet carries a nats_auth block: expected_mode, grace_until, poll_status, open_total (the connections covered) and findings[] (kind, subject, severity, count, sample_hosts, first_seen_at, occurrence_id). It does not change the response's status or HTTP code, so an existing monitor keeps behaving as before; a monitor that should alert on this must read nats_auth.findings. A poll_status other than ok means the audit could not judge the whole broker (authentication_uncertain, incomplete, endpoint_unreadable, settings_unreadable, state_unreadable, stale, or pending before its first poll), and an empty list is then not a clean result.

What is never shown. The reported value of a connection may be a secret on some broker builds, so findings carry fixed vocabularies (kind, severity, the role bucket agent / backend / cli / unnamed / other), counts, registered host samples and an occurrence reference. Admin-only connection evidence adds broker-issued identifiers and parsed timestamps, as described below. Neither the credential, nor the reported user name, nor the raw connection name is copied into notice evidence.

Retained NATS authentication evidence

Each new occurrence has its own numeric occurrence_id in the authenticated fleet health finding. The aggregate backend_notices row remains compatible; reopening it does not overwrite earlier occurrences. Dashboard's admin-only NATS authentication evidence and resolved history details and GET /api/notices/nats-auth expose the latest 100 occurrences, including resolved ones. Pass ?before=<oldest-id> to page backwards. Viewers and operators cannot read this endpoint. Existing service-token and auth-disabled administrator semantics still apply. Identifiers are selectable and copyable for log searches.

A host claim is saved only for an agent role whose exact pc_id is registered; registration does not authenticate the claim. Stored host text removes control characters and is limited to 64 Unicode characters. Other sample text is bounded by typed timestamps, fixed role/credential vocabularies, decimal u64 cids and ASCII alphanumeric server IDs of at most 64 bytes. Invalid identifiers are rejected rather than truncated. Older samples have no host field; it is not backfilled. The Dashboard separates broker identifiers and credential classes from untrusted client role/host claims.

A sample records the broker-issued decimal cid, server_id and broker startup timestamp (/varz.start), backend first/last observation times (page fetch completion, before settings/database work), broker page observation time, and connection start/last activity timestamps when supplied. Occurrence timestamps describe notice processing; individual sample timestamps describe page acquisition. A cid alone, or server_id alone, must not be treated as a restart identity. The collector reads /varz before and after the paginated /connz walk and checks each page's server_id, timestamp and duplicate cids, within the existing 20-second timeout. Missing identity or an inconsistent/restarted broker rejects the snapshot, reports an unreadable poll and leaves open findings unresolved. A paging cap reports an incomplete poll and likewise cannot resolve findings. This is a best-effort walk of live connections, not an atomic broker snapshot.

Evidence includes only a fixed credential class (user, shared-token, no-auth, unknown), a fixed claimed_role bucket and an optional bounded registered host claim. Role and host are untrusted client claims, unlike broker-issued connection identifiers. Aggregate notifications also retain registered host samples. Raw authorized_user, tokens, passwords, connection names, arbitrary client metadata and unrestricted monitoring responses are never copied into connection evidence. Peer addresses are deliberately omitted: proxies and loopback commonly identify an intermediary rather than the originating client. The admin boundary limits exposure of retained connection correlation data.

Per occurrence, at most the first 10 sampled identities and 10 most recently observed other identities are retained, deduplicated by server_id/start/cid, within a 32 KiB JSON limit. Each sample and the latest occurrence observation record whether their poll was complete; partial polls may undercount connections. Broker IDs are limited to 64 alphanumeric bytes; other context is parsed timestamps or fixed vocabulary. sampled_out indicates sample/size omissions. Evidence updates on each successful poll even if group counts/host samples have not changed. Ledger changes, occurrence creation, resolution and evidence updates commit in one SQLite transaction.

Samples expire 30 days after their last observation, including samples in open findings. Resolved occurrences expire 30 days after resolution and are also capped at the newest 1000. Cleanup runs when the projector starts and every poll (including failed polls); an offline backend performs cleanup when restarted. There are only the classifier's fixed kind/role buckets of open occurrences. SQLite backups must use the operator's corresponding retention/access policy; logical expiry does not securely erase old database pages or backups.

Old notices have no recoverable connection evidence; the UI labels absent or expired evidence accordingly. The identifiers lost in the 2026-10-18 incident cannot be reconstructed by this migration. Last observed is not a disconnect time. Connections that open and close between minute polls can be missed, and wiping the projection database loses these snapshots permanently. Retained evidence uses the same authentication classification as aggregate findings through Observed and the shared evaluate_observed adapter. Pending and unproven connections do not contribute finding samples or resolve existing occurrences. The additive evidence migration is named 20261018000000_nats_auth_evidence.sql.


3. Deploying the Backend API & SPA

The backend manages operator connections and processes event logs.

  1. Stage the backend binaries and React SPA bundle using scripts/build-release.ps1 -Roles backend.
  2. Deploy the service:
    # Elevated PowerShell prompt
    & "dist\backend\deploy-backend.ps1" `
        -NatsToken "your-secure-nats-token" `
        -StaticToken "your-operator-spa-bearer-token" `
        -ForceConfig -Recreate
    
  • -NatsToken: Connects the backend to the local NATS server securely.
  • -StaticToken: Defines the API bearer token required for operator CLI/SPA logins.

The deployment script registers the KanadeBackend Windows service, sets the appropriate ACLs, and verifies the endpoint.


4. Installing the Agent on Target Endpoints

Install the agent on every endpoint PC that you want to manage.

  1. Stage the agent bundle using scripts/build-release.ps1 -Roles agent.
  2. Copy the contents of the dist/agent folder to the target PC.
  3. On the target PC, run the installer:
    # Elevated PowerShell prompt
    & ".\deploy-agent.ps1" -NatsToken "your-secure-nats-token" -ForceConfig -Recreate
    

The script:

  • Places kanade-agent.exe into its destination directory.
  • Secures the configuration and NATS token in the Windows registry path (HKLM:\SOFTWARE\Kanade\agent).
  • Registers and starts the KanadeAgent service.

Agent-role NATS user in generated installers

deploy-agent.ps1 also accepts -NatsUser / -NatsPassword (the shell scripts take KANADE_NATS_USER / KANADE_NATS_PASSWORD), both or neither. The installers the backend generates (Windows ZIP, Linux / macOS tarball) embed such a pair when the server settings' agent_install section holds both a user and a password, next to the token, so machines installed from now on already carry the credential the broker may later require.

  • The pair is the agent role's credential and is shared by all agents by design. The backend's own credential and the break-glass credential must never be entered in these settings.
  • It is write-only, like the token: the API returns only nats_user_set / nats_password_set, and a settings update that omits both keeps the stored pair. A settings update that sets only one of the two is rejected, so replacing the password means sending the user again.
  • A generated README.txt states whether a user pair is included, without revealing it; with no paragraph about it, none was embedded and the installer is exactly what it was before this setting existed.

Once the service is active, the agent establishes an outbound NATS connection, subscribes to command streams, and reports its online heartbeat back to the fleet backend.