Slack

Embed
Preview of the card
<a href="https://forumulate.com/p/watchtower/docs/slack"><img src="https://forumulate.com/api/public/orgs/watchtower/spaces/docs/pages/5cd2cad8-de19-4760-ba7f-f1438aa7e445/social-card.svg?v=3e7ae33eec8b" alt="Slack" width="600" height="315"></a>

The Watchtower Slack app posts alerts as threads, lets responders act from buttons and slash commands, and opens incident channels. One Slack app serves every workspace; each workspace is installed once and belongs to one Watchtower organization.

Set up (operator)

  1. Create the app from infra/connectors/slack-app-manifest.yaml (replace WT_PUBLIC_URL).

  2. Set on the notifier: WT_PUBLIC_URL, WT_INTERNAL_TOKEN, WT_API_INTERNAL_URL, WT_SLACK_CLIENT_ID, WT_SLACK_CLIENT_SECRET, WT_SLACK_SIGNING_SECRET, WT_SECRET_KEY. On the API set the same WT_INTERNAL_TOKEN (this turns on the internal listener, WT_INTERNAL_ADDR, default 127.0.0.1:8081, which must not be published).

  3. Expose the notifier's public listener (WT_NOTIFIER_PUBLIC_ADDR, default 0.0.0.0:8893) under /connect/; nothing else of the notifier is public.

Install (organization admin)

The API asks the notifier for an install link with POST /internal/connect/slack/state {org_id, user_id} (internal token required; the API must first check ManageConnectors) and sends the admin to the returned URL. The link is single use and lives 10 minutes; only its SHA-256 is stored. Opening it sets an HttpOnly, Secure, SameSite=Lax cookie with a random nonce (its hash is stored with the state); the OAuth callback must carry the same cookie. Slack redirects back to /connect/slack/oauth, where the bot token is sealed into connector_installations.secret_enc. The token is never shown, logged or put in audit events. A workspace already connected to another organization cannot be taken over (409).

Create a receiver of kind slack_app with public config {"installation_id": "<id>", "channel": "C0123456789"} and no secrets.

What it does

  • Threads. The first notification of an alert group posts a message with one section per firing alert (up to five) and the buttons Ack, Resolve, Silence 1h and Declare incident. Later state changes update that message in place (chat.update); when the group resolves, a "Resolved." reply is added and the next firing starts a new thread. Redelivered payloads are dropped by their seq.

  • Buttons and commands call the API as the linked user: POST /internal/v1/actions/{ack|resolve|silence|declare} with X-WT-Internal-Token and {user_id, project_id, target}. The API applies that user's role and writes the audit event. A 403 shows "Your Watchtower role does not allow this action."

  • /wt oncall asks the API, which checks your role in the project, and shows display names only. /wt ack <alert id> acknowledges. /wt incident declare <title> declares an incident, creates a channel (conversations.create) and invites the responders and you (conversations.invite). Slash commands use the installation's project (its project_id, or config.project_id).

  • Linking accounts. A chat user acts only as a Watchtower user they linked on purpose. They run /wt link, get a one-time code (10 minutes, stored as a hash), and enter it in the Watchtower web app while signed in (POST /v1/connectors/link). The link is scoped to that workspace and organization. Until then every button and command answers "Run /wt link" and runs nothing. An organization can opt in to linking by email (auto_link_email in the installation config); that only links a user whose address Slack confirmed, who belongs to this workspace (not a guest, restricted, bot or Slack Connect user), when exactly one organization member has that address.

Security

Every inbound request (interact, commands, events) is verified with the v0 HMAC of v0:<timestamp>:<body> using the signing secret, in constant time, and rejected when the timestamp is more than 5 minutes from now. response_url is only called when it points at hooks.slack.com. All outbound calls use the SSRF-guarded client. app_uninstalled and tokens_revoked events revoke the installation and wipe its sealed token.

  • Silence 1h creates a silence (needs ManageAlerts) from the alert's labels; Declare incident declares an incident (needs RespondIncidents) from the alert or the title.

Known limits

Slash commands run their calls in sequence inside Slack's 3 second window; a slow API can time the command out in Slack although the action ran.

Incident channels

The workflow action incident_channel {incident_id, provider: slack|teams, name_template, invite_responders, archive_on_resolve} opens one chat space per incident and stores it in incident_chat_links (shown by GET /v1/projects/{prj}/incidents/{inc}/chat-links).

  • Slack: conversations.create, conversations.invite (when invite_responders), and conversations.archive once the incident is resolved (when archive_on_resolve). All three need the channels:manage scope the app already requests; private channels would additionally need groups:write (requested). name_template accepts {{.Number}} and {{.Slug}} (default inc-{{.Number}}-{{.Slug}}).

  • Teams: channel creation needs tenant admin consent, so Watchtower does not create channels. It starts one thread per incident in the channel the app was connected to; nothing is archived.

Updated