Storage tiers, per-filter retention, trace pinning, archive search
Embed
<a href="https://forumulate.com/p/watchtower/docs/tiering-and-retention"><img src="https://forumulate.com/api/public/orgs/watchtower/spaces/docs/pages/544e6dea-d61c-4f05-85d1-f8b2eaef83e9/social-card.svg?v=3e7ae33eec8b" alt="Storage tiers, per-filter retention, trace pinning, archive search" width="600" height="315"></a>Tiers (LOG-008)
Set WT_CH_TIERING=1 on the janitor and mount infra/clickhouse/storage-policy.xml on ClickHouse (storage policy wt_tiered with volumes hot, warm, cold). The janitor then sets, on logs, spans, metric samples and error events:
TTL ts + warm TO VOLUME 'warm', ts + cold TO VOLUME 'cold', ts + retention_days DELETE
Boundaries are cluster-wide: WT_TIER_WARM_DAYS (default 7) and WT_TIER_COLD_DAYS (default 30). Without the policy the janitor logs tiering unavailable once and changes nothing.
Log search answers with X-Wt-Tier: hot|warm|cold, the tier of the oldest instant of the range. The logs explorer shows a banner for warm or cold ranges and stops the live tail.
Retention rules
POST|GET|PUT|DELETE /v1/projects/{p}/retention-rules (existing API). match keys: service, env, severity_min, severity_gte (level name, logs), status: "error" (traces), duration_ms_gte (traces), attr {k, v} (or {key, eq}). At ingest the worker applies the first rule by priority to each record; no match keeps the project default. At most 50 rules per project, 1 to 3650 days. A batch is inserted once per distinct override with dedupe token {token}.r{days} (at-least-once; a rule change between redeliveries can store a record twice).
Trace pinning (TRC-004)
POST /v1/projects/{p}/traces/{trace_id}/pin {days} runs a mutation on spans_v1 and trace_index_v1. Shortening is refused (409) while a legal hold covers traces. Audited (trace.pinned). The archive copy under pinned/ is not built yet.
Archive search (PLT-027)
POST /v1/projects/{p}/archive-search {signal, from, to, query, limit <= 10000} reads the archived Parquet days through ClickHouse s3() (class background). Day granularity. The project's archive credentials are unsealed for the request and sent as query parameters; load infra/clickhouse/masking.xml so they also stay out of system.query_log.