Cloud metric pulls

Embed
Preview of the card
<a href="https://forumulate.com/p/watchtower/docs/cloud-pull"><img src="https://forumulate.com/api/public/orgs/watchtower/spaces/docs/pages/704bab9a-969d-48e8-8a4a-e28c90604354/social-card.svg?v=3e7ae33eec8b" alt="Cloud metric pulls" width="600" height="315"></a>

Agents pull metrics from provider APIs with the cloud_pull module. Credentials live only in the agent's own config file (never remote config, never a label, log line or error). The /v1/projects/{p}/cloud-pulls API (ManageConnectors) stores the project's pull settings; its secret is sealed with wt-secrets (bound to project and row) and is never returned or audited.

cloud_pull:
  pulls:
    - provider: gcp            # aws | gcp | azure | cloudflare
      interval_s: 300          # minimum 60
      config: {project: my-proj, metrics: ["compute.googleapis.com/instance/cpu/utilization"],
               service_account: {client_email: ..., private_key: ..., token_uri: ...}}
    - provider: azure
      config: {tenant_id: ..., client_id: ..., client_secret: ..., metrics: ["Percentage CPU"],
               resources: ["/subscriptions/<id>/resourceGroups/<g>/providers/Microsoft.Compute/virtualMachines/<vm>"]}
    - provider: cloudflare
      config: {api_token: ..., zone_ids: [<zone>], account_id: <acct>, workers: true}
    - provider: aws            # same settings as the aws module (namespaces, region)
      config: {region: us-east-1, namespaces: [AWS/Lambda]}

Metrics are named cloud.<provider>.<namespace>.<metric> with resource attributes cloud.provider, cloud.account.id, cloud.region, cloud.resource_id. Each run asks for the time since the last success plus a 5-minute overlap; points are deduplicated by (series, timestamp), so delivery is at-least-once without double counting. A 429 backs off (Retry-After, else exponential). The gauge cloud.pull.pulled_series counts the series of each run.

GCP builds its service-account JWT (RS256) with the standard library. Vercel and Cloudflare logs are received by the gateway: see vercel.md and cloudflare.md.

Updated