OpenCensus and Kafka span receivers (ING-011)

Embed
Preview of the card
<a href="https://forumulate.com/p/watchtower/docs/trace-receivers"><img src="https://forumulate.com/api/public/orgs/watchtower/spaces/docs/pages/6e0ac0be-2801-4a55-95d5-8eb389eb463b/social-card.svg?v=3e7ae33eec8b" alt="OpenCensus and Kafka span receivers (ING-011)" width="600" height="315"></a>

Both authenticate with an ingest key that has ingest:traces, convert to OTLP and use the same pipeline as OTLP (tenant from the key, wt.* attributes stripped, quotas).

OpenCensus

  • gRPC: opencensus.proto.agent.trace.v1.TraceService/Export on the gRPC port (key in authorization: Bearer metadata). The node's service_info.name becomes service.name for the whole stream.

  • HTTP: POST /opencensus/v1/trace with one protobuf ExportTraceServiceRequest body.

  • Mapped: ids, name, kind, times, attributes, status. Dropped: time events, links, stack traces.

  • WT_GATEWAY_OPENCENSUS=false turns it off.

Kafka span consumer

Off unless WT_GATEWAY_KAFKA_SPANS_BROKERS is set. Also: _TOPICS, _KEY (required), _GROUP (default wt-gateway-spans), _FORMAT (otlp_proto default, otlp_json, zipkin_json), _USER/_PASSWORD and _MECHANISM (PLAIN default, SCRAM-SHA-256, SCRAM-SHA-512; TLS on). Offsets commit after ingest (at-least-once); undecodable records are skipped, transient failures retry.

OTel Arrow (OTAP)

ArrowTracesService, ArrowLogsService and ArrowMetricsService streams on the gRPC port, decoded with the Apache-2.0 otel-arrow Consumer. Auth, scopes, wt.* stripping and quotas are those of OTLP; every batch gets a BatchStatus. A malformed batch gets INVALID_ARGUMENT and the stream ends so the exporter reconnects. WT_GATEWAY_ARROW=false turns it off (exporters then downgrade to OTLP).

Updated