Fluent Bit

Embed
Preview of the card
<a href="https://forumulate.com/p/watchtower/docs/fluent-bit"><img src="https://forumulate.com/api/public/orgs/watchtower/spaces/docs/pages/466377ae-2190-4ca9-9b55-57930b0b290a/social-card.svg?v=3e7ae33eec8b" alt="Fluent Bit" width="600" height="315"></a>

Forward logs from your own Fluent Bit (3.x) with its opentelemetry output. Needs an ingestion key with ingest:logs. Export WT_OTLP_HOST (for example ingest.example.com) and WT_INGEST_KEY in the Fluent Bit environment.

[SERVICE]
    Flush        1
    Log_Level    info

[INPUT]
    Name         tail
    Path         /var/log/app/*.log
    Tag          app.*
    DB           /var/lib/fluent-bit/app-tail.db

[OUTPUT]
    Name         opentelemetry
    Match        *
    Host         ${WT_OTLP_HOST}
    Port         443
    Tls          On
    Logs_uri     /v1/logs
    Logs_Body_Key $log
    Compress     gzip
    Retry_Limit  False
    Header       Authorization Bearer ${WT_INGEST_KEY}

For a plain-HTTP test gateway use Port 4318 and Tls Off. Logs_Body_Key $log makes the tail line (record key log) the OTLP log body; without it the body is not the plain line. Retry_Limit False retries failed sends without limit (the default is 1 retry, after which the chunk is dropped), and the tail DB file stores file offsets so a restart resumes where it left off.

Delivery is at-least-once: a chunk whose acknowledgement was lost is sent again, so duplicates are possible. Chunks are buffered in memory by default, so a Fluent Bit crash can still lose data not yet flushed; enable filesystem storage buffering if that matters.

Status: structure-checked, not yet run against the real agent.

Updated