Collectors move logs
Fluent Bit, Fluentd, Vector, and OpenTelemetry Collector can read pod and container logs, attach Kubernetes metadata, and forward records.
Compare / Kubernetes
Kubernetes Logging Tools: What Each Layer Actually Does
Kubernetes logging tools are easier to compare when collection, storage, search, and live log analysis stay separate. The right setup keeps pod logs moving, preserves Kubernetes metadata, and gives the resulting records a readable place to land.
Kubernetes logging tools
Collectors
Storage and query
Live log analysis
Storage keeps history
Loki, Elastic, OpenSearch, ClickHouse-based stacks, and cloud logging products store Kubernetes logs and expose query paths.
Log management makes records usable
Streams, live tail, filters, facets, histograms, alerts, diagnostics, and AI chat matter after logs leave the cluster.
Metadata decides the result
Namespace, pod, container, node, service, labels, timestamp, level, and message should survive every tool boundary.
Layers
Start by Separating Collection, Storage, and Reading
A Kubernetes logging setup usually uses more than one tool. One part reads logs from nodes, another keeps them searchable, and another makes them practical to read during deploys and failures.
01
Collection
Collectors and agents read container stdout/stderr, parse records, attach Kubernetes metadata, batch safely, and forward logs to another system.
02
Storage and search
Storage/query tools retain Kubernetes logs and make historical search possible. The tradeoff is the storage, indexing, retention, and upgrade work required to keep the stack healthy.
03
Live reading and analysis
The final destination should keep the stream readable, preserve fields, support filters and facets, show volume over time, and alert on structured Kubernetes fields.
Comparison
Compare Kubernetes logging tools by responsibility
A collector, a storage/query stack, a cloud logging product, and a log management destination solve different parts of Kubernetes logging.
Example Kubernetes log collection path
text
1pod stdout/stderr2 -> node container log files3 -> Fluent Bit, Vector, or OpenTelemetry Collector4 -> Kubernetes metadata enrichment5 -> HTTP, OTLP, Syslog, or GELF output6 -> Fluxtail receiver7 -> Fluxtail stream with live tail, filters, facets, histograms, and alerts
The collector gets the logs out of the cluster. Fluxtail receives the forwarded records and keeps them readable.
| Layer | Examples | Does | Does not do alone |
|---|---|---|---|
| Collector or agent | Fluent Bit, Fluentd, Vector, OpenTelemetry Collector | Reads Kubernetes logs, enriches records with metadata, and forwards them | Usually not the long-term search, alerting, or live analysis surface |
| Storage and query stack | Loki, Elastic, OpenSearch, ClickHouse-based stack | Stores logs and supports query patterns over retained data | Does not automatically guarantee clean collection or readable log rows |
| Cloud provider logging | AWS, Google Cloud, Azure logging products | Integrates with provider infrastructure and managed retention/search | May be less portable when clusters span providers or self-hosted environments |
| Log management destination | Fluxtail | Receives logs into streams with live tail, filters, facets, histograms, alerts, MCP diagnostics, and AI chat | Still expects a collector, agent, or emitter to send pod and container logs from the cluster |
Collectors
Collector Tools: Fluent Bit, Fluentd, Vector, and OpenTelemetry Collector
Collector tools handle Kubernetes log collection. They run close to workloads, read pod and container logs, enrich records, and send the result to a destination.
01
Fluent Bit
Fluent Bit is commonly used as a lightweight node-level collector. It can run as a DaemonSet, read container log files, enrich records with Kubernetes metadata, and forward logs through output plugins.
02
Fluentd
Fluentd is mature and plugin-rich. It can make sense when routing and transformation needs are more complex, but it is usually heavier than Fluent Bit for simple node collection.
03
Vector
Vector is often used for fast collection, transforms, and routing. It can be a good fit when logs need parsing or reshaping before they reach the destination.
04
OpenTelemetry Collector
OpenTelemetry Collector is useful when services already use OTel conventions. For logs, preserve Kubernetes resource attributes such as k8s.namespace.name, k8s.pod.name, k8s.container.name, k8s.node.name, and service.name.
OpenTelemetry resource attributes
json
1{2 "service.name": "checkout-api",3 "k8s.namespace.name": "production",4 "k8s.pod.name": "checkout-api-7d9f7c9f9b-42qkp",5 "k8s.container.name": "checkout-api",6 "k8s.node.name": "worker-03"7}
Comparison
Compare Kubernetes logging tools by responsibility
A collector, a storage/query stack, a cloud logging product, and a log management destination solve different parts of Kubernetes logging.
Example Kubernetes log collection path
text
1pod stdout/stderr2 -> node container log files3 -> Fluent Bit, Vector, or OpenTelemetry Collector4 -> Kubernetes metadata enrichment5 -> HTTP, OTLP, Syslog, or GELF output6 -> Fluxtail receiver7 -> Fluxtail stream with live tail, filters, facets, histograms, and alerts
The collector gets the logs out of the cluster. Fluxtail receives the forwarded records and keeps them readable.
| Layer | Examples | Does | Does not do alone |
|---|---|---|---|
| Collector or agent | Fluent Bit, Fluentd, Vector, OpenTelemetry Collector | Reads Kubernetes logs, enriches records with metadata, and forwards them | Usually not the long-term search, alerting, or live analysis surface |
| Storage and query stack | Loki, Elastic, OpenSearch, ClickHouse-based stack | Stores logs and supports query patterns over retained data | Does not automatically guarantee clean collection or readable log rows |
| Cloud provider logging | AWS, Google Cloud, Azure logging products | Integrates with provider infrastructure and managed retention/search | May be less portable when clusters span providers or self-hosted environments |
| Log management destination | Fluxtail | Receives logs into streams with live tail, filters, facets, histograms, alerts, MCP diagnostics, and AI chat | Still expects a collector, agent, or emitter to send pod and container logs from the cluster |
Storage
Storage and Query Tools: Loki, Elastic, OpenSearch, and ClickHouse-Based Stacks
Storage/query tools keep Kubernetes logs searchable over time. The right option depends on query style, retention needs, volume, and how much infrastructure you want to operate.
01
Label-oriented storage
Loki-style systems work well when queries mostly group by namespace, pod, service, and environment labels. Label discipline matters because high-cardinality labels can make the setup harder to manage.
02
Full-text and indexed-field search
Elastic and OpenSearch-style systems are useful when message search, indexed fields, dashboards, and broader search tooling are central to the setup.
03
Columnar analytics stacks
ClickHouse-based stacks can be strong for high-volume structured logs, facets, histograms, and aggregate queries. They also require schema, ingestion, and query design work.
04
Managed storage
Managed storage reduces maintenance work, but metadata quality still depends on the collector and the fields sent with each record.
Comparison
Compare Kubernetes logging tools by responsibility
A collector, a storage/query stack, a cloud logging product, and a log management destination solve different parts of Kubernetes logging.
Example Kubernetes log collection path
text
1pod stdout/stderr2 -> node container log files3 -> Fluent Bit, Vector, or OpenTelemetry Collector4 -> Kubernetes metadata enrichment5 -> HTTP, OTLP, Syslog, or GELF output6 -> Fluxtail receiver7 -> Fluxtail stream with live tail, filters, facets, histograms, and alerts
The collector gets the logs out of the cluster. Fluxtail receives the forwarded records and keeps them readable.
| Layer | Examples | Does | Does not do alone |
|---|---|---|---|
| Collector or agent | Fluent Bit, Fluentd, Vector, OpenTelemetry Collector | Reads Kubernetes logs, enriches records with metadata, and forwards them | Usually not the long-term search, alerting, or live analysis surface |
| Storage and query stack | Loki, Elastic, OpenSearch, ClickHouse-based stack | Stores logs and supports query patterns over retained data | Does not automatically guarantee clean collection or readable log rows |
| Cloud provider logging | AWS, Google Cloud, Azure logging products | Integrates with provider infrastructure and managed retention/search | May be less portable when clusters span providers or self-hosted environments |
| Log management destination | Fluxtail | Receives logs into streams with live tail, filters, facets, histograms, alerts, MCP diagnostics, and AI chat | Still expects a collector, agent, or emitter to send pod and container logs from the cluster |
Destination
Log Management Destinations: Reading Kubernetes Logs in Fluxtail
Fluxtail receives Kubernetes logs from collectors, agents, emitters, or protocol outputs into receivers and streams. It does not need to be the node collector; it is the place where forwarded records become readable.
01
Receivers accept common outputs
Send Kubernetes logs through HTTP, Syslog, OTLP, or GELF receiver paths depending on the collector or emitter already used by the cluster.
02
Streams keep cluster logs readable
Streams can group logs by cluster, environment, namespace, service family, or source type, while the original Kubernetes fields remain available for filtering.
03
Analysis uses preserved fields
Live tail, filters, facets, histograms, alerts, MCP diagnostics, and AI chat work best when namespace, pod, container, node, service, labels, level, and message survive ingestion.
Readable Kubernetes row
output
14:12:30.421Z error production checkout-api checkout-api-7d9f7c9f9b-42qkp worker-03 payment authorization failed request_id=req_9d81f3Comparison
Compare Kubernetes logging tools by responsibility
A collector, a storage/query stack, a cloud logging product, and a log management destination solve different parts of Kubernetes logging.
Example Kubernetes log collection path
text
1pod stdout/stderr2 -> node container log files3 -> Fluent Bit, Vector, or OpenTelemetry Collector4 -> Kubernetes metadata enrichment5 -> HTTP, OTLP, Syslog, or GELF output6 -> Fluxtail receiver7 -> Fluxtail stream with live tail, filters, facets, histograms, and alerts
The collector gets the logs out of the cluster. Fluxtail receives the forwarded records and keeps them readable.
| Layer | Examples | Does | Does not do alone |
|---|---|---|---|
| Collector or agent | Fluent Bit, Fluentd, Vector, OpenTelemetry Collector | Reads Kubernetes logs, enriches records with metadata, and forwards them | Usually not the long-term search, alerting, or live analysis surface |
| Storage and query stack | Loki, Elastic, OpenSearch, ClickHouse-based stack | Stores logs and supports query patterns over retained data | Does not automatically guarantee clean collection or readable log rows |
| Cloud provider logging | AWS, Google Cloud, Azure logging products | Integrates with provider infrastructure and managed retention/search | May be less portable when clusters span providers or self-hosted environments |
| Log management destination | Fluxtail | Receives logs into streams with live tail, filters, facets, histograms, alerts, MCP diagnostics, and AI chat | Still expects a collector, agent, or emitter to send pod and container logs from the cluster |
Cloud logging
Cloud Provider Logging
Cloud provider logging can be a practical default when clusters live in one provider and native integration matters more than portability.
01
Good for provider-native infrastructure
Provider logging can reduce setup work for managed Kubernetes services and surrounding infrastructure logs.
02
Check portability and cost early
When clusters span providers, self-hosted environments, or hybrid deployments, compare export paths, retention costs, query pricing, and field preservation before standardizing.
Comparison
Compare Kubernetes logging tools by responsibility
A collector, a storage/query stack, a cloud logging product, and a log management destination solve different parts of Kubernetes logging.
Example Kubernetes log collection path
text
1pod stdout/stderr2 -> node container log files3 -> Fluent Bit, Vector, or OpenTelemetry Collector4 -> Kubernetes metadata enrichment5 -> HTTP, OTLP, Syslog, or GELF output6 -> Fluxtail receiver7 -> Fluxtail stream with live tail, filters, facets, histograms, and alerts
The collector gets the logs out of the cluster. Fluxtail receives the forwarded records and keeps them readable.
| Layer | Examples | Does | Does not do alone |
|---|---|---|---|
| Collector or agent | Fluent Bit, Fluentd, Vector, OpenTelemetry Collector | Reads Kubernetes logs, enriches records with metadata, and forwards them | Usually not the long-term search, alerting, or live analysis surface |
| Storage and query stack | Loki, Elastic, OpenSearch, ClickHouse-based stack | Stores logs and supports query patterns over retained data | Does not automatically guarantee clean collection or readable log rows |
| Cloud provider logging | AWS, Google Cloud, Azure logging products | Integrates with provider infrastructure and managed retention/search | May be less portable when clusters span providers or self-hosted environments |
| Log management destination | Fluxtail | Receives logs into streams with live tail, filters, facets, histograms, alerts, MCP diagnostics, and AI chat | Still expects a collector, agent, or emitter to send pod and container logs from the cluster |
Examples
Example Kubernetes Log Record
Any Kubernetes logging tool combination should preserve both the cluster fields and the application fields needed for filtering.
01
Preserve this shape through the tool chain
The exact field names can vary by collector and destination. The important part is that Kubernetes metadata and application context stay attached to the same record.
Kubernetes log record
json
1{2 "timestamp": "2026-04-25T14:12:30.421Z",3 "level": "error",4 "message": "payment authorization failed",5 "namespace": "production",6 "pod": "checkout-api-7d9f7c9f9b-42qkp",7 "container": "checkout-api",8 "node": "worker-03",9 "service": "checkout-api",10 "labels": {11 "app": "checkout-api",12 "version": "2026.04.25",13 "environment": "production"14 },15 "request_id": "req_9d81f3",16 "route": "POST /api/checkout",17 "error_code": "PAYMENT_GATEWAY_TIMEOUT"18}
Comparison
Compare Kubernetes logging tools by responsibility
A collector, a storage/query stack, a cloud logging product, and a log management destination solve different parts of Kubernetes logging.
Example Kubernetes log collection path
text
1pod stdout/stderr2 -> node container log files3 -> Fluent Bit, Vector, or OpenTelemetry Collector4 -> Kubernetes metadata enrichment5 -> HTTP, OTLP, Syslog, or GELF output6 -> Fluxtail receiver7 -> Fluxtail stream with live tail, filters, facets, histograms, and alerts
The collector gets the logs out of the cluster. Fluxtail receives the forwarded records and keeps them readable.
| Layer | Examples | Does | Does not do alone |
|---|---|---|---|
| Collector or agent | Fluent Bit, Fluentd, Vector, OpenTelemetry Collector | Reads Kubernetes logs, enriches records with metadata, and forwards them | Usually not the long-term search, alerting, or live analysis surface |
| Storage and query stack | Loki, Elastic, OpenSearch, ClickHouse-based stack | Stores logs and supports query patterns over retained data | Does not automatically guarantee clean collection or readable log rows |
| Cloud provider logging | AWS, Google Cloud, Azure logging products | Integrates with provider infrastructure and managed retention/search | May be less portable when clusters span providers or self-hosted environments |
| Log management destination | Fluxtail | Receives logs into streams with live tail, filters, facets, histograms, alerts, MCP diagnostics, and AI chat | Still expects a collector, agent, or emitter to send pod and container logs from the cluster |
Selection
How to Choose a Kubernetes Logging Tool Combination
Compare tools by responsibility, not by brand name alone. A strong setup keeps collection reliable, storage searchable, and live log review fast.
01
If logs are not leaving the cluster reliably
Start with the collector or agent. Confirm pod churn, node movement, and container restarts do not drop the metadata needed later.
02
If historical search is slow or expensive
Inspect storage, indexing, retention, label cardinality, and query shape before changing the collector.
03
If logs arrive but are hard to read
Improve the destination, stream structure, live tail, filters, facets, and alerting around the central log stream.
04
If many fields are missing
Fix enrichment before changing dashboards. Missing namespace, pod, container, node, service, labels, or request_id fields make every later tool weaker.
Comparison
Compare Kubernetes logging tools by responsibility
A collector, a storage/query stack, a cloud logging product, and a log management destination solve different parts of Kubernetes logging.
Example Kubernetes log collection path
text
1pod stdout/stderr2 -> node container log files3 -> Fluent Bit, Vector, or OpenTelemetry Collector4 -> Kubernetes metadata enrichment5 -> HTTP, OTLP, Syslog, or GELF output6 -> Fluxtail receiver7 -> Fluxtail stream with live tail, filters, facets, histograms, and alerts
The collector gets the logs out of the cluster. Fluxtail receives the forwarded records and keeps them readable.
| Layer | Examples | Does | Does not do alone |
|---|---|---|---|
| Collector or agent | Fluent Bit, Fluentd, Vector, OpenTelemetry Collector | Reads Kubernetes logs, enriches records with metadata, and forwards them | Usually not the long-term search, alerting, or live analysis surface |
| Storage and query stack | Loki, Elastic, OpenSearch, ClickHouse-based stack | Stores logs and supports query patterns over retained data | Does not automatically guarantee clean collection or readable log rows |
| Cloud provider logging | AWS, Google Cloud, Azure logging products | Integrates with provider infrastructure and managed retention/search | May be less portable when clusters span providers or self-hosted environments |
| Log management destination | Fluxtail | Receives logs into streams with live tail, filters, facets, histograms, alerts, MCP diagnostics, and AI chat | Still expects a collector, agent, or emitter to send pod and container logs from the cluster |
Checklist
Kubernetes Logging Checklist
Use this checklist before settling on any Kubernetes logging tool combination.
01
Field preservation
Confirm namespace, pod, container, node, service, labels, timestamp, level, message, request_id, trace_id, route, and error fields survive collection.
02
Live reading
Confirm new pod logs appear in live tail, filters target Kubernetes fields directly, and facets reveal top namespaces, pods, services, nodes, and error fields.
03
Operations
Confirm retention, ingestion volume, query cost, retry behavior, auth, alert targets, and export paths are clear before relying on the setup.
Comparison
Compare Kubernetes logging tools by responsibility
A collector, a storage/query stack, a cloud logging product, and a log management destination solve different parts of Kubernetes logging.
Example Kubernetes log collection path
text
1pod stdout/stderr2 -> node container log files3 -> Fluent Bit, Vector, or OpenTelemetry Collector4 -> Kubernetes metadata enrichment5 -> HTTP, OTLP, Syslog, or GELF output6 -> Fluxtail receiver7 -> Fluxtail stream with live tail, filters, facets, histograms, and alerts
The collector gets the logs out of the cluster. Fluxtail receives the forwarded records and keeps them readable.
| Layer | Examples | Does | Does not do alone |
|---|---|---|---|
| Collector or agent | Fluent Bit, Fluentd, Vector, OpenTelemetry Collector | Reads Kubernetes logs, enriches records with metadata, and forwards them | Usually not the long-term search, alerting, or live analysis surface |
| Storage and query stack | Loki, Elastic, OpenSearch, ClickHouse-based stack | Stores logs and supports query patterns over retained data | Does not automatically guarantee clean collection or readable log rows |
| Cloud provider logging | AWS, Google Cloud, Azure logging products | Integrates with provider infrastructure and managed retention/search | May be less portable when clusters span providers or self-hosted environments |
| Log management destination | Fluxtail | Receives logs into streams with live tail, filters, facets, histograms, alerts, MCP diagnostics, and AI chat | Still expects a collector, agent, or emitter to send pod and container logs from the cluster |
Comparison
Compare Kubernetes logging tools by responsibility
A collector, a storage/query stack, a cloud logging product, and a log management destination solve different parts of Kubernetes logging.
Example Kubernetes log collection path
text
1pod stdout/stderr2 -> node container log files3 -> Fluent Bit, Vector, or OpenTelemetry Collector4 -> Kubernetes metadata enrichment5 -> HTTP, OTLP, Syslog, or GELF output6 -> Fluxtail receiver7 -> Fluxtail stream with live tail, filters, facets, histograms, and alerts
The collector gets the logs out of the cluster. Fluxtail receives the forwarded records and keeps them readable.
| Layer | Examples | Does | Does not do alone |
|---|---|---|---|
| Collector or agent | Fluent Bit, Fluentd, Vector, OpenTelemetry Collector | Reads Kubernetes logs, enriches records with metadata, and forwards them | Usually not the long-term search, alerting, or live analysis surface |
| Storage and query stack | Loki, Elastic, OpenSearch, ClickHouse-based stack | Stores logs and supports query patterns over retained data | Does not automatically guarantee clean collection or readable log rows |
| Cloud provider logging | AWS, Google Cloud, Azure logging products | Integrates with provider infrastructure and managed retention/search | May be less portable when clusters span providers or self-hosted environments |
| Log management destination | Fluxtail | Receives logs into streams with live tail, filters, facets, histograms, alerts, MCP diagnostics, and AI chat | Still expects a collector, agent, or emitter to send pod and container logs from the cluster |
Related
Related pages
Related page
Kubernetes logging
Practical guide to Kubernetes logging: what metadata to keep, how collection paths work, and how Fluxtail keeps cluster logs searchable.
Related page
Kubernetes log aggregation
Aggregate Kubernetes logs from pods, nodes, and collectors into Fluxtail streams with live tail, filters, facets, histograms, alerts, MCP diagnostics, and AI chat.
Related page
Fluent Bit for Kubernetes
Use Fluent Bit for Kubernetes log collection, forward records to a Fluxtail receiver, and keep namespace, pod, container, service, and severity fields readable.
Related page
OpenTelemetry logging
Learn how OpenTelemetry logging preserves service, resource, trace, and severity context with readable Fluxtail log views.
Related page
Live log viewer
Watch production logs in real time with a live log viewer for readable streams, live tail, filters, and diagnostics.
Related page
Log management
Centralized log management with readable live tail, clear streams, and straightforward ingest.
Related page
Log aggregation
Learn how log aggregation works across apps, hosts, syslog, containers, Kubernetes, OTLP, GELF, and collectors.
Next step
Send one real source and read the logs
The fastest check is to point one real source at Fluxtail and see whether the resulting stream is easier to read.
Point one real source at Fluxtail.
Create a receiver, send one source, and inspect the first stream.