Rust's tracing crate records structured events and spans: an event marks something that happened, while a span represents a period of work and can carry fields into its events. A subscriber decides which records are enabled and how they are written. For a practical first setup, use tracing-subscriber to emit newline-delimited JSON to stdout; add a separate trace exporter only if you actually need distributed traces. The Tokio tracing project documents this separation between instrumentation and collection.
Run a JSON stdout example
Create a new binary project with cargo new tracing-demo, then use these dependencies in Cargo.toml:
[dependencies]
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter", "json"] }
Replace src/main.rs with:
use tracing::{info, instrument};
use tracing_subscriber::EnvFilter;
#[instrument(skip_all, fields(job_kind = "import"))]
fn run_job() {
let run_id = "demo-42";
info!(%run_id, "job started");
info!(items = 3, "job completed");
}
fn main() {
let filter = EnvFilter::try_from_default_env().unwrap_or_else(|_| EnvFilter::new("info"));
tracing_subscriber::fmt()
.json()
.with_env_filter(filter)
.with_writer(std::io::stdout)
.init();
run_job();
}
Run cargo run --quiet. You should get two JSON lines. Each line includes metadata such as a timestamp and level, a fields object containing the event message and typed values, and span context for run_job. The exact keys and formatting depend on the subscriber version and options. #[instrument(skip_all, fields(...))] prevents automatic recording of function arguments, then adds only the fixed job_kind field. The instrument attribute reference explains skip, skip_all, and explicit fields; the subscriber formatter reference documents its JSON-lines format.
The example uses RUST_LOG through EnvFilter; if the variable is missing or invalid, it falls back to info. Try RUST_LOG=debug cargo run --quiet and then narrow a target with a directive such as RUST_LOG=warn,tracing_demo=info cargo run --quiet. The actual target is derived from your crate/module name, so check the emitted target field when adapting the command. Do not treat an invalid filter silently as a production configuration success: validate operator-provided directives at startup when an incorrect level would affect required records. The EnvFilter API defines directive matching.
Distinguish spans from the JSON events they decorate
The run_job span starts when the instrumented function runs and is the parent context of the two info! events. JSON formatting can include the current span or span stack in each event. That does not mean the application exported an OpenTelemetry span, supplied a distributed trace ID, or followed a request across network hops. It means the subscriber wrote log records containing local span context.
An event should answer a concrete question: what happened, for which operation, and what was the outcome? A span should identify a meaningful operation with a stable name and bounded fields. Choose field names and types deliberately. In the example, items is numeric while job_kind and run_id are strings. Avoid formatting every structure with ?value; a Debug implementation can include secrets, personal data, large payloads, or high-cardinality internals. #[instrument] records arguments by default, so skip_all or explicit skip(...) is important at sensitive boundaries.
Keep levels tied to event meaning. trace! and debug! are for diagnostics that may be disabled in normal operation; info! for expected progress; warn! for recoverable anomalies; error! for failed work. An ERROR event does not by itself establish incident severity or justify a page. Filtering at the producer or subscriber means excluded records never reach the log collector. During an incident, use a bounded change to the filter and verify both volume and data exposure.
Preserve context in async work
tracing is useful in asynchronous Rust, but a span's scope is not automatically equivalent to an async task's lifetime. Holding a guard returned by Span::enter() across an .await can attribute other polled work to the wrong span. The official Span documentation warns against that pattern. For an async function, #[instrument] instruments the returned future; for a manually created future or spawned task, use the Instrument trait to attach the intended span when it is polled.
Even correctly scoped local spans do not create cross-process correlation on their own. If one service calls another, distributed tracing requires an explicit propagation protocol, compatible instrumentation on both sides, and a trace exporter/backend. Do not infer a trace ID from a job ID, nor trust incoming correlation fields as authorization. Keep external values bounded and validated before adding them to a span or event.
Make the output safe and useful in production
Initialize the global subscriber once in the application entry point. Libraries should emit tracing events and let the application select a subscriber; attempting to initialize it again can fail or panic depending on whether you call try_init() or init(). Prefer try_init() if the embedding environment may already have installed a subscriber. The subscriber builder reference documents that behavior.
For each event family, decide which fields operators will filter by and which fields should never be emitted. A reasonable small contract includes a stable message, service and environment from trusted configuration, operation kind, status or outcome, and a safe correlation ID when one exists. Log neither request bodies nor credentials, tokens, cookies, arbitrary headers, or raw error structures by default. Limit cardinality in indexed fields and retain source context without turning untrusted values into control fields.
Test the JSON format with a parser, not only visual inspection. In a container, confirm the application writes complete newline-delimited records to stdout, then inspect the container log. Verify that a normal event and a failure event appear with the intended level and fields. Simulate a collector outage in a controlled environment to understand buffering, dropped records, and recovery; tracing instrumentation by itself gives no end-to-end delivery guarantee. Keep event timestamps and observed/ingest times distinct when comparing delayed records.
Check missing or duplicate output at each boundary
If cargo run emits nothing, first check that the subscriber initialized before run_job() and that RUST_LOG enables the event's target and level. A filter directive for a different crate name will not enable this application's events. Confirm which stream the process writes and whether its runner captures stdout. If local JSON exists but a container log is empty, inspect the container runtime's logging configuration before changing application code.
If the container log has a line but the backend does not, inspect collector parsing, receiver response, authentication, and buffering. If the backend shows duplicates, check whether the application emits the same failure at multiple layers or whether two collectors read the same source. Avoid treating repeated lines as proof that a job ran twice until those paths are ruled out. For an error event, confirm its actual level, fields.message, and safe correlation fields in the JSON before writing downstream filters. A displayed span name is useful context, not a guaranteed distributed trace relationship.
Route logs to Fluxtail without promising trace ingestion
The practical path here is tracing-subscriber JSON stdout → Docker's captured stdout → a configured Fluent Bit collector → Fluxtail's HTTP JSON log receiver. Fluxtail's Docker with Fluent Bit guide documents the receiver-bound token and collector path. The collector must parse the Docker line and the nested tracing JSON, then map a nonempty event message plus supported fields such as timestamp, severity, and service name into the HTTP JSON log payload. Verify that mapping with a unique, non-sensitive marker in Live Tail before relying on search filters. Do not assume raw tracing-subscriber JSON, with its nested fields.message, is already the receiver's final payload shape.
Fluxtail is a paid Starter/Pro, logs-focused service. Its OTLP receiver accepts logs only, not spans or metrics. The Rust OpenTelemetry exporter documentation describes exporters for separate telemetry pipelines; it is not evidence that an OTLP trace exporter can send spans to Fluxtail. If distributed traces are a requirement, choose a trace-capable backend independently. Fluxtail can help inspect log events that a supported collector has actually delivered, but cannot reconstruct missing spans from them.