Fluxtail
Log Management Guides

Data Ingestion Example: Send and Verify One JSON Log

Send one JSON log to a Fluxtail HTTP receiver, check the 202 response, and verify the fresh event in Live Tail before adding batching or retries.

By Fluxtail Engineering Updated

A useful data ingestion example proves both sides of the path: the receiver accepts one fresh JSON log, and the intended stream later shows that same event. For Fluxtail's HTTP JSON receiver, send a POST to the exact receiver URL over HTTPS with Content-Type: application/json and a receiver-bound Bearer token. A successful single-event request returns HTTP 202 and {"accepted":1}. That means accepted for processing, not yet proof that the row is stored and searchable. Verify it in Live Tail before building a larger sender. HTTP JSON guide, Ingest API reference.

Prepare the destination and one synthetic event

In the Fluxtail account where you intend to see the log, create or select a stream, then create an active HTTP JSON receiver routed to that stream. Copy the exact receiver URL; it has this form:

https://ingest.fluxtail.io/v1/receivers/RECEIVER_ID/logs

Replace RECEIVER_ID with the ID in your own receiver URL. Do not append a second /logs or add a query parameter to choose a stream; routing is configured on the receiver. Create a token with ingest:write bound to that same receiver. A token for another receiver is not interchangeable. The public first-log guide shows the stream, receiver, and token setup.

The test below sends a small synthetic object with a UTC event time, message, severity, service, host, and a low-cardinality environment label. These are documented log payload fields. It also includes a unique request_id as additional structured context. The marker is placed in the message too, so the verification does not depend on an extra field being preserved or exposed as an exact filter by this receiver. Use no customer identifiers, payloads, passwords, or access tokens in the event.

Send one log without putting the token on the command line

Run this in an interactive Bash shell with curl 7.76 or newer. Paste your receiver's exact URL into the first line. When prompted, paste its receiver-bound token; the prompt hides input. Bash's built-in printf passes one authorization header to curl --header @- on standard input, rather than expanding the token into curl's process arguments or writing a credential file. This also avoids interpreting token characters as curl configuration syntax. The JSON event is synthetic and safe to appear as a command argument. The token still exists briefly in shell and curl memory, so use an authorized workstation and avoid shell tracing or verbose curl output. -q is curl's first option to ignore user curl configuration, including unexpected redirect or tracing defaults. curl documents headers from standard input and --fail-with-body.

receiver_url='https://ingest.fluxtail.io/v1/receivers/RECEIVER_ID/logs'
read -r -s -p 'Receiver token: ' receiver_token
printf '\n'

event_time=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
marker="ingest-smoke-$(date -u +'%Y%m%dT%H%M%SZ')-$$-$RANDOM-$RANDOM"
printf -v event '{"timestamp":"%s","message":"%s","severity":"INFO","service_name":"ingest-smoke","host":"manual-test","labels":{"environment":"test"},"request_id":"%s"}' "$event_time" "$marker" "$marker"

printf 'Authorization: Bearer %s\n' "$receiver_token" |
  curl -q --header @- --silent --show-error --fail-with-body --max-time 15 \
    --request POST "$receiver_url" \
    --header 'Content-Type: application/json' \
    --data-binary "$event" \
    --write-out '\nHTTP %{http_code}\n'
unset receiver_token
printf 'Search for: %s\n' "$marker"

Use the URL shown by Fluxtail rather than constructing another hostname or protocol path. Leave TLS certificate verification enabled. This example does not use --retry: if a transfer times out after the server received the request, the sender cannot tell from the timeout alone whether it was accepted. For automated production use, obtain the secret from an approved secret store and make the sender's retry, deduplication, and failure-reporting behavior explicit; do not copy an interactive prompt into a background job.

On success, the body and final status should look like this (the marker itself changes on every run):

{"accepted":1}
HTTP 202

accepted counts objects in the request accepted for processing. The HTTP JSON endpoint accepts either one JSON object or a non-empty JSON array; it is not a newline-delimited JSON endpoint. A 202 is an ingest-boundary acknowledgment, not a storage receipt, exactly-once guarantee, or promise that every application field became a filter. HTTP JSON OpenAPI contract, log payload reference.

Verify the row in the intended stream

Open Live Tail in the same account and select the stream configured on the receiver. Search the exact marker printed by the command, within a time window that includes the fresh event. Clear unrelated filters if the row is not immediately visible. Confirm the message, event time, severity, service name, host, and environment label in the stored row. Check whether the extra request_id was preserved before relying on it in later investigations.

A visible row proves that this one event became stored and readable by the active account. It does not prove that future batches, a different token, another container, or an unattended sender will behave the same way. If the 202 arrived but the row is absent, allow for processing delay, then verify the active account, selected stream, receiver route, time range, and filters. Do not change payload fields to override the receiver's configured stream. The no-logs troubleshooting guide covers these boundaries.

Use a new marker on every test. A fixed message or old timestamp can make an earlier row look like a successful current run, or place the event outside the visible time window.

Interpret errors before retrying

Use the HTTP status, curl's transfer result, and the response body as separate evidence. The public Ingest API reference documents these failures:

  • 400: malformed JSON or invalid payload shape. Correct the object before another attempt.
  • 401: missing, conflicting, or invalid token. Check the single Bearer header without printing the token.
  • 403: receiver, protocol, or source-IP policy denied. Confirm the token's binding and the receiver policy.
  • 404: unknown route or receiver identity. Recopy the exact URL and confirm that receiver still exists.
  • 413: request body too large. Reduce object size or split a later batch.
  • 415: unsupported content type or encoding. Use application/json and a JSON object or array here.
  • 429: account or receiver event limit exceeded. Respect the limit and reduce load; repeated immediate requests are not a fix.
  • 503: a required ingest dependency is unavailable. Pause or use bounded backoff, but do not assume the previous request was rejected without checking its outcome.

A network error or timeout may leave the outcome unknown even when no HTTP status was seen. Preserve the marker and check the intended stream before any manual retry. If the event is still absent, a retry may be reasonable under a documented duplicate policy; it is not exactly-once delivery. Conversely, do not endlessly poll or retry a 400, 401, 403, 404, or 415 configuration error. Avoid curl --verbose, trace dumps, copied authorization headers, or raw response bundles in shared tickets because they can expose credentials or operational data.

Move from one event to a reliable sender

After the single-event path works, a sender can batch multiple objects in a JSON array. A successful batch response reports the number of accepted objects. Keep batch size configurable and respond to 413 by reducing it; the public API contract does not promise a universal byte or object-count limit. Each event still needs a useful message, event timestamp when available, consistent severity, stable service identity, and small non-sensitive labels. Additional fields require verification at the receiver and read path.

For unattended ingestion, define a bounded queue, finite request timeout, retry budget with backoff and jitter for potentially transient failures, and a way to report queued, rejected, retried, and dropped events. A sender must decide how it handles an unknown outcome; replaying a whole accepted-or-uncertain batch can create duplicates. Never claim that the receiver automatically deduplicates request_id, or that 202 guarantees a row will later appear. Test one marker end to end after each route or credential change and monitor the sender as well as the destination.

Fluxtail is a paid Starter/Pro, logs-focused destination for this example. The same receiver can serve an application or a compatible collector, but the collector's mapping, buffering, and retry behavior must be checked separately. Once the smoke test succeeds, use search and filters to confirm that the fields you plan to investigate are actually present, then document that tested schema for the producing service.