Data Ingestion Example: Send Logs with HTTP JSON

A useful data ingestion example should prove one complete path: create a destination, send one known event, read the receiver response, and confirm the event in the correct log stream. This guide does that with Fluxtail's HTTP JSON receiver. It uses the documented request shape and does not assume undocumented routing parameters, response fields, or delivery guarantees.

The result is a repeatable smoke test for a new service, deployment script, or collector. Once the single-event path works, the same receiver can accept a JSON array for batching.

Table of contents

• What this example sends
• How the ingestion path works
• Step 1: create the stream and receiver
• Step 2: prepare a structured log event
• Step 3: send one event
• Step 4: verify the event in Live Tail
• Batch multiple events
• Handle failures without unsafe retries
• Troubleshooting checklist
• Turn the example into a production sender

What this example sends

The example sends one JSON object over HTTPS. It includes the fields that make a log useful during an investigation:

• timestamp: when the application created the event, in ISO 8601 format
• message: a stable, human-readable description
• severity: a consistent level such as INFO, WARN, or ERROR
• service_name: the application or component that emitted the event
• service_namespace: an optional larger grouping for related services
• host: the host or instance that emitted the event
• labels: small dimensions used for exact-match filtering, such as environment and region
• request_id: additional structured context used to find this exact test event

The first seven fields follow Fluxtail's documented common payload. HTTP JSON accepts ordinary JSON objects and can preserve additional structured data where the receiver permits it. That makes a request identifier useful for this test without pretending that it is a required field. The log payload field reference is the source of truth when standardizing the event shape used by an application.

Do not include passwords, access tokens, session cookies, payment data, or other secrets in the message, labels, or additional fields. A log platform can only protect data that a sender was allowed to transmit; preventing sensitive values from entering the pipeline is the safer first control.

How the ingestion path works

The route is intentionally simple:

application or script
        |
        | HTTPS POST + receiver-bound Bearer token
        v
HTTP JSON receiver
        |
        | configured receiver route
        v
named Fluxtail stream
        |
        v
Live Tail verification

A receiver is the authenticated inbound boundary. A stream is the account-owned destination used to organize accepted events. The stream is chosen when the receiver is created or edited. The sender should use the exact receiver URL and must not add a query parameter to select a stream. Sender fields such as stream_id, host, or a tag cannot switch accounts or bypass the receiver's configured route.

This separation is useful during troubleshooting. The HTTP response proves what happened at the receiver boundary. Looking in the configured stream proves whether the accepted event became visible to the account.

Step 1: create the stream and receiver

In Fluxtail, create a stream for the test. A name such as Quickstart is clear and easy to find. Then create an HTTP JSON receiver and route it to that stream.

Copy the receiver URL exactly as shown. It has this shape:

https://ingest.fluxtail.io/v1/receivers/RECEIVER_ID/logs

Next, create an access token with ingest:write that is bound to the same receiver. Store the URL and token in shell variables rather than inserting a real secret into source code or a tracked configuration file:

export FLUXTAIL_RECEIVER_URL='PASTE_THE_EXACT_RECEIVER_URL'
export FLUXTAIL_TOKEN='PASTE_THE_RECEIVER_BOUND_TOKEN'

The URL, receiver identity, token, account, and protocol must agree. A valid token belonging to a different receiver is not interchangeable with the token created for this HTTP JSON endpoint. The HTTP JSON receiver guide documents this request contract and should be checked before changing client behavior.

Step 2: prepare a structured log event

Start with one small event. A single object makes syntax, authentication, and routing failures easier to isolate than a production-sized batch.

{
  "timestamp": "2026-08-30T14:30:00Z",
  "message": "checkout ingestion smoke test",
  "severity": "INFO",
  "service_name": "checkout-api",
  "service_namespace": "storefront",
  "host": "checkout-api-01",
  "labels": {
    "environment": "staging",
    "region": "ca-east"
  },
  "request_id": "ingest-test-20260830-143000"
}

Use a timestamp generated by the sender when it has a reliable event time. Keep severity spelling consistent across services so a filter does not need to handle many variations. Use a stable service name rather than a container ID that changes on every deployment. Put only small filtering dimensions in labels; detailed structured context can remain in additional fields when the receiver supports preserving it.

The message and request identifier are deliberately unique. They make it possible to distinguish the new event from an older smoke test with the same service and severity.

Step 3: send one event

Send the object to the exact URL copied from Fluxtail:

curl --fail-with-body -X POST "$FLUXTAIL_RECEIVER_URL" \
  -H "Authorization: Bearer $FLUXTAIL_TOKEN" \
  -H 'Content-Type: application/json' \
  -d '{
    "timestamp": "2026-08-30T14:30:00Z",
    "message": "checkout ingestion smoke test",
    "severity": "INFO",
    "service_name": "checkout-api",
    "service_namespace": "storefront",
    "host": "checkout-api-01",
    "labels": {
      "environment": "staging",
      "region": "ca-east"
    },
    "request_id": "ingest-test-20260830-143000"
  }'

The required request details are:

• method: POST
• content type: application/json
• authentication: one Authorization: Bearer ... header
• body: one JSON object or an array of JSON objects

A successful single-event request returns HTTP 202 with this body:

{"accepted":1}

The 202 response means the receiver accepted and queued one object. It does not replace the Live Tail check in the next step. Keep both checks in a deployment smoke test because they answer different questions.

For a diagnostic run that needs the response headers and body, add -i:

curl -i -X POST "$FLUXTAIL_RECEIVER_URL" \
  -H "Authorization: Bearer $FLUXTAIL_TOKEN" \
  -H 'Content-Type: application/json' \
  -d '{"message":"receiver diagnostic","severity":"INFO","service_name":"checkout-api"}'

Do not append /logs if the copied receiver URL already includes it. Do not add ?stream=...; stream routing belongs to the receiver configuration.

Step 4: verify the event in Live Tail

Open Live Tail and select the stream configured on the receiver. Search for checkout ingestion smoke test or the exact request identifier ingest-test-20260830-143000.

Confirm these facts:

1. The event appears in the intended stream.
2. The message matches the test payload.
3. Severity is INFO.
4. Service is checkout-api.
5. Host is checkout-api-01.
6. The environment and region labels are available for filtering.

If the request returned 202 but no event appears, first clear Live Tail filters and confirm the selected stream. Then inspect the receiver's configured destination. Changing a payload field is not the correct way to override the receiver route.

This verification should use a fresh identifier on every run. Reusing one value makes it easy to mistake an older event for the current deployment's result.

Batch multiple events

After the single object works, send a JSON array to reduce per-request overhead:

curl --fail-with-body -X POST "$FLUXTAIL_RECEIVER_URL" \
  -H "Authorization: Bearer $FLUXTAIL_TOKEN" \
  -H 'Content-Type: application/json' \
  -d '[
    {
      "timestamp": "2026-08-30T14:31:00Z",
      "message": "batch event one",
      "severity": "INFO",
      "service_name": "checkout-api"
    },
    {
      "timestamp": "2026-08-30T14:31:01Z",
      "message": "batch event two",
      "severity": "WARN",
      "service_name": "checkout-api"
    }
  ]'

For a successful batch, the accepted value is the number of accepted objects. HTTP JSON batching uses a JSON array. It is not newline-delimited JSON, and it should not be confused with the OTLP request format. OTLP has its own endpoint, content types, and response contract.

If the edge returns 413, reduce the number or size of objects in each request. Do not hard-code an undocumented maximum into the sender. Start with conservative batches, observe actual responses, and make request sizing configurable.

Handle failures without unsafe retries

Retry behavior should follow the HTTP status and preserve uncertainty. Fluxtail documents the following receiver errors:

Do not describe the receiver as idempotent unless that behavior is explicitly documented for the endpoint. If a client times out before receiving the response, it may not know whether the server accepted the request. A resend can therefore produce a duplicate. Use bounded exponential backoff with jitter for temporary failures, retain enough local context to investigate ambiguous results, and make downstream processing tolerant of duplicate log events where practical.

A 4xx response generally requires a change to the request, credentials, policy, or send rate. Blindly replaying the same malformed or unauthorized request only creates noise. A 503 is different because it represents temporary unavailability, but retries should still be limited.

Troubleshooting checklist

Use this order to isolate a failure quickly:

1. Confirm that FLUXTAIL_RECEIVER_URL is the complete URL copied from the receiver.
2. Confirm that the token has ingest:write and is bound to that receiver.
3. Confirm that the request sends exactly one Bearer authorization header.
4. Confirm Content-Type: application/json.
5. Validate that the body is one JSON object or a JSON array.
6. Read the HTTP status and response body before changing the application.
7. If the result is 413, reduce the batch size.
8. If the result is 202, open the receiver's configured stream in Live Tail.
9. Clear filters and search for the fresh test message or request identifier.
10. Confirm the receiver route instead of trying to select a stream from the sender.

Authentication is checked before the ingest service reads or decodes the body. That means a 401 or 403 should be solved at the token, receiver, protocol, or policy boundary before spending time changing JSON fields.

Turn the example into a production sender

The smoke test is intentionally small, but its contract can become a production health check. Keep the receiver URL and token in a secret manager or protected runtime configuration. Generate timestamps at the event source. Standardize severity and service names. Redact secrets before serialization. Make batch size and retry limits configurable. Emit a unique diagnostic value during deployments, and verify it in the intended stream.

Most importantly, keep protocol contracts separate. This article demonstrates HTTP JSON: one JSON object or an array sent to the HTTP JSON receiver. Syslog, GELF, and OTLP are different sender paths with their own request formats. Reusing an HTTP example as if it were an OTLP payload would create a misleading integration guide.

For a quick proof, start with Fluxtail, create one stream and one HTTP JSON receiver, send the single event above, and verify it in Live Tail before adding batches or more sources.

The complete success condition is straightforward: the receiver returns 202 with {"accepted":1}, and the same uniquely identified event appears in the stream configured for that receiver. That is a small test, but it proves the full ingestion path without relying on invented parameters or assumptions.