Syslog Forwarding: A Practical Setup and Scaling Guide

A syslog pipeline usually looks simple until a collector restarts, a firewall team changes a rule, or a device keeps speaking an older message format. The logs are still being produced, but the path between source and receiver is now the core system to reason about. Syslog forwarding is the discipline of making that path explicit, observable, and fit for the type of logs being moved.

At the wire level, syslog forwarding is about three roles. A source emits the event, a relay or agent packages and ships it, and a collector receives and stores it. That collector may not read local logs at all, which is why forwarding is different from just writing to /var/log and hoping someone notices later. The practical choice starts with transport, then framing, then the endpoint that will accept the messages.

Table of Contents

• What Syslog Forwarding Actually Does
  • The transport choice changes the failure mode
• RFC 3164 vs RFC 5424 and Why It Matters
  • The wire format tells the parser how much it can trust
• rsyslog Configuration for Forwarding
  • A minimal forwarding file that stays readable
  • Queueing is the actual reliability control
• syslog-ng Configuration for Forwarding
  • A working source, destination, and log path
  • Formatting the outbound message matters too
• Forwarding Into Fluxtail
  • Map the sender to the documented receiver path
• Routing and Stream Strategy
  • Route on context, not just on volume
  • Make failure visible instead of silent
• Validation, Troubleshooting, and Day One Best Practices
  • A simple validation sequence
  • The habits that keep the pipeline stable

What Syslog Forwarding Actually Does

Syslog forwarding moves log messages off the machine or device that created them and onto a remote receiver for retention, search, and incident work. The source can be a Linux host, a network appliance, or a relay daemon running on an intermediate box. The relay is usually where policy lives, because that's where the pipeline can filter, queue, and choose where each class of event should go. The basics of log management matter here because forwarding only helps if the receiver can make the data usable again.

The transport choice changes the failure mode

UDP 514 is the traditional syslog path. RFC 3164 treated any IP packet sent to UDP destination port 514 as a syslog message, and that historical habit still shows up in older environments (RFC 5424 history and transport notes). UDP is lightweight and easy to enable, but it's also fire-and-forget. If a collector is congested, rebooting, or unreachable, the sender usually has no clean way to know which messages never arrived.

TCP changes that behavior. It gives ordered delivery at the connection level and lets the sender retry when the session drops, but it also introduces state, buffering, and framing concerns. TLS on 6514 adds encryption and peer authentication on top of that stream, and RFC 5425 assigns TCP port 6514 as the default for syslog over TLS (RFC 5425). That doesn't remove the need for queues, though, because transport encryption doesn't magically make forwarding resilient.

Practical rule: pick UDP only when occasional loss is acceptable, pick TCP when ordered delivery matters, and add TLS when the logs cross a trust boundary or the receiver must verify the peer.

A working design always answers three questions before anyone writes a config file. Which transport will carry the messages, which framing will the receiver accept, and which endpoint will receive them. If those three aren't decided first, forwarding tends to fail in ways that look like “the logs disappeared.”

RFC 3164 vs RFC 5424 and Why It Matters

A syslog pipeline breaks in small ways when the sender and receiver disagree on format. Two messages from the same device can arrive with different parse results if one side emits legacy BSD-style syslog and the other expects the newer standard defined in RFC 5424 history. For a router, that is not a cosmetic difference. It changes what the collector can extract, route, and preserve.

The wire format tells the parser how much it can trust

RFC 3164 uses the older BSD-style line. It starts with PRI, then a free-form timestamp, hostname, tag, and message. There is no standardized structured data block, and the timestamp carries less context. RFC 5424 adds a version field, a structured header, year-aware timestamps, UTF-8 support, and structured content, which makes forwarded logs easier to normalize across mixed sources and downstream parsers, including how log parsing handles both formats.

Older Cisco IOS and Juniper Junos appliances often default to RFC 3164 unless you reconfigure them. The collector should accept that reality without treating the older format as the preferred one. A practical ingest path normalizes both formats, then keeps the fields that still identify origin, severity, and application context.

Operational takeaway: turn on RFC 5424 on anything under direct control, accept RFC 3164 from appliances, and make the receiver tolerant of both without losing source identity.

The PRI field still matters in both formats. RFC 5424 defines PRI = Facility × 8 + Severity, with eight severity levels from Emergency (0) to Debug (7), and the formula works because PRI is an 8-bit value where the three least significant bits carry Severity and the remaining five bits carry Facility (RFC 5424, PRI field basics). That lets forwarding systems route on one number while still preserving origin context.

rsyslog Configuration for Forwarding

rsyslog works best when the forwarding config is small and opinionated. Load the inputs that matter, define the outbound actions clearly, and give the queue enough room to survive a collector outage. The rest is usually just syntax.

A minimal forwarding file that stays readable

module(load="imuxsock")      # Read local system logs from the Unix socket.
module(load="imjournal")     # Read systemd journal entries when journald is in use.
module(load="imudp")         # Enable UDP input if the host also receives remote syslog.
module(load="imtcp")         # Enable TCP input if the host also receives remote syslog.

# Forward noisy legacy events over UDP.
action(
  type="omfwd"
  target="collector.example"
  port="514"
  protocol="udp"
)

# Forward reliable streams over TCP with octet-counted framing.
action(
  type="omfwd"
  target="collector.example"
  port="601"
  protocol="tcp"
  action.resumeRetryCount="-1"
)

# Forward with TLS on 6514.
action(
  type="omfwd"
  target="collector.example"
  port="6514"
  protocol="tcp"
  StreamDriver="gtls"
  StreamDriverMode="1"
  StreamDriverAuthMode="x509/name"
  StreamDriverPermittedPeers="collector.example"
  queue.type="LinkedList"
  queue.filename="fwd-tls"
  queue.maxdiskspace="500m"
  queue.saveonshutdown="on"
  action.resumeRetryCount="-1"
)

The TLS action above uses the documented forwarding model where syslog over TLS runs on TCP 6514 and depends on explicit peer authentication (rsyslog forwarding guidance). The important part isn't just the crypto, it's the queue. If the session breaks or the collector goes away, forwarding remains safe only if the sender has somewhere to hold messages until the path recovers.

Queueing is the actual reliability control

A forwarding pipeline that depends on memory alone is fragile. Disk-assisted queues give rsyslog a place to buffer during collector outages, restart cycles, and network churn. That's the part operators want when they say they need reliability, because TLS by itself only protects the transport, it doesn't guarantee end-to-end delivery. The queue is what keeps the sender from turning a temporary outage into permanent loss.

For validation, rsyslog should be checked before restart. The file syntax check is the first gate:

rsyslogd -N1 -f /etc/rsyslog.conf

That command catches malformed directives before the daemon reloads a broken config. It's cheap, and it avoids the common failure mode where the forwarding change looked fine in review but never started.

syslog-ng Configuration for Forwarding

syslog-ng tends to feel cleaner when the routing logic is separated from the transport. Source blocks define what the host can hear, destination blocks define where it can send, and log paths connect the two. That makes it easier to preserve semantics when the fleet mixes older hosts with newer ones.

A working source, destination, and log path

@version: 3.38
@include "scl.conf"

options {
    keep-hostname(yes);
    mark-freq(60);
    time-reap(30);
};

source s_src {
    system();
    udp(ip("0.0.0.0") port(514));
    tcp(ip("0.0.0.0") port(601));
    unix-stream("/dev/log");
};

destination d_tls {
    network(
        "collector.example"
        port(6514)
        transport("tcp")
        tls(
            ca-dir("/etc/syslog-ng/ca.d")
            cert-file("/etc/syslog-ng/cert.pem")
            key-file("/etc/syslog-ng/key.pem")
            peer-verify(optional-untrusted)
        )
    );
};

filter f_authpriv { facility(authpriv); };

rewrite r_keep_source { set("app-host" value("HOST")); };

log {
    source(s_src);
    filter(f_authpriv);
    rewrite(r_keep_source);
    destination(d_tls);
};

That layout keeps the semantics visible. keep-hostname(yes) preserves the sender's hostname when it arrives, which matters when the collector is supposed to keep source identity intact. The TLS destination uses the network transport with certificate material attached, and the log path can route only the subset that matters for a given collector.

Formatting the outbound message matters too

syslog-ng can emit a structured RFC 5424 message so the receiver doesn't have to guess at field boundaries. That's useful when the upstream source is messy or the collector needs consistent headers for downstream parsing. The receiver side becomes simpler when the forwarder does the normalization before shipping.

Reload and verification are also part of the job. syslog-ng provides its own control interface for reloading and checking the active config without treating every edit like a full service outage. That keeps change windows tight and makes forwarding less brittle during rollout.

Forwarding Into Fluxtail

Fluxtail documents syslog ingestion as a receiver-based path, so the sender points at an assigned host and port and forwards into the platform's stream model. Public access and pricing are available by request, so the operational question is usually how to shape the messages before they land, not how to discover the product exists. A data ingestion example is the most relevant place to line up sender and receiver behavior.

Map the sender to the documented receiver path

For rsyslog, the documented pattern is omfwd to the Fluxtail host over TCP with TLS on 6514, with StreamDriverMode=1 so the sender treats the link as TLS-enabled. For syslog-ng, the equivalent is a network() destination with TLS turned on and the receiver configured to accept the secure connection. Those two approaches match the same forwarding idea, but the syntax is different because the agents are different.

A practical rsyslog snippet looks like this:

action(
  type="omfwd"
  target="fluxtail-host.example"
  port="6514"
  protocol="tcp"
  StreamDriver="gtls"
  StreamDriverMode="1"
  StreamDriverAuthMode="x509/name"
  StreamDriverPermittedPeers="fluxtail-host.example"
  queue.type="LinkedList"
  action.resumeRetryCount="-1"
)

And the equivalent syslog-ng destination stays close to the same model:

destination d_fluxtail {
    network(
        "fluxtail-host.example"
        port(6514)
        transport("tcp")
        tls(
            ca-dir("/etc/syslog-ng/ca.d")
            cert-file("/etc/syslog-ng/cert.pem")
            key-file("/etc/syslog-ng/key.pem")
            peer-verify(required-trusted)
        )
    );
};

Practical rule: if the sender is using TLS, verify the certificate chain, hostname, and port before blaming the collector.

The same section should also cover verification, because a config that looks right still needs proof. A sender-side debug log is the fastest place to check connection setup, certificate trust, and reconnect behavior during onboarding. Once messages reach the receiver, the query path should show them in the intended stream, which is the confirmation that routing matches expectation.

Routing and Stream Strategy

Forwarding becomes much easier to operate when it's treated as a routing problem instead of a pure transport problem. Mixed legacy and cloud estates rarely need one pipe for everything. They need clear rules about which messages belong together, where they go, and how to avoid silent loss when a destination is down.

Route on context, not just on volume

Facility and severity are the first obvious filters because they already exist in the syslog header. A host generating authpriv messages deserves a different path from a noisy background service, and emergency or alert-level events should not be mixed with routine chatter. Hostname, program name, and message content are the next filters when a team needs to split by service or environment.

A representative rsyslog rule can look like this:

if ($syslogfacility-text == 'authpriv' and $syslogseverity <= 4) then {
    action(
      type="omfwd"
      target="collector.example"
      port="6514"
      protocol="tcp"
      StreamDriver="gtls"
      StreamDriverMode="1"
      action.resumeRetryCount="-1"
    )
    stop
}

The same idea in syslog-ng uses filters and a log path rather than inline conditionals. That structure is easier to extend when one collector wants security events and another wants application noise. The key is to preserve enough source identity that the receiver can still explain where a message came from after it gets routed.

Make failure visible instead of silent

A dead-letter queue or retrying action is the right answer when a destination is unreachable. Silent drop is the worst possible default because it makes a logging pipeline look healthy while records are disappearing. Backpressure, restart gaps, and uneven load distribution are all real risks in syslog routing paths, especially once collectors sit behind a shared entry point and multiple sources converge.

Operational takeaway: route aggressively, but never route without a queue behind it.

Fluxtail fits naturally here as one destination option for centrally organized streams, but the same routing logic applies anywhere a team wants to preserve source identity and severity boundaries. The collector matters less than the policy used to get logs there.

Validation, Troubleshooting, and Day One Best Practices

A syslog pipeline is only real once a known test message comes out the other end. Send one from a source, capture the packet in transit, and confirm the receiver indexed it in the expected stream. Without that loop, the forwarding setup is still just a config file.

A simple validation sequence

1. Generate a known message. Use logger with a distinctive tag so the receiver can search for it quickly.
2. Confirm packets on the wire. Use tcpdump or ncat on the sender or receiver side to verify the transport you configured.
3. Check collector visibility. Search the receiver index or ingestion view for the tag, then confirm the facility, severity, hostname, and timestamp look right.

That sequence catches the common mistake where the sender is alive but the receiver never sees the event, or sees it with the wrong format. It also helps distinguish transport failure from parsing failure, which is where a lot of debugging time gets wasted.

The habits that keep the pipeline stable

• Keep time aligned. Correlation breaks fast when hosts disagree on timestamps.
• Normalize near the agent. Format and filter before the receiver has to guess.
• Plan certificate rotation. TLS forwarding fails hard when certs expire.
• Design for backpressure. Queue locally so a collector outage delays delivery instead of erasing it.
• Document every stream. New services should know exactly which facility, severity, and destination apply.

Those habits are boring, and that's the point. A syslog pipeline should behave like infrastructure, not like a mystery box.

Fluxtail gives engineering teams a central place to receive syslog, route it into named streams, and inspect live logs without turning the pipeline into a black box. For teams setting up syslog forwarding across legacy and cloud systems, the useful move is to keep transport, routing, and verification explicit from the start. Visit Fluxtail to review the current receiver model and line it up with the forwarding rules already in use.