sensor-logs: joined multiline syslog record fails to re-parse (dropped) #584

New issue

Closed

opened 2026-07-23 18:29:11 +00:00 by p13marc · 0 comments

p13marc commented

2026-07-23 18:29:11 +00:00

(Migrated from github.com)

Found by the #548 integration harness (epic #542).

Problem

MultilineJoiner folds a head line + indented continuations into one raw string with embedded \ns, then handle_stream_connection passes that joined string to parser::parse (receiver.rs process_record). But the RFC 3164 and simple-PRI regexes end with (.*)$ — . doesn't match newline and $ (non-multiline) won't match with content after the first line's newline — so a joined record like:

<11>Oct 11 22:14:15 host app: Traceback (most recent call last):
    File "x.py", line 1, in <module>

fails all three patterns and is dropped (counted as parse_failed). So multiline stack traces arriving over a syslog stream (TCP/Unix, <PRI>-prefixed) are lost — exactly the case MultilineJoiner exists to handle.

Note: the joiner's folding is unit-tested (multiline.rs), and the file source (#549, plain-line format) won't hit this. The gap is specifically PRI-framed multiline over the stream listeners.

Fix options

Parse only the first line for the syslog envelope (PRI/timestamp/host/tag), then append the remaining folded lines to the message field verbatim. This is the RFC-agnostic correct behavior and keeps the envelope from the head line.
Or run the regexes against the first line only and treat the rest as message continuation.

Acceptance

A <PRI>-framed head + indented continuation over a TCP/Unix listener arrives as one LogRecord whose message contains all folded lines and whose severity/host/app come from the head.
Add the end-to-end assertion to the #548 harness (multiline_idle_flush_* currently only asserts a single buffered line, with a comment pointing here).

Found by the #548 integration harness (epic #542). ## Problem `MultilineJoiner` folds a head line + indented continuations into one raw string with embedded `\n`s, then `handle_stream_connection` passes that joined string to `parser::parse` (`receiver.rs` `process_record`). But the RFC 3164 and simple-PRI regexes end with `(.*)$` — `.` doesn't match newline and `$` (non-multiline) won't match with content after the first line's newline — so a joined record like: ``` <11>Oct 11 22:14:15 host app: Traceback (most recent call last): File "x.py", line 1, in <module> ``` fails all three patterns and is **dropped** (counted as parse_failed). So multiline stack traces arriving over a **syslog stream** (TCP/Unix, `<PRI>`-prefixed) are lost — exactly the case `MultilineJoiner` exists to handle. Note: the joiner's folding is unit-tested (`multiline.rs`), and the file source (#549, plain-line format) won't hit this. The gap is specifically PRI-framed multiline over the stream listeners. ## Fix options - Parse only the **first line** for the syslog envelope (PRI/timestamp/host/tag), then append the remaining folded lines to the `message` field verbatim. This is the RFC-agnostic correct behavior and keeps the envelope from the head line. - Or run the regexes against the first line only and treat the rest as message continuation. ## Acceptance - A `<PRI>`-framed head + indented continuation over a TCP/Unix listener arrives as **one** `LogRecord` whose message contains all folded lines and whose severity/host/app come from the head. - Add the end-to-end assertion to the #548 harness (`multiline_idle_flush_*` currently only asserts a single buffered line, with a comment pointing here).

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

marcpardo/zensight#584

No description provided.

Rows
Columns