Replay logs with fresh timestamps

Some test scenarios do not call for synthetic data at all — they call for a file that already exists, played back. A captured production sample from a system that's since been decommissioned, an incident capture pulled straight from a security ticket, a golden dataset a test suite has quietly depended on for months: reaching for a template that generates a plausible approximation throws away the one thing that made any of these worth keeping in the first place — their exact content, in their exact order. Generating from scratch reproduces the shape a source would produce; it cannot reproduce the one sequence of events that actually happened.

Eventum's replay event plugin replays historical logs without shipping their original, stale timestamps along with them. Every line in a captured file still carries whatever timestamp it had the moment the file was captured, and replay reads that file back line by line and, wherever a line carries a timestamp, rewrites it to the moment the event is actually replayed — the same real content, arriving with a fresh timestamp.

When to replay instead of generate

Replay and the template plugin solve different problems, and the choice comes down to what a given test actually needs held constant. Replay preserves exact content and exact ordering — the file handed to it is the file that comes back out, line for line, with nothing invented and nothing left out. Generation synthesizes fresh data on every run instead: parameterized, varied from one event to the next, and able to keep producing for as long as the generator keeps running.

Reach for replay when the value sits in one specific file: a captured incident that needs to run past a detection rule exactly as it happened, an exact reproduction of a known-bad or known-good case, a fixed golden dataset a test suite already depends on. Reach for generation instead when a test needs variety across many runs, parameters someone can adjust, or a stream that keeps going rather than stopping once a fixed file runs out.

How log replay works

The replay event plugin reads a log file from path line by line and emits each line as one event, in file order — one line consumed per timestamp the input plugin schedules. Content by itself is not enough for most replay scenarios, though: a line's own timestamp is fixed at whatever moment the file was originally captured, and replaying it verbatim would ship stale timestamps into a system expecting current ones.

Two fields fix that. timestamp_pattern is a regular expression with a named timestamp group that locates the original timestamp inside each line; timestamp_format is a strftime format string describing how the replacement should be rendered — left unset, replay falls back to ISO 8601. Together, they let the plugin swap out just that one substring for the event's actual current timestamp, leaving the rest of the line untouched. A line that doesn't match the pattern at all is emitted exactly as read, logged as a warning rather than treated as an error — which matters for a captured file that mixes timestamped entries with the odd header, separator, or footer line.

repeat decides what happens once the file runs out: left at its default of false, the plugin stops producing once the last line has been replayed; set to true, it loops back to the start and keeps going instead, useful for pairing a small captured sample with a schedule that keeps ticking. Two more fields cover less common cases: chunk_size controls how much of the file is buffered in memory at a time — the default reads it in 1 MiB chunks rather than loading it whole, which keeps a large file safe to replay — and encoding overrides the default UTF-8 for a file captured with a different codec.

Replay a log with fresh timestamps in Eventum

2026-03-14T02:11:03 sshd: Failed password for invalid user admin from 198.51.100.23 port 51422 ssh2
2026-03-14T02:11:05 sshd: Failed password for invalid user admin from 198.51.100.23 port 51423 ssh2
2026-03-14T02:11:08 sshd: Failed password for invalid user admin from 198.51.100.23 port 51424 ssh2
2026-03-14T02:11:12 sshd: Accepted password for root from 198.51.100.23 port 51425 ssh2
[end of capture]

input:
  - cron:
      expression: "* * * * * *"
      count: 1

event:
  replay:
    path: samples/source.log
    timestamp_pattern: '(?P<timestamp>\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2})'
    timestamp_format: '%Y-%m-%dT%H:%M:%S'

output:
  - file:
      path: output/events.log
      write_mode: overwrite
      formatter:
        format: plain

The result

Running this generator replays all five lines and then stops on its own — nothing forces it to continue once the last line has been replayed:

2026-07-17T14:53:19 sshd: Failed password for invalid user admin from 198.51.100.23 port 51422 ssh2
2026-07-17T14:53:20 sshd: Failed password for invalid user admin from 198.51.100.23 port 51423 ssh2
2026-07-17T14:53:21 sshd: Failed password for invalid user admin from 198.51.100.23 port 51424 ssh2
2026-07-17T14:53:22 sshd: Accepted password for root from 198.51.100.23 port 51425 ssh2
[end of capture]

Every one of the first four lines kept its exact original content — the same source IP, the same port, the same username, the same outcome — with only the leading timestamp moved from March 2026 to the moment this run actually happened, one second apart, matching the per-second cron schedule. The fifth line has nothing for timestamp_pattern to match, so it came back exactly as written in source.log, byte for byte — the only sign it was even considered is a warning logged at the time (Failed to substitute timestamp into original message, reason No match found), not an error, and not a dropped event.

FAQ

Related

• The Realism pillar for the other techniques that make synthetic data behave like production
• The Log and event formats field guide for the record shapes a replayed file is likely to carry
• The Streaming vs bulk lesson for how live mode paces a replay and sample mode replays a whole file at once
• The Sigma detection testing tutorial for replaying a captured incident straight into a detection rule instead of building attack telemetry from scratch
• The replay reference for every field, including chunk_size and encoding
• The template reference for the alternative event plugin, when a test needs generated variety instead of one exact file
• Ready-made generators in the Eventum Hub