Eventum Logo

Eventum

Test Sigma Rules with Synthetic Attack Telemetry

Generate benign and attack-shaped telemetry to test Sigma rules and ATT&CK-mapped detections, without running a real attack or touching production.

Shipping a detection rule means answering two questions with evidence, not intuition: does it fire on the activity it was written to catch, and does it stay silent on everything else. A live attack answers both, but only after standing up an isolated lab and running the technique itself — expensive to repeat, and never something to point at a production identity provider or domain controller. A static log sample pulled from a public repository answers the question once, for one host, one account, one moment, and stops answering it the day naming conventions, account names, or timing change.

Eventum generates the telemetry a detection rule actually inspects — the event IDs, fields, and repetition a technique leaves behind — parameterized across as many source addresses, accounts, and volumes as a test needs, blended into the ordinary traffic the rule has to ignore.

This lesson generates telemetry shaped like a specific attack pattern — event IDs, field values, and repetition — to exercise a detection rule. It runs no exploit code, payload, or attacker tooling — every event below is an ordinary Windows Security log entry, mechanically identical to the benign traffic surrounding it.

Attack telemetry for detection testing

Attack telemetry, in this sense, is not an attack. It is the sequence of ordinary log events a technique produces as a side effect — repeated authentication failures from one address, a process spawned by an unexpected parent, a registry key written by a process that doesn't normally write it. Reproducing that sequence exercises a rule exactly the way the real technique would, without executing anything resembling the technique itself.

Two open, vendor-neutral standards make that reproduction checkable rather than improvised. Sigma is a generic, structured format for writing a detection rule once — a YAML document naming a log source, the field values that must match, and the condition that must hold — and converting it to any backend's native query language. MITRE ATT&CK is a taxonomy of adversary techniques; a Sigma rule tags the techniques it's meant to catch, so the telemetry that tests it can be described the same way — this stream contains a Brute Force (T1110) pattern, not just this stream contains some failed logons.

Testing a rule against telemetry like this means generating both halves of the stream at once: an attack-like pattern shaped to the rule's own logic, and enough ordinary noise around it to prove the rule doesn't fire on the noise too.

A chance-weighted mix of templates produces exactly that split — a low-probability pattern buried in a high-probability background — without a state machine tracking which stage of an attack is underway.

What you'll build

A single Eventum generator that produces a Windows Security authentication stream with a brute-force pattern deliberately planted in it, alongside the Sigma rule that catches it:

  • Three template files — a benign logon that dominates the stream, plus the two halves of the attack: a repeated failed logon and the occasional success that follows it.
  • chance picking mode weights the mix so attack-shaped events stay a small minority, concentrated onto a fixed pool of source addresses held in shared state.
  • A bounded static input produces a fixed, rerunnable batch — the shape a repeatable detection test wants.
  • A file output writes the stream as newline-delimited JSON, ready to grep or forward to a SIEM.
  • A Sigma correlation rule that fires on the planted pattern and stays quiet on the benign background.

Prerequisites

Project structure

generator.yml
benign-logon.jinja
brute-force-attempt.jinja
brute-force-success.jinja
security.jsonl

Build it

Create the project directory

mkdir -p detection-lab/templates
cd detection-lab

Write the event templates

The example below reproduces a classic Brute Force (T1110) pattern against Windows authentication: many failed logons from one source address, occasionally followed by a success once a guessed credential lands. All three templates share the JSON shape from the Windows Event Log lesson — event.code, winlog.event_id, and the native event_data fields a real Security-channel event carries.

templates/benign-logon.jinja covers the overwhelming majority of traffic: an ordinary successful logon, with both the source address and the account drawn fresh at random every time.

detection-lab/templates/benign-logon.jinja
{%- set username = module.rand.choice(["jsmith", "ajohnson", "mwilliams", "kbrown", "tpatel"]) -%}
{%- set src_ip = module.rand.network.ip_v4_public() -%}
{%- set record_id = shared.get('record_id', 1) -%}
{
  "@timestamp": "{{ timestamp.isoformat() }}",
  "event": { "code": "4624", "action": "logged-in", "outcome": "success" },
  "user": { "name": "{{ username }}", "domain": "{{ params.domain }}" },
  "source": { "ip": "{{ src_ip }}" },
  "winlog": {
    "channel": "Security",
    "provider_name": "Microsoft-Windows-Security-Auditing",
    "event_id": "4624",
    "record_id": "{{ record_id }}",
    "logon": { "type": "Network" },
    "event_data": {
      "TargetUserName": "{{ username }}",
      "TargetDomainName": "{{ params.domain }}",
      "LogonType": "3",
      "IpAddress": "{{ src_ip }}"
    }
  }
}
{%- do shared.set('record_id', record_id + 1) -%}

templates/brute-force-attempt.jinja covers the failed side of the pattern. Instead of a fresh address per event, it draws from a small pool of three addresses generated once and kept in shared state — the same three addresses recur across many events, which is exactly what makes them stand out against the benign traffic's high-cardinality noise. The targeted account also comes from a small, fixed set, since a real credential-guessing campaign aims at known or predictable account names rather than the general user directory:

detection-lab/templates/brute-force-attempt.jinja
{%- if not shared.get('attacker_ips') -%}
{%- do shared.set('attacker_ips', [module.rand.network.ip_v4_public(), module.rand.network.ip_v4_public(), module.rand.network.ip_v4_public()]) -%}
{%- endif -%}
{%- set src_ip = module.rand.choice(shared.get('attacker_ips')) -%}
{%- set username = module.rand.choice(["administrator", "admin", "svc-sql", "backup-admin"]) -%}
{%- set status = module.rand.weighted_choice(["0xC000006A", "0xC0000064"], [90, 10]) -%}
{%- set record_id = shared.get('record_id', 1) -%}
{
  "@timestamp": "{{ timestamp.isoformat() }}",
  "event": { "code": "4625", "action": "logon-failed", "outcome": "failure" },
  "user": { "name": "{{ username }}", "domain": "{{ params.domain }}" },
  "source": { "ip": "{{ src_ip }}" },
  "winlog": {
    "channel": "Security",
    "provider_name": "Microsoft-Windows-Security-Auditing",
    "event_id": "4625",
    "record_id": "{{ record_id }}",
    "event_data": {
      "TargetUserName": "{{ username }}",
      "TargetDomainName": "{{ params.domain }}",
      "Status": "{{ status }}",
      "IpAddress": "{{ src_ip }}"
    }
  }
}
{%- do shared.set('record_id', record_id + 1) -%}

templates/brute-force-success.jinja covers the rare case where a guess lands: the same address pool, the same account pool, a successful outcome instead of a failure. It reads the same attacker_ips key from shared state that the attempt template above writes, so both draw from the same three addresses regardless of which template happens to render first:

detection-lab/templates/brute-force-success.jinja
{%- if not shared.get('attacker_ips') -%}
{%- do shared.set('attacker_ips', [module.rand.network.ip_v4_public(), module.rand.network.ip_v4_public(), module.rand.network.ip_v4_public()]) -%}
{%- endif -%}
{%- set src_ip = module.rand.choice(shared.get('attacker_ips')) -%}
{%- set username = module.rand.choice(["administrator", "admin", "svc-sql", "backup-admin"]) -%}
{%- set record_id = shared.get('record_id', 1) -%}
{
  "@timestamp": "{{ timestamp.isoformat() }}",
  "event": { "code": "4624", "action": "logged-in", "outcome": "success" },
  "user": { "name": "{{ username }}", "domain": "{{ params.domain }}" },
  "source": { "ip": "{{ src_ip }}" },
  "winlog": {
    "channel": "Security",
    "provider_name": "Microsoft-Windows-Security-Auditing",
    "event_id": "4624",
    "record_id": "{{ record_id }}",
    "logon": { "type": "Network" },
    "event_data": {
      "TargetUserName": "{{ username }}",
      "TargetDomainName": "{{ params.domain }}",
      "LogonType": "3",
      "IpAddress": "{{ src_ip }}"
    }
  }
}
{%- do shared.set('record_id', record_id + 1) -%}

Configure the generator

mode: chance mixes the three templates by weight: successful logons dominate at 87, failed attempts sit at 10, and the rare successful guess at 3 — roughly one attack-shaped event in ten, and about one in thirty of those a completed compromise. A bounded static input produces a fixed, rerunnable batch instead of an open-ended stream, and a file output writes one JSON object per line:

detection-lab/generator.yml
input:
  - static:
      count: 3000

event:
  template:
    mode: chance
    params:
      domain: CONTOSO
    templates:
      - benign_logon:
          template: templates/benign-logon.jinja
          chance: 87
      - brute_force_attempt:
          template: templates/brute-force-attempt.jinja
          chance: 10
      - brute_force_success:
          template: templates/brute-force-success.jinja
          chance: 3

output:
  - file:
      path: output/security.jsonl
      formatter:
        format: json

Run it

eventum generate in sample mode (--live-mode false) runs the whole batch immediately instead of pacing it against the clock:

eventum generate \
  --path detection-lab/generator.yml \
  --id detection-lab \
  --live-mode false

The batch writes in under a second. static assigns every event the same current timestamp — fine for this exercise, since the rule below groups events by source address rather than by time; the realistic timing lesson covers spreading a stream like this across real time instead. A slice from the start of an actual run — ordinary, unrelated logons:

{"@timestamp": "2026-07-12T15:23:15.761004+00:00", "event": {"code": "4624", "action": "logged-in", "outcome": "success"}, "user": {"name": "tpatel", "domain": "CONTOSO"}, "source": {"ip": "200.12.121.73"}, "winlog": {"channel": "Security", "provider_name": "Microsoft-Windows-Security-Auditing", "event_id": "4624", "record_id": "1", "logon": {"type": "Network"}, "event_data": {"TargetUserName": "tpatel", "TargetDomainName": "CONTOSO", "LogonType": "3", "IpAddress": "200.12.121.73"}}}
{"@timestamp": "2026-07-12T15:23:15.761004+00:00", "event": {"code": "4624", "action": "logged-in", "outcome": "success"}, "user": {"name": "jsmith", "domain": "CONTOSO"}, "source": {"ip": "192.147.22.147"}, "winlog": {"channel": "Security", "provider_name": "Microsoft-Windows-Security-Auditing", "event_id": "4624", "record_id": "2", "logon": {"type": "Network"}, "event_data": {"TargetUserName": "jsmith", "TargetDomainName": "CONTOSO", "LogonType": "3", "IpAddress": "192.147.22.147"}}}
{"@timestamp": "2026-07-12T15:23:15.761004+00:00", "event": {"code": "4624", "action": "logged-in", "outcome": "success"}, "user": {"name": "ajohnson", "domain": "CONTOSO"}, "source": {"ip": "195.120.95.182"}, "winlog": {"channel": "Security", "provider_name": "Microsoft-Windows-Security-Auditing", "event_id": "4624", "record_id": "3", "logon": {"type": "Network"}, "event_data": {"TargetUserName": "ajohnson", "TargetDomainName": "CONTOSO", "LogonType": "3", "IpAddress": "195.120.95.182"}}}

Across the full 3,000-event batch, 302 events came out as EventID 4625 — close to the configured 1-in-10 weight — and 2,698 as 4624. Three source addresses account for nearly all of the 4625s, with 97 to 107 each. Every other address, drawn fresh per benign event, turns up far less often — usually once, occasionally a few times where the public-IP pool's small reserved range happens to collide. None comes anywhere near those three.

Test detection rules against the stream

Sigma expresses "many events from one source within a time window" as a correlation — a separate document that references a base rule and adds the grouping, the time span, and the threshold, rather than folding an aggregation into the base rule's own condition. The rule below, built from the Sigma correlation rules specification, matches the pattern the generator above produces: five or more EventID 4625 events from the same address within 10 minutes, followed by an EventID 4624 from that same address:

detection-lab/mr_brute_force_logon.yml
title: Multiple failed logons followed by a successful logon
id: c5e49499-ab06-43a7-b423-2783681d1945
status: experimental
description: Detects repeated failed logons from one source followed by a successful logon from the same source
correlation:
    type: temporal_ordered
    rules:
        - failed_logon_burst
        - successful_logon
    group-by:
        - IpAddress
    timespan: 10m
falsepositives:
    - Bulk password resets or account migrations that briefly fail before succeeding
    - A misconfigured service retrying stale credentials
level: high
tags:
    - attack.credential_access
    - attack.t1110
---
title: Five or more failed logons from one source
id: b86a5346-d45f-4bbf-9c3d-b47da5137e34
name: failed_logon_burst
description: Detects five or more failed logon attempts from the same source address within 10 minutes
correlation:
    type: event_count
    rules:
        - failed_logon
    group-by:
        - IpAddress
    timespan: 10m
    condition:
        gte: 5
---
title: Failed logon
id: 9668233f-0dd6-4827-8a35-24079b398ae0
name: failed_logon
logsource:
    product: windows
    service: security
detection:
    selection:
        EventID: 4625
    condition: selection
---
title: Successful logon
id: 9b5fc948-abcf-49f2-b5ee-7c0d0149bdb3
name: successful_logon
logsource:
    product: windows
    service: security
detection:
    selection:
        EventID: 4624
    condition: selection

The two innermost documents are ordinary Sigma rules — one matches any EventID 4625, the other any EventID 4624 — and say nothing about volume or timing on their own. failed_logon_burst correlates the first one: it counts EventID 4625 events grouped by IpAddress and fires once a single address crosses 5 within a 10-minute span. The outermost rule chains that burst to a successful logon from the same address within the same window — the two-stage pattern the templates above generate, expressed as detection logic instead of Jinja.

Filtering the generated stream for one of the three attacker addresses shows exactly the shape the rule is looking for — five failures against different targeted accounts, then a success:

grep '"IpAddress": "178.161.163.143"' output/security.jsonl
{"@timestamp": "2026-07-12T15:23:15.761004+00:00", "event": {"code": "4625", "action": "logon-failed", "outcome": "failure"}, "user": {"name": "administrator", "domain": "CONTOSO"}, "source": {"ip": "178.161.163.143"}, "winlog": {"channel": "Security", "provider_name": "Microsoft-Windows-Security-Auditing", "event_id": "4625", "record_id": "202", "event_data": {"TargetUserName": "administrator", "TargetDomainName": "CONTOSO", "Status": "0xC000006A", "IpAddress": "178.161.163.143"}}}
{"@timestamp": "2026-07-12T15:23:15.761004+00:00", "event": {"code": "4625", "action": "logon-failed", "outcome": "failure"}, "user": {"name": "backup-admin", "domain": "CONTOSO"}, "source": {"ip": "178.161.163.143"}, "winlog": {"channel": "Security", "provider_name": "Microsoft-Windows-Security-Auditing", "event_id": "4625", "record_id": "234", "event_data": {"TargetUserName": "backup-admin", "TargetDomainName": "CONTOSO", "Status": "0xC000006A", "IpAddress": "178.161.163.143"}}}
{"@timestamp": "2026-07-12T15:23:15.761004+00:00", "event": {"code": "4625", "action": "logon-failed", "outcome": "failure"}, "user": {"name": "administrator", "domain": "CONTOSO"}, "source": {"ip": "178.161.163.143"}, "winlog": {"channel": "Security", "provider_name": "Microsoft-Windows-Security-Auditing", "event_id": "4625", "record_id": "236", "event_data": {"TargetUserName": "administrator", "TargetDomainName": "CONTOSO", "Status": "0xC000006A", "IpAddress": "178.161.163.143"}}}
{"@timestamp": "2026-07-12T15:23:15.761004+00:00", "event": {"code": "4625", "action": "logon-failed", "outcome": "failure"}, "user": {"name": "svc-sql", "domain": "CONTOSO"}, "source": {"ip": "178.161.163.143"}, "winlog": {"channel": "Security", "provider_name": "Microsoft-Windows-Security-Auditing", "event_id": "4625", "record_id": "269", "event_data": {"TargetUserName": "svc-sql", "TargetDomainName": "CONTOSO", "Status": "0xC000006A", "IpAddress": "178.161.163.143"}}}
{"@timestamp": "2026-07-12T15:23:15.761004+00:00", "event": {"code": "4625", "action": "logon-failed", "outcome": "failure"}, "user": {"name": "admin", "domain": "CONTOSO"}, "source": {"ip": "178.161.163.143"}, "winlog": {"channel": "Security", "provider_name": "Microsoft-Windows-Security-Auditing", "event_id": "4625", "record_id": "281", "event_data": {"TargetUserName": "admin", "TargetDomainName": "CONTOSO", "Status": "0xC000006A", "IpAddress": "178.161.163.143"}}}
{"@timestamp": "2026-07-12T15:23:15.761004+00:00", "event": {"code": "4624", "action": "logged-in", "outcome": "success"}, "user": {"name": "svc-sql", "domain": "CONTOSO"}, "source": {"ip": "178.161.163.143"}, "winlog": {"channel": "Security", "provider_name": "Microsoft-Windows-Security-Auditing", "event_id": "4624", "record_id": "284", "logon": {"type": "Network"}, "event_data": {"TargetUserName": "svc-sql", "TargetDomainName": "CONTOSO", "LogonType": "3", "IpAddress": "178.161.163.143"}}}

Running the same filter against any of the other benign source addresses in this batch turns up only a few matches — under the rule's threshold of 5, and nowhere near the attacker pool's hundred-plus, which is exactly why that threshold separates the two without also catching ordinary coincidental reuse.

Sigma rule testing across formats

The same brute-force shape generalizes past Windows Security. The OCSF format lesson generates Authentication-class events (class_uid 3002) for the same kind of logon: activity_id: 1 ("Logon") on every attempt, with status_id/status distinguishing a failure from a success — the identical pair of facts a Sigma rule targeting an OCSF-normalized backend would group and count, just named status_id and src_endpoint.ip instead of EventID and IpAddress. Both examples even land on the same logon_type_id/LogonType value (3, "Network") for the same underlying reason: this is a remote authentication attempt either way, whatever the schema calls the fields that describe it.

The technique-to-telemetry mapping this lesson builds on carries over to any other Sigma rule with a logsource and a condition — a Sysmon process-ancestry rule, a DNS-tunneling detection, an unusual PowerShell command line. Decide what the rule's detection block is matching, then shape a template's fields — and, where volume matters, a shared-backed pool like the one above — to produce exactly that.

Going further

  • Widen the benign side — more usernames, mixed logon types, a second protocol — so the contrast with the attacker pool stays realistic at higher volumes.
  • Replace the static input with time_patterns to spread the burst across real minutes instead of one instant, closer to how a correlation rule's timespan behaves against production traffic.
  • Point a third template at a different Sigma-mapped pattern — process-ancestry detections over Sysmon telemetry are a natural next target, and the windows-sysmon Hub generator is a ready-made source of that event shape.

What's next

chance picking mode

Full parameter reference for weighted template selection.

Template state

How locals, shared, and globals persist values across renders.

eventum generate reference

Every flag for running a single generator from the command line.

FAQ

SIEM Test Data: Index Windows Security Events in OpenSearch

Generate SIEM test data — Windows Security Event Log sessions indexed into OpenSearch — for detection development, parser validation, and analyst training.

Seed a Database with Realistic Test Data

Seed a database with realistic test data — generate a shaped e-commerce dataset of purchases, refunds, and chargebacks as CSV, ready to load into a dev or staging database instead of copying production.

On this page

Attack telemetry for detection testingWhat you'll buildPrerequisitesProject structureBuild itCreate the project directoryWrite the event templatesConfigure the generatorRun itTest detection rules against the streamSigma rule testing across formatsGoing furtherWhat's nextFAQRelated