generator.yml

The generator config file defines the three-stage pipeline that produces events. It has exactly three top-level keys — input, event, and output — each configuring one stage:

input:
  - <plugin_name>:
      <plugin_params>

event:
  <plugin_name>:
    <plugin_params>

output:
  - <plugin_name>:
      <plugin_params>

All relative paths inside the file are resolved from the directory containing generator.yml, not from the working directory. This makes generator directories fully portable. See Project structure — Path resolution for details.

Top-level structure

Each plugin config is a single-key mapping where the key is the plugin name and the value is a mapping of plugin-specific parameters:

- cron:          # plugin name
    expression: "* * * * * *"  # plugin parameter
    count: 1                   # plugin parameter

Variable substitution

Generator config file supports variable substitution using ${ } syntax. This lets you parameterize configs without hardcoding values:

output:
  - opensearch:
      hosts:
        - ${params.opensearch_host}
      username: ${params.opensearch_user}
      password: ${secrets.opensearch_password}

Two types of variables are available:

See Parameters and Secrets for details on how to pass and manage these values.

Input plugins

Input plugins define when events are generated. Multiple input plugins can be combined — their timestamp streams are merged chronologically. Every input plugin supports an optional tags field.

See the input plugin reference for all available plugins and their parameters.

Event plugins

Event plugins define what events look like. Exactly one event plugin is configured per generator.

See the event plugin reference for all available plugins and their parameters.

Output plugins

Output plugins define where events are sent. Multiple output plugins can be configured — every event is delivered to all of them (fan-out). Every output plugin supports a formatter.

See the output plugin reference for all available plugins and their parameters.

Versatile datetime

Several input plugins accept datetime values in a flexible format called versatile datetime. The following formats are accepted:

See Scheduling — Date ranges and versatile datetime for more details and examples.

Complete example

input:
  - cron:
      expression: "* * * * * *"
      count: 1
      tags: [web]

  - cron:
      expression: "*/10 * * * * *"
      count: 1
      tags: [error]

event:
  template:
    mode: chance
    samples:
      users:
        type: csv
        source: samples/users.csv
        header: true
    templates:
      - access:
          template: templates/access.jinja
          chance: 95
      - error:
          template: templates/error.jinja
          chance: 5

output:
  - stdout:
      formatter:
        format: plain

  - file:
      path: output/events.jsonl
      formatter:
        format: json

  - opensearch:
      hosts:
        - ${params.opensearch_host}
      username: ${params.opensearch_user}
      password: ${secrets.opensearch_password}
      index: logs