Structured logging: from plain text to JSON

A service that logs a single sentence per event is simple to write and easy to read on a terminal, until someone has to answer a real question with that log: how many payment failures affected a specific customer in the last hour, or which requests exceeded a set duration. Answering that from a sentence means writing a pattern that matches its exact wording — and that pattern breaks the moment the sentence is reworded, a field moves, or a new one appears. The only contract between writer and reader is prose, and nothing enforces it.

Structured logging replaces that contract with one a program can rely on: each piece of data is emitted as its own named field, in a consistent shape, instead of folded into a sentence. Eventum generates events this way directly from a template — every field rendered on its own — and its JSON formatter validates the result before delivery, so a parser, a search query, or a detection rule can be exercised against realistic structured data without a live service producing it.

Structured logging vs plain text

Plain-text logging composes a human-readable sentence and interpolates the event's data directly into it. The line below and the structured event after it carry the same information — the same order, the same customer, the same failure reason:

2026-07-11 17:44:01 ERROR order 39402 failed for user bob: payment declined

{"timestamp": "2026-07-11T17:44:01+00:00", "level": "ERROR", "service": "orders", "message": "order failed", "order_id": 39402, "user": "bob", "reason": "payment declined"}

Reading the order ID back out of the first line means matching digits that happen to follow the word "order" — a pattern tied to that exact sentence. Reading it out of the second means reading the order_id field directly, regardless of where other data sits in the line or how the message text happens to be worded. That difference is the entire point of structured logging: a field sits at the same name every time, so software queries it directly instead of reverse-engineering it from a sentence written for a person to read.

Structured logging formats: key-value and JSON

A key-value line keeps the flat, single-line feel of plain text but separates each piece of data into its own key=value pair. JSON goes further and nests and types the data as a real object instead of flat text, so a number stays a number and a list stays a list rather than everything being a string that happens to look like one:

Fixing the shape of a log line does not fix what each field is called: two structured loggers can both emit JSON and still name the same piece of data differently — user, username, actor.name — so a query written against one does not match the other. A schema convention closes that gap by standardizing field names across every source that adopts it, so one query or detection rule matches all of them. ECS and OCSF are the two most widely adopted schemas built this way: structured logging is the mechanism, a schema is the shared vocabulary layered on top of it.

Rules that keep a structured log queryable

A structured log is only as useful as the discipline behind it:

• Pick one format per service, and default to JSON — most log shippers, search backends, and SIEMs already parse it, so nothing extra needs configuring downstream.
• Keep a field's name and type stable — a field that is a string in one event and a number in the next breaks any query or detection rule written against it.
• Keep variable data out of the message text — a message field should read the same regardless of which order or customer triggered the event; the order ID, the customer, and the failure reason belong in their own fields.
• Attach the fields that make one event usable on its own — a timestamp, a severity level, and whatever identifies the source, such as a service name or host.
• Adopt a field-naming schema, such as ECS or OCSF, instead of inventing names per service — the payoff is a query or detection rule that already works across every source that follows the same schema.

Generate structured logs with Eventum

The generator below simulates an order-processing service logging both outcomes as structured JSON: a common success case and a rarer failure, rendered from two separate templates, each carrying only the fields its event needs.

The templates

Fields are filled with module.rand, the template event plugin's built-in randomization module. templates/order-processed.jinja renders the common case, level INFO:

{%- set order_id = module.rand.number.integer(10000, 99999) -%}
{%- set user = module.rand.choice(["alice", "bob", "carol", "dave"]) -%}
{%- set duration_ms = module.rand.number.integer(80, 400) -%}
{"timestamp": "{{ timestamp.isoformat() }}", "level": "INFO", "service": "orders", "message": "order processed", "order_id": {{ order_id }}, "user": "{{ user }}", "duration_ms": {{ duration_ms }}}

templates/order-failed.jinja renders the rarer failure, level ERROR, with a reason field in place of duration_ms — a failure and a success do not describe themselves with the same data, so each template only carries the fields it needs:

{%- set order_id = module.rand.number.integer(10000, 99999) -%}
{%- set user = module.rand.choice(["alice", "bob", "carol", "dave"]) -%}
{%- set reason = module.rand.choice(["payment declined", "inventory unavailable", "address validation failed"]) -%}
{"timestamp": "{{ timestamp.isoformat() }}", "level": "ERROR", "service": "orders", "message": "order failed", "order_id": {{ order_id }}, "user": "{{ user }}", "reason": "{{ reason }}"}

The generator config

mode: chance picks one of the two templates per timestamp, weighted so successes dominate and failures stay rare. A cron input ticks once a second, and a file output writes each rendered event to a local file through the json formatter, which validates every line as JSON before it is written:

input:
  - cron:
      expression: "* * * * * *"
      count: 1

event:
  template:
    mode: chance
    templates:
      - order_processed:
          template: templates/order-processed.jinja
          chance: 90
      - order_failed:
          template: templates/order-failed.jinja
          chance: 10

output:
  - file:
      path: output/events.jsonl
      formatter:
        format: json

The result

Running the generator above produces one structured event per second, mostly order_processed and, at the configured 10% chance, an occasional order_failed — three consecutive lines from an actual run:

{"timestamp": "2026-07-11T17:44:00+00:00", "level": "INFO", "service": "orders", "message": "order processed", "order_id": 74824, "user": "carol", "duration_ms": 205}
{"timestamp": "2026-07-11T17:44:01+00:00", "level": "ERROR", "service": "orders", "message": "order failed", "order_id": 39402, "user": "bob", "reason": "payment declined"}
{"timestamp": "2026-07-11T17:44:02+00:00", "level": "INFO", "service": "orders", "message": "order processed", "order_id": 91759, "user": "dave", "duration_ms": 148}

Every line is a complete, independently valid JSON object — the json formatter checked each one before writing it, so a malformed line would have been dropped rather than corrupt the file. Filtering this file for "level":"ERROR" or "user":"bob" is a direct field match, the same query a search backend or a detection rule runs once the data is ingested, with no pattern tuned to a particular sentence's wording required.

FAQ

Related

• The Foundations pillar for the broader path from synthetic event data to a working pipeline
• The ECS lesson for a schema built on top of structured logging that standardizes field names across sources
• The log and event formats field guide for the formats structured events are wrapped in on the wire
• The template event plugin and formatters reference for every field and format used above
• Ready-made generators that already emit structured JSON events in the Eventum Hub