Seed a Database with Realistic Test Data

A development database seeded with three hand-typed rows and a staging environment running against an empty table hide the same class of bug: a pagination control never tested past page one, a query that returns instantly against a dozen rows and takes seconds against ten thousand, an edge case that only exists because a real distribution has a long tail. Copying a snapshot of production data fixes the volume and shape problem but trades it for a worse one — real customer names, payment details, and order histories sitting in an environment that was never built to hold them, reachable by anyone with access to dev.

Eventum closes that gap by generating the dataset instead of copying it. Build a generator that produces an e-commerce transaction dataset as a CSV file: purchases, refunds, and chargebacks in a realistic 80/15/5 mix, with correlated fields — customer, product, amount — instead of independently random values. The result is a file shaped like a real transactions export, ready to load into whatever database backs dev or staging today.

Seed dev database with realistic data

Three rows of hand-typed fixture data confirm that a transactions table accepts the right columns and nothing more. They don't confirm that a paginated transaction list still responds quickly on page 40, that a monthly revenue report holds up once chargebacks and refunds are mixed in at their real, rare frequency instead of one of each by hand, or that a report which only ever saw positive amounts survives its first negative refund row. All three failure modes need the volume and the shape a handful of manual rows can't provide — which is exactly what the generator built below produces: ten thousand transactions with a realistic distribution, not ten thousand copies of the same row.

Copying a production snapshot instead would supply that volume, but it drags real customer names, card details, and order histories into a database any developer on the team can query — a liability a synthetic dataset never carries, since none of it describes a real person or a real purchase. It also goes stale the moment it's taken; regenerating a synthetic dataset costs one command, not a new export.

Why CSV for database seeding

A CSV file is the most portable way to get that dataset into a database, because every mainstream engine already has a bulk-import path built around it: PostgreSQL's COPY, MySQL's LOAD DATA INFILE, and the CSV import wizard built into most GUI database clients all read the same file format. The dataset generated below drops into whatever engine currently backs dev — no custom loader script, no per-database export format to maintain.

A destination that accepts inserts directly instead of a file can skip the CSV step entirely. Eventum's clickhouse output inserts each transaction into a table over HTTP as it's generated, so an analytical store built for that kind of write never needs a file in between — the Going further section revisits this once the CSV build below is in place.

What you'll build

The generator uses:

• linspace input — 10,000 timestamps spread evenly across a date range.
• chance picking mode — weighted random selection between three transaction types.
• file output with the default plain formatter — each template already renders a complete CSV row, so nothing needs reshaping before it's written.
• JSON samples — product catalog loaded from a file.
• shared state — running revenue counter across all events.

The entire dataset is generated in one burst using sample mode (live_mode: false), so 10,000 rows complete in seconds.

Prerequisites

• Eventum installed

Project structure

Build it

mkdir -p ecommerce-csv/{templates,data}
cd ecommerce-csv

[
  { "name": "Wireless Mouse", "category": "Electronics", "min_price": 15, "max_price": 45 },
  { "name": "USB-C Hub", "category": "Electronics", "min_price": 25, "max_price": 80 },
  { "name": "Desk Lamp", "category": "Home", "min_price": 20, "max_price": 60 },
  { "name": "Notebook Set", "category": "Office", "min_price": 8, "max_price": 25 },
  { "name": "Bluetooth Speaker", "category": "Electronics", "min_price": 30, "max_price": 120 },
  { "name": "Coffee Mug", "category": "Home", "min_price": 10, "max_price": 30 },
  { "name": "Backpack", "category": "Accessories", "min_price": 35, "max_price": 90 },
  { "name": "Webcam", "category": "Electronics", "min_price": 40, "max_price": 150 },
  { "name": "Plant Pot", "category": "Home", "min_price": 12, "max_price": 35 },
  { "name": "Mechanical Keyboard", "category": "Electronics", "min_price": 60, "max_price": 200 }
]

{%- set product = module.rand.choice(samples.products) -%}
{%- set amount = module.rand.number.floating(product.min_price, product.max_price) -%}
{%- do shared.set("revenue", shared.get("revenue", 0) + amount) -%}
{{ module.rand.crypto.uuid4() }},{{ timestamp.strftime("%Y-%m-%d %H:%M:%S") }},purchase,C-{{ module.rand.number.integer(10000, 99999) }},{{ module.faker.locale.en.name() }},{{ product.name }},{{ product.category }},{{ "%.2f" | format(amount) }},USD

{%- set product = module.rand.choice(samples.products) -%}
{%- set amount = module.rand.number.floating(product.min_price, product.max_price) -%}
{%- do shared.set("revenue", shared.get("revenue", 0) - amount) -%}
{{ module.rand.crypto.uuid4() }},{{ timestamp.strftime("%Y-%m-%d %H:%M:%S") }},refund,C-{{ module.rand.number.integer(10000, 99999) }},{{ module.faker.locale.en.name() }},{{ product.name }},{{ product.category }},-{{ "%.2f" | format(amount) }},USD

{%- set product = module.rand.choice(samples.products) -%}
{%- set amount = module.rand.number.floating(product.min_price, product.max_price) -%}
{%- do shared.set("revenue", shared.get("revenue", 0) - amount) -%}
{{ module.rand.crypto.uuid4() }},{{ timestamp.strftime("%Y-%m-%d %H:%M:%S") }},chargeback,C-{{ module.rand.number.integer(10000, 99999) }},{{ module.faker.locale.en.name() }},{{ product.name }},{{ product.category }},-{{ "%.2f" | format(amount) }},USD

input:
  - linspace:
      start: "2025-01-01T00:00:00"
      end: "2025-12-31T23:59:59"
      count: 10000
      endpoint: true

event:
  template:
    mode: chance
    samples:
      products:
        type: json
        source: data/products.json
    templates:
      - purchase:
          template: templates/purchase.jinja
          chance: 0.80
      - refund:
          template: templates/refund.jinja
          chance: 0.15
      - chargeback:
          template: templates/chargeback.jinja
          chance: 0.05

output:
  - file:
      path: transactions.csv
      write_mode: overwrite

eventum generate --path generator.yml --id ecommerce --live-mode false

sed -i '1i transaction_id,timestamp,type,customer_id,customer_name,product,category,amount,currency' transactions.csv
head -6 transactions.csv

transaction_id,timestamp,type,customer_id,customer_name,product,category,amount,currency
5d842291-9c9e-4c88-a9b0-76d3ae3e0ccb,2024-12-31 21:00:00,purchase,C-86918,Rachel Hill,Desk Lamp,Home,53.31,USD
df9d7962-6a0d-467a-888b-7221ab42b51f,2024-12-31 22:45:07,purchase,C-94340,David Evans,Mechanical Keyboard,Electronics,168.20,USD
c120dd12-cfe3-4e95-b585-d273e52b626e,2024-12-31 21:52:33,purchase,C-34622,Jacob Smith,Webcam,Electronics,71.57,USD
8ef4f5d0-99a9-47d3-9920-e387d699987e,2024-12-31 23:37:41,refund,C-16314,Ricky Ewing DVM,Mechanical Keyboard,Electronics,-128.78,USD
4fc84fe8-be85-4920-8681-f9c389e67b68,2025-01-01 00:30:15,refund,C-72557,Andrea Barnes,Wireless Mouse,Electronics,-43.52,USD

8b11c3e4-2c40-4d20-8063-82fcb0a7bc6d,2025-01-01 17:09:00,chargeback,C-36012,Justin Russell,Plant Pot,Home,-17.73,USD

tail -n +2 transactions.csv | cut -d',' -f3 | sort | uniq -c | sort -rn

   7977 purchase
   1560 refund
    463 chargeback

Seed staging environment from the CSV

The header written in the previous step already names every column a transactions table needs, so the file drops straight into a bulk-import command instead of a custom loader script.

PostgreSQL reads it with \copy, the client-side form of COPY that works against a local file without needing server-side filesystem access:

\copy transactions FROM 'transactions.csv' WITH (FORMAT csv, HEADER true)

MySQL uses LOAD DATA:

LOAD DATA LOCAL INFILE 'transactions.csv'
INTO TABLE transactions
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES;

Most GUI database clients — pgAdmin, DBeaver, TablePlus, MySQL Workbench — wrap the same operation behind an "Import from CSV" wizard, for a one-off load that doesn't need a saved command.

None of that changes when the target shifts: the same command seeds a staging environment exactly the way it seeds dev, pointed at a different connection string. Regenerating a fresh CSV whenever a schema change or a new test scenario calls for different data costs one command — waiting on the next production snapshot doesn't.

Going further

• Add customer loyalty — use shared state to track repeat customers by keeping a customer pool and reusing IDs with some probability.
• Seasonal patterns — replace linspace with time-patterns using a triangular distribution to create Black Friday and holiday spikes.
• Multiple currencies — add params for currency and run multiple generators in parallel with different currency settings.
• Seed an analytical store directly — swap the file output for ClickHouse and every transaction inserts straight into a table over HTTP as it's generated, with no CSV or import step in between.

What's next

FAQ

Related

• The Realistic values lesson for the distributions and weighted choices behind the price ranges and the 80/15/5 transaction mix
• The ClickHouse delivery lesson for seeding an analytical database directly, with no CSV in between
• The Test data pipeline lesson for testing transport correctness instead of seeding a database at rest
• The Scenarios pillar for the course's other synthetic-data scenarios
• The Eventum Hub for datasets already shaped for common sources, ready to load without building one first