📘 Event Sourcing: When Kafka Becomes Your Database

When Kafka Becomes Your Database

Event Sourcing is not a storage optimization.
It is a truth model.

If you adopt it casually, it will break your system.
If you adopt it deliberately, it will change how you think about data forever.

1️⃣ The Fundamental Shift (This Is the Whole Game)

Traditional CRUD Model

“Store the current state.”

Order {
  status = SHIPPED
  total = 100
}

Event Sourcing Model

“Store everything that happened.”

OrderCreated
ItemAdded
PaymentCaptured
OrderShipped

State is derived. Events are the source of truth.

2️⃣ Why Kafka Fits Event Sourcing Naturally

Kafka provides:

• Append-only log

• Ordered per partition

• Durable history

• Replayability

That maps perfectly to event sourcing requirements.

Kafka doesn’t store rows.
It stores facts.

3️⃣ Core Principle (Non-Negotiable)

Events are immutable.

You do NOT:

• Update events

• Delete events

• Fix events

You only:

• Append new events

If this scares you — good. It should.

4️⃣ The Event Store (Kafka as DB)

Topic as Table

orders-events

Partitioning Rule

partition key = orderId

This guarantees:

• Per-order ordering

• Parallelism across orders

5️⃣ Commands vs Events (Critical Distinction)

Command

“Please do this.”

CreateOrder { orderId: 42 }

Event

“This happened.”

OrderCreated { orderId: 42 }

Commands can fail.
Events never lie.

6️⃣ Building State by Replaying Events

Example: Order Aggregate

function replay(events) {
  let state = { status: "NEW", items: [] };

  for (const e of events) {
    switch (e.type) {
      case "OrderCreated":
        state.id = e.orderId;
        break;
      case "ItemAdded":
        state.items.push(e.item);
        break;
      case "OrderShipped":
        state.status = "SHIPPED";
        break;
    }
  }
  return state;
}

The database is now a function of history.

7️⃣ Why This Is Powerful (Correctness Wins)

You gain:

• Full audit log

• Perfect traceability

• Temporal queries (“what did we know then?”)

• Replay after bugs

• Deterministic recovery

Debugging becomes re-running history, not guessing.

8️⃣ Event Sourcing vs Traditional DB (Truth Table)

Dimension	CRUD DB	Event Sourcing
History	❌ Lost	✅ Preserved
Audit	Hard	Free
Replay	❌	✅
Fix bugs	Manual	Reprocess
Simplicity	✅	❌

9️⃣ The First Big Trap — Read Performance

Replaying 10,000 events on every read is not acceptable.

❌ Naive

Read → replay all events

✅ Solution — Projections (Read Models)

1️⃣0️⃣ Projections (Materialized Views)

Projections turn events into queryable state.

events → projection → read API

Example Projection

on("OrderCreated", e => db.insert({ id: e.orderId }));
on("ItemAdded", e => db.addItem(e.orderId, e.item));
on("OrderShipped", e => db.updateStatus(e.orderId));

Kafka is the write database.
Projections are read databases.

1️⃣1️⃣ Eventual Consistency (You Must Accept This)

Writes:

• Immediate

Reads:

• Eventually consistent

If you need:

• Strong read-after-write

• Immediate consistency

Event sourcing may be a bad fit.

1️⃣2️⃣ Schema Evolution (The Silent Killer)

Events live forever.

You cannot:

• Change event meaning

• Rename fields casually

Rule

Events are public contracts.

1️⃣3️⃣ Versioned Events (Required)

OrderCreatedV1 { orderId }
OrderCreatedV2 { orderId, currency }

Consumers must handle all historical versions.

1️⃣4️⃣ The Second Big Trap — Exactly-Once Illusion

Kafka gives:

• At-least-once delivery

Your projection must be:

• Idempotent

• Offset-aware

if (alreadyApplied(eventId)) return;
apply(event);

1️⃣5️⃣ Deleting Data (GDPR Nightmare)

You cannot delete events.

Common Solutions:

• Encrypt PII and delete keys

• Redaction events

• Data minimization

Event sourcing + GDPR requires design-time planning.

1️⃣6️⃣ When Kafka Becomes Your Database (Danger Zone)

You are now responsible for:

• Backups

• Retention policies

• Compaction strategy

• Disaster recovery

• Cross-region replication

Kafka is a database, whether you admit it or not.

1️⃣7️⃣ When Event Sourcing Is the Right Choice

Use it when:

• Auditability is critical

• Business logic is complex

• Reprocessing is valuable

• History matters

• You can afford complexity

Examples:

• Payments

• Financial ledgers

• Order lifecycles

• Trading systems

1️⃣8️⃣ When Event Sourcing Is a Bad Idea

Avoid it when:

• CRUD is simple

• Strong consistency required

• Team is small

• Schema changes frequent

• Operational maturity low

🎯 Mental Model to Remember

Traditional DBs store answers.
Event sourcing stores evidence.