Durability and the write-ahead log

Every change is written to one write-ahead log before it reaches the data file, each transaction chooses when its commit is acknowledged, and recovery replays the log at startup.

KDBS records every change to data in a write-ahead log (WAL) before the changed page may be written to the data file. Each transaction picks a durability class that decides when COMMIT returns: after the commit record is on disk (strict, group), or as soon as it is in the log’s memory buffer (relaxed). After a crash, the server replays the log when it starts: committed work is restored and unfinished transactions are rolled back before any client is served.

How it works

One log for the instance

An instance has one log stream, whatever the number of cores. Core 0 owns it; the other cores append to the same stream and ask core 0’s writer thread to sync it. Every fdatasync the instance pays is issued once, over one file. Each record has a position in the log, its LSN, which is one ordering for the whole instance.

The log is stored as segment files of 64 MiB each, in wal_dir (by default <data_file>.wal). The segment size is fixed by the data format, not a setting.

The write path

 statement changes a page
        │
        ▼
 1. append a log record to the in-memory WAL ring      (page remembers the record's LSN)
        │
        ▼
 2. COMMIT appends a commit record
        │
        ├── strict / group: wait until the log is synced past the commit record
        └── relaxed:        return now; the record is synced within the flush interval
        │
        ▼
 3. later: the checkpoint writes dirty pages to the data file,
    but only pages whose log records are already on disk

Records describe pages. The log is physiological: each record names one page and the change made to it, and replaying a record is safe to repeat. The undo records that hold old row versions are logged too, so both redo and rollback information survives a crash.
Log before data, enforced in code. The buffer pool refuses to write a page to the data file until the log is durable up to that page’s last record.
Group commit. Commits that arrive together share one sync.

Durability classes

Class	Also	`COMMIT` returns after	Can a crash lose an acknowledged commit?
`strict`	`D1`	The commit record is synced to disk	No
`group` (default)	`D2`	The commit record is synced to disk; the sync is shared by commits that arrive together	No
`relaxed`	`D3`	The commit record is in the log buffer	Yes, commits from the last `relaxed_flush_interval_us` (default 10 ms)

strict and group have the same durability point and differ only in batching. Use relaxed for data you can rebuild, such as a bulk load.

The class is chosen in three layers, each overriding the one before:

the server setting durability;
SET DURABILITY for the session’s next transactions;
BEGIN ... DURABILITY for one transaction.

kdbs> BEGIN ISOLATION LEVEL REPEATABLE READ DURABILITY STRICT
BEGIN trx_id=19 isolation=repeatable read durability=strict
kdbs> SET DURABILITY RELAXED
ERR cannot change the durability class inside a transaction
kdbs> ROLLBACK
ROLLBACK
kdbs> SET DURABILITY RELAXED
SET durability=relaxed

SET DURABILITY never applies to an open transaction: one that has already logged writes under relaxed cannot be given strict’s promise afterwards.

Checkpoints

A checkpoint writes dirty pages to the data file in the background and then records a redo start: the log position from which recovery must replay. It runs every checkpoint_interval_ms (default 5,000 ms) and is fuzzy: writes continue while it runs. Log segments wholly below the redo start are removed. A shorter interval means a shorter replay after a crash, and more checkpoint work.

Recovery at startup

When the server opens a data file, core 0 runs recovery over the whole log before any other core starts:

Analysis. Read the log from the redo start to its last valid record. A record with a bad checksum marks the end. Sort transactions into winners (a commit record was found) and losers.
Redo. Replay every record whose page has not seen it yet. This rebuilds the state at the crash, including the losers’ changes and undo records.
High-water repair. Move the page and transaction-id allocators past every id the log names, so no page or id the log uses is handed out again.
Undo. Roll back every loser through its undo chain. The compensations are logged, so a crash during recovery is itself recoverable.
Checkpoint. Write a checkpoint so the next recovery starts from here.

If the log holds losers and no undo phase is available, the server refuses to start rather than serve uncommitted writes as committed.

What is in the log

Logged	Not logged
Row inserts, updates, deletes and purges	Waystone trails (advisory; rebuilt by traffic)
B+ tree and index changes; a split as one record	Cabin entry sets (memory only; rebuilt by traffic after a restart)
Long values in the var-heap	`sys.access_stats`
Undo records
Catalog changes (DDL)
Assertion state (the Bound Cabin)

DDL is logged and recovered: SHOW META reports ddl_durable=1 and catalog_recovered=1. DDL that runs outside a transaction (assertions, Cabins, ALTER TABLE) has no commit record, so under strict and group the server syncs the log before it acknowledges the statement.

What it guarantees

No acknowledged loss under strict and group. A commit acknowledged under these classes survives a crash, with no partial transaction.
A bounded loss window under relaxed. A crash can lose only relaxed commits from the last flush interval.
No uncommitted data after a crash. Recovery rolls back every unfinished transaction before the database is served.
Torn log tails are detected. Every record carries a CRC32C checksum; replay stops at the first damaged record.
A failed write stops writes, not correctness. If a log write or a log sync fails, the log stops: every later write on every core is refused until a restart, and SHOW META shows wal_stopped=1. The engine does not retry a failed sync, because on Linux a failed fsync can drop data and report success the next time.

What it costs

strict and group wait for the disk. Each commit, or each batch of commits, pays a device sync.
relaxed trades a loss window for latency. Acknowledged commits from the last few milliseconds can be lost.
Recovery time grows with the log since the last checkpoint. The checkpoint interval is the knob.
A segment switch holds the log. When a segment fills, the switch syncs and creates files while holding the log’s append latch. Measured at relaxed under one bulk writer producing about 75 MiB/s of log (engine v2.7.0-909-g4cf30270): the hold was about 330 ms at p50 and 790 ms at p99. At group it was about 6 ms at p50 and 20 ms at p99.

How to observe it

SHOW META reports the log and the last recovery. An excerpt from a sandbox instance:

kdbs> SHOW META
version=23 ... wal_topology=single ... wal_durable_lsn=46600 wal_syncs=19 wal_interval_syncs=0 wal_sync_failures=0 wal_first_segment=0 wal_segments_removed=0 ... wal_group_commits=13 wal_group_batches=13 wal_mean_group_batch=1.000 ... recovery_records=0 recovery_committed=0 recovery_rolled_back=0 ... catalog_recovered=1 ... ddl_durable=1 ...

Field	Meaning
`wal_durable_lsn`, `wal_append_lsn`	How far the log is synced, and how far it is written
`wal_syncs`, `wal_group_commits`, `wal_mean_group_batch`	Syncs paid, and how many commits each sync covered on average (`1.000`: every commit paid its own)
`wal_first_segment`, `wal_segments_removed`	Log recycling
`recovery_*`	What the last startup’s recovery did: records read, transactions committed and rolled back, time per phase
`wal_stopped`	`1` after a failed log write has stopped all writes
`checkpoint_*`	The last checkpoint’s work and timing

The SYNC command forces the log and the data file to disk and answers OK synced.

Limitations

Torn data pages are detected, not always healed. Every page has a CRC32C checksum, so a torn page is found when it is read. It is restored only if the log holds a full image of it. Logging a full image of every page after each checkpoint is not built yet; today only some paths log images (an index build, a bulk fill, part of an assertion’s storage).
A crash while undoing a failed commit can replay it as committed. If a COMMIT’s sync fails, the engine rolls the transaction back, but the abort record is written without waiting for the disk. If the commit record reached the disk and the abort did not, recovery treats the transaction as committed, although the client was told it failed.
Cabins and Waystone trails start empty after a restart. Their contents are rebuilt from traffic; results do not change, only speed.