No VACUUM, no query planner: how less machinery makes KDBS more flexible

Two pieces of machinery run inside nearly every relational database: something that cleans up old row versions, and a planner that chooses how to run each query from statistics. KDBS has neither. That is not because they were left for later. The engine’s storage and execution are built so that it does not need them, and removing them turns out to give the engine more freedom rather than less.

Why there is no VACUUM

Where the garbage comes from

Multiversion concurrency control (MVCC) lets readers see a consistent snapshot while writers keep working. The question is where the old versions live.

In PostgreSQL, an UPDATE writes a complete new copy of the row into the table, and the old copy stays behind as a dead tuple. Something has to find dead tuples and reclaim their space later. That is VACUUM, usually run by autovacuum. When it falls behind, tables and indexes bloat, scans read pages full of dead rows, and operators tune thresholds and schedule maintenance windows.

What KDBS does instead

KDBS keeps the newest version in the row itself and the older versions somewhere else:

• An UPDATE writes in place, in the same page and slot. The previous version is copied into an undo record linked from the row. A reader with an older snapshot follows the link back until it reaches a version it may see.
• Rows have a fixed length per table. A longer value changes a cell’s tag and stores the bytes in a separate area. The row never grows, so it never has to move.
• Undo records are recycled automatically once no open snapshot can reach them. The engine tracks the oldest open snapshot, the purge horizon.
• That horizon is kept short. An explicit transaction older than txn_lifetime_ceiling_ms (60 seconds by default) is rolled back, so one forgotten session cannot hold old versions for the whole instance.

 row on the page             undo records
┌──────────────────┐        ┌──────────────────┐        ┌──────────────────┐
│ balance = 900    │──────▶ │ balance = 950    │──────▶ │ balance = 1000   │
│ trx_id = T3      │        │ prior trx = T2   │        │ prior trx = T1   │
└──────────────────┘        └──────────────────┘        └──────────────────┘
   newest                                                  oldest

An update leaves no dead copy in the table. Updates do not bloat it, no background process rewrites it, and a row’s location stays stable across updates.

What is left

Deleted rows are the one kind of leftover. They stay in place, with their keys, until an explicit PURGE. This is on purpose: the key is the row’s identity (see the previous post), so freeing it is a decision an operator makes, not one a background task makes alone.

There are other costs to know about. The data file never shrinks: pages of a dropped table are reused, but space is not returned to the file system (What KDBS is not). Superseded secondary-index entries are not reclaimed yet (Secondary indexes).

Why there is no query planner

What a planner does, and what it costs

A cost-based planner reads table statistics and picks a plan: which index, which join order, which join algorithm. Most of the time it chooses well. But the plan depends on statistics, and statistics change. A query that took 2 ms yesterday can take 2 seconds today because a table crossed a threshold and the planner switched strategies. The SQL did not change, and neither did the schema. That is a plan regression, and it is one of the most common causes of sudden latency incidents.

What KDBS does instead

KDBS compiles each statement once, when it is parsed, into a step chain: one step per table access, run in the order the statement is written. The access method for each step is chosen from two things only:

1. The statement’s shape. An equality on the primary key becomes a Lookup. A BETWEEN on the key becomes a Range. A join on the inner table’s key becomes a Probe.
2. What the catalog holds. For an equality on a non-key column, the order of preference is fixed: secondary index, then Cabin, then filter scan.

Row counts and data statistics are never consulted. The plan changes when the catalog changes, never when the data does:

kdbs> ANALYZE SELECT * FROM passengers WHERE name = 'Park Jiho'
step 0 FilterScan passengers opens=1 examined=8 matched=1 sel=12% pages=1

kdbs> CREATE INDEX ix_name ON passengers(name)
kdbs> ANALYZE SELECT * FROM passengers WHERE name = 'Park Jiho'
step 0 IndexProbe passengers opens=1 examined=1 matched=1 sel=100% pages=2 index_scanned=1 index_resolved=1

The same statement over the same catalog compiles to the same chain at ten rows or ten billion. You can predict a statement’s cost from its text, and a statement that touches too many rows fails with an error instead of running away.

There are costs to this too. You choose the join order, so put the table that narrows the result first. The engine also cannot decide to skip an index on a tiny table, the way a cost-based planner can.

Forms a step chain does not run, such as outer joins, OR, HAVING, set operations and derived tables, go to a separate query layer. It walks each table with a single-table step chain, then joins, filters and groups the rows in memory, in written order, within fixed row bounds. It does not consult statistics either, and a statement past a bound fails instead of spilling. See the query layer.

Why fixed plans make the engine more flexible

It sounds backwards: an engine that refuses to rethink its plans should be the rigid one. In practice the fixed plan is what lets KDBS change its own physical structures freely.

A statement shape has a stable identity

Because the plan depends only on the shape, the shape can be named. The parser replaces literals with placeholders and hashes the result into a pattern_id, so WHERE id = 42 and WHERE id = 7 share one. That identity holds across executions, table sizes and restarts.

The engine attaches what it learns to it. From the second time a statement runs with the same arguments, KDBS records where it found its rows as a Waystone trail, and later runs can read those locations directly. SHOW PATTERNS and SHOW ACCESS report the workload by shape. None of this would hold together if the same SQL could run as a different plan tomorrow.

Structures can come and go without risk

In a planner-driven database, adding an index is a change with side effects: it can change plans for queries you never meant to touch. Teams test index changes and sometimes add query hints so the planner keeps its choices.

In KDBS, a new structure affects only the steps that the fixed preference order sends to it, and every learned structure is advisory: it can make a query faster or slower, never change its result. Trails and Cabin location hints are checked against the row they lead to (keys never change, so the check is exact). A Cabin that has forgotten a value sends queries for it back to a scan. Because nothing can go wrong, the engine is allowed to act on its own:

• The Cabin controller watches how often each statement shape runs and how many pages it scans. It creates a Cabin on a column when the measured benefit clears its cost by a wide margin, and drops the Cabin when the benefit fades. It is off by default and is turned on with one setting.
• Its decision is a pure function of a statistics snapshot, so a given trace always produces the same decisions and can be replayed in tests.
• A wrong decision costs page reads, never a wrong answer.

A planner adapts by changing how it runs a query, and it puts every result-affecting choice at risk each time. KDBS adapts by changing what exists next to the data, and the result never depends on it. The fixed plan is the stable base that lets everything above it move.

Predictability is something you can build on

When the execution order is the written order, behavior that is usually undefined becomes defined. LIMIT n OFFSET m without ORDER BY returns rows [m, m+n) of the unlimited result. An application, or an agent writing SQL for one, can work out what a statement will cost before it runs it, and the work does not change as the data grows.

Summary

	Typical engine	KDBS
Old row versions	In the table; VACUUM reclaims them	In undo records; recycled past a short purge horizon
Deleted rows	Reclaimed by VACUUM	Kept with their keys until an explicit PURGE
Plan choice	From statistics; can change with the data	From the statement’s shape and the catalog; fixed
Adding a structure	Can change unrelated plans	Affects only the steps that use it; advisory structures never change a result
Adapting to the workload	Re-planning	Creating and dropping advisory structures

To see it yourself, run ANALYZE on a few statements in the sandbox, add an index, and run them again. The details are in Statements and execution plans, Transactions and isolation and Physical optimizer.