Known gaps

v3.0.0

What KDBS v3.0.0 does not do yet, what a restart loses, and the costs and limits that are known but not fixed.

In this article

KDBS keeps an engine-wide list of what is missing, what does not survive a restart, and where the code falls short of its design. This page summarizes that list in plain language, grouped by area. Read it before you rely on KDBS for data you care about. Most gaps cost performance or cause a retryable refusal; the page says so where a gap could lose a write or change an answer.

What a restart loses

Some state is rebuilt after every restart, clean or not:

  • Cabin contents. A Cabin’s entries live in memory. After a restart every Cabin starts empty and traffic fills it again; until then, queries on that column scan. The Cabin’s definition is kept.
  • The Cabin controller’s state. Its managed state and decision log live in memory and are rebuilt by observing the workload again.
  • Waystone trails and access statistics. Every start clears all trails and empties sys.access_stats. Repeated queries record new trails. The pages the old trails used stay allocated and are never freed, so the data file grows a little with every restart.
  • Assertions are rebuilt, not lost. Each assertion’s running totals are rebuilt when the database is mounted, and it enforces immediately. SHOW ASSERTIONS reports enforcing=0 for an assertion whose totals could not be rebuilt.

After a crash (a killed process, a power loss), also:

  • Writes inside the relaxed loss window. A relaxed commit is acknowledged before it reaches the device. See Choose a durability class.
  • The free-page list. Pages freed during the run are hidden until a later clean restart finds them again.
  • A PURGE that was cut off leaves its key range partly purged, each key whole. Running it again finishes the range. Spilled values of a row cut off between its retire and its release are never reclaimed.

Not built

These are not in v3.0.0.

FeatureStatus
Expressions in INSERT ... VALUES, in a select list, or in WHERENot built yet. Expressions work in UPDATE ... SET only.
Inside UPDATE ... SET: IN, BETWEEN, LIKE, EXISTS, IS TRUE, boolean literals, <>, correlated subqueries, UPDATE ... FROM, DEFAULT, INTERVALRefused by name at their byte position
Reuse of a prepared statement’s parseNot built yet. A prepared statement is re-parsed at every execution, and a text parameter containing a single quote cannot be bound.
Physical relayout (moving rows to a better layout)Not built yet. SHOW RELAYOUT only reports, and it proposes nothing for tables created since new tables became B+ tree only.
Incremental, compressed or encrypted backup; object-storage target; parallel copy; concurrent backups; logical dump; streaming standbyNot built yet
Transition constraints (allowed state changes per column)Concept
At-most-once execution of a retried requestConcept
Cache relations kept consistent with the rows they cacheConcept
Identity succession (an old id resolving to the row that replaced it)Concept
Optimizer levers exposed for an external agentConcept

Transactions and locking

  • The transaction lifetime ceiling covers explicit transactions only. txn_lifetime_ceiling_ms rolls back an explicit transaction that runs too long. An autocommit statement is not bounded: a long autocommit SELECT holds its snapshot for as long as it reads. A statement that is running is never interrupted, so it delays the rollback until it ends. Background read views are not reached.
  • A rolled-back session keeps its portals. On the KWP protocol, a portal bound inside a transaction the ceiling rolled back still delivers its rows, read under a transaction that no longer exists.
  • A wake is not a grant. A writer that waits for a row is woken when the holder finishes, but a third writer can take the row first. Under sustained contention the wait ends at the 1 s fault limit (lock_wait_fault_net_ms) with a retryable refusal.
  • A failed statement can keep parent locks. A statement that fails for any reason other than a foreign-key violation keeps the shared locks it took on missing parent rows until the client rolls back. Other writers are refused meanwhile; no answer is wrong.
  • Large transactions pay for lock release. Releasing locks grows with the square of the number held. At 16,384 distinct parent rows in one transaction, the decide step took 49 ms.
  • PURGE waits on older snapshots. One open older snapshot blocks every PURGE of a key deleted after it; the PURGE is refused after 1 s with a retryable TxnConflict. Nothing wakes the wait early.
  • An insert that leaves out the key can be refused AlreadyExists. This happens only if other clients keep naming the exact id the engine is about to issue, eight times in a row. The refusal carries no byte position.
  • Eight cores appending to fresh tables can see a retryable refusal: btree descent for key K from page P gave up after 5 attempts. The cause is not established. A client that retries loses nothing.
  • A subquery in UPDATE or DELETE can resolve a table another transaction created and has not committed.

Statement scheduling

  • Only a SELECT’s outer scan gives its core back, every 64 pages. A write, a primary-key lookup, an index walk or a join’s inner scan holds the core until it ends or hits its row budget. Other sessions on that core wait.
  • A cancel takes effect only where a scan yields, or at the next request. A write that a client cancels or disconnects from runs to its end. A dropped connection can still find its write applied.
  • At most 256 suspended scans per core. The 257th is refused with OutOfSpace.
  • TRACE ON is per core, not per session. When a session moves to another core, its trace stops, and another session on the traced core is traced instead.

Write-ahead log and checkpoints

  • A log segment switch holds the log’s lock during device I/O. Under one bulk writer at relaxed, the hold is about 330 ms at p50, 790 ms at p99 and up to 1.2 s; at group it is about 6 ms at p50 and 20 ms at p99. While it lasts, point reads on every core stall: their scheduling p99 stays near 277–295 ms. No fix is planned yet.
  • Every log sync covers every live segment, about 50 µs per segment, so the cost grows with the log kept since the last checkpoint.
  • A relaxed checkpoint stall is 226–258 ms per server life (median). A checkpoint still holds its core for more than 10 ms while it writes.
  • An undo that meets a full buffer pool while holding a page can be refused, leaving a write not undone. No code path that reaches this was found, and nothing prevents one.

Storage and space

  • The data file never shrinks. Freed pages are reused inside the file.
  • Some pages are never freed: pages left by a failed tree split or failed table or index creation, a refused catalog write’s page, a previous run’s undo pages, a dropped assertion’s pages, and Waystone pages.
  • A dropped table’s pages are freed after the next checkpoints, about two checkpoint intervals later. With cores > 1 and checkpoint_interval_ms = 0, nothing is freed until a clean shutdown. A refused reclaim waits for the next start.
  • An UPDATE that replaces a spilled value never releases the old one.
  • Space freed by PURGE returns only when the leaf page fills again, and a purged row’s secondary-index entries are never removed.
  • SHOW PAGE on a reused page id shows the new owner’s bytes.

Cabins, trails and the optimizer

  • A Cabin builds rarely on a busy instance. Any write in flight on any core declines every Cabin build. A Cabin may never fill on a write-heavy instance; SHOW CABINS reports this as unbankable_views=.
  • SHOW CABIN_OPTIMIZER can stall its core for a whole Cabin build when the controller is building one.
  • Two cores recording the same query can lose a trail. This costs a replay, never a result.
  • The Cabin controller is off by default. Its demand score bottoms out after about 16 half-lives, so a Cabin dropped after a long quiet period is dropped by timeout rather than by judgement. A Cabin that is no longer used and one that is quiet overnight send the same signal.

Backup and restore

  • A torn page is refused, not repaired. A snapshot that does not freeze the file system is a crash image; every page checksum is checked, and a torn page is named.
  • A restore leaks pages that the source freed during the copy, plus old Waystone and statistics pages.
  • Point-in-time recovery targets a log position, not a time. Note wal_durable_lsn from SHOW META.
  • kds_backup reaches the admin role only on a server with auth = off.
  • An archive_command that hangs holds the archive until shutdown.
  • Archiving shares the log device’s bandwidth: wide INSERTs lose 27–29 % with the archive on the data device, 3 % elsewhere. At cores = 1, a base copy multiplies the workload’s p99 by four while it runs.

See Back up and restore.

Performance

Measured costs that are known and not resolved:

  • A scanned row costs about twice PostgreSQL’s: 67 ns of server CPU against 34 ns on the same host and driver. No single part dominates.
  • Waystone records a trail for a point SELECT by primary key, costing 7–9 % of its server CPU for a lookup the tree already answers.
  • The request round trip has a floor of about 26 µs at p50 on a quiet host, mostly in the kernel’s network calls.
  • A scan of resident pages is 2.5–3.7 % slower than before the bounded buffer pool, partly unexplained.
  • A large scan can push out a hot working set that is touched less often than the scan cycles through the pool.
  • An unexplained stall of about 0.4 s appeared in three of seven runs of one multi-core benchmark.

Testing

  • A few multi-core tests time out or fail intermittently under a loaded host. Their bounds were not widened, because no cause has been observed.
  • Some rules are argued, not tested: parts of the bounded buffer pool, and the fix to how assertions resume after recovery.
  • There is no sanitizer build.

See also