Thread-per-core execution

How KDBS runs one pinned, cooperative reactor per core, runs every statement on its session's core, and shares pages, the buffer pool and the log.

KDBS runs one worker thread per configured core, pinned to a CPU, and creates no threads after startup. Each thread runs a reactor: a cooperative event loop that runs tasks until they finish or yield. Every statement runs, whole, on the core its session is on. No table or page belongs to a core: pages, the buffer pool and the write-ahead log are shared by all cores, and locks and latches decide who may touch what.

The idea

There is no placement to get wrong. A statement does not need to be routed to the core that owns its data, because no core owns any data. Two writers on two cores meet in the same two places two writers on one core meet: the page latch, for a page’s bytes, and the lock table, for a row or a table.

Terms used on this page:

Reactor: the event loop on one core.
Task: a unit of work on a reactor. Tasks are C++20 stackless coroutines.
Lock: a transaction-scoped right to a row or table. A task that cannot get one parks until the holder commits or rolls back.
Latch: a short right to a page or structure, held for a critical section and never across a park.
Kick: a wake-up signal from one core to another. It carries no data.

How it works

            clients (one port; every core listens)
                 |         |          |
            +----v---+ +---v----+ +---v----+
            | core 0 | | core 1 | | core 2 |   one pinned reactor each
            +----+---+ +---+----+ +---+----+   every statement runs whole
                 |         |          |         on its session's core
   ==============+=========+==========+==================
     shared: pages (page latch) · buffer pool (one frame table)
             lock table · one WAL stream (core 0 owns it, all append)

Sessions and cores

With cores > 1, every core binds the port with SO_REUSEPORT and the kernel picks which core accepts a connection. A client cannot choose its core.
A session can move between transactions. At the start of each transaction the server may move the session to a core that has been markedly less busy. A session moves at most once per half second, and a session whose own work makes its core busy is not moved. A transaction never crosses cores: it begins, waits and commits on one.
Where the platform refuses SO_REUSEPORT, core 0 accepts every connection and hands each to a core round-robin. Linux never refuses it.

The reactor loop

Each iteration runs fixed phases in a fixed order: drain I/O completions, expire timers, run ready tasks, submit pending I/O, and block if nothing advanced. A block always has a ceiling (10 ms), so a missed wake-up costs latency, never liveness.

No preemption and no work-stealing. A task runs on the core that created it until it finishes or yields.
Scheduling groups. Every task belongs to foreground (statements), maintenance (statistics, relayout and similar) or system (checkpoint and log housekeeping). Each group has a fixed share; the scheduler picks from the group that has used least of its share, and every group with a ready task gets at least one turn per iteration. Shares are static: nothing adjusts them from measured latency.
Long reads yield. The outermost walk of a SELECT gives its core back every 64 pages, so a long scan does not hold every other session on that core until it ends.

Shared state

Pages and the buffer pool. One frame table serves the instance. A page one core read from disk is served to every other core from the same frame. buffer_pool_frames is one total for the instance.
The log. There is one write-ahead log stream per instance. Core 0 owns it, every other core appends through it, and every fdatasync is issued once. See Durability and the write-ahead log.
Catalog and statistics. Every core writes sys.patterns and sys.access_stats itself; both are one relation for the instance. A Cabin store is one for the instance, and every core appends into it.
Cross-core signals. A core that needs another to act writes shared state under that state’s own latch, then kicks the other core. The kick has no payload; the woken core learns what to do from the state it was waiting on. There are no messages or queues between cores. A kick can be missed, which costs at most one idle block (10 ms).

What it guarantees

A statement reads under one snapshot, on one core, so no part of its answer is read at a different instant.
A transaction runs on one core from BEGIN to COMMIT.
The core count is not stored on disk. You can restart with a different cores value; the new count is recorded and logged.
The whole engine can run single-threaded under a simulated scheduler. The clock, randomness and all I/O are injected, so a failure reproduces from a seed.

What it costs

Each extra core is a pinned thread. Cores must not exceed the machine’s reported CPU count, because reactors never block and overcommitting them serializes work.
Shared structures need latches above one core. At cores = 1 most of them compile to a branch and cost nothing.
Every core checkpoints in its turn. In one census, checkpoint runs per 5 s period were 1, 2 and 3.9 at 1, 2 and 4 cores, and pages written per 1,000 rows rose 11 to 18% from one core to four (known gap; bench/v3.0.0/results-bp-s1-census-v2.7.0-909-gccf9cfc1.md).

How to observe it

SHOW META reports per-core scheduler counters, among them:

Field	Meaning
`sched_wall_us`, `sched_iterations`	Reactor wall time and loop iterations.
`sched_<group>_polled_us`, `sched_<group>_polls`	Cumulative time and polls per group (`foreground`, `maintenance`, `system`).
`sched_<group>_longest_poll_us`	Longest single poll in the group.
`sched_idle_block_us`, `sched_idle_blocks`	Time and count of idle blocks.
`sched_wakes_sent`, `sched_wakes_received`	Kicks sent and received. Across the instance the two must be equal.
`load_c<k>_*`	Each core’s published load, above one core only.
`contention_*`	Summed over every core: where one core waited for another.

sched_wall_us − Σ sched_*_polled_us − sched_idle_block_us is reactor time that no group was charged for and that was not sleep. The log’s sync counters are read on core 0.

Configuration

Key	Default	Effect
`cores`	`1`	Reactor threads, one pinned per CPU. Above 1, all share one log stream.
`reactor_cpus`	(unset)	The CPU each reactor is pinned to, core 0 first, such as `reactor_cpus = 0,2`. It must name exactly `cores` distinct CPUs of this machine, or the server refuses to start. Unset, reactor `k` runs on CPU `k` and core 0 runs unpinned, which on hosts that number SMT siblings next to each other puts reactors 0 and 1 on one physical core.
`buffer_pool_frames`	(required)	Buffer pool size for the whole instance, in 8 KiB frames.

The keys placement, peer_listeners and range_size_ids are refused at startup: placement of tables on cores no longer exists. See Server configuration.

Limitations

Only a SELECT’s outermost walk yields. A write, a primary-key lookup, an index walk and a join’s inner walk under one outer page hold their core until they end or the statement budget stops them.
Cancel is seen only where a walk yields, or at the next request. A write asked to stop runs to its end, so a dropped connection can still find its write applied.
At most 256 suspended walks per core. The 257th is refused OutOfSpace at its start.
A bulk writer can stall point readers on every core. A log segment roll holds the log’s latch across device I/O, and a reactor waiting for that latch blocks every session on its core.
TRACE ON is per core, not per session. A traced session’s trace stops when the session moves.
No setting turns the session balancer off.

See Known gaps for the full list.