SYLEN
AboutNewsConferenceMembershipDonate

Email updates

Conference, news, and membership updates by email.

Site

  • About
  • News
  • Membership
  • Waitlist
  • Donate

Conference

  • Conference 2027
  • Call for papers

Account

  • Create account
  • Membership details

SYLEN

  • Guidelines
  • Privacy
  • Terms

© 2026 Systems Leadership and Engineering Network. sylen.org.

Membership details →
Back to news
InfrastructureSource: github.comOctober 23, 2026

ClickBench Disqualifies pgrust Engine Over Embedded Query Kernels and Result Caching

ClickHouse maintainer Alexey Milovidov merged Pull Request #2165 removing pgrust from the ClickBench benchmark after binary and source analysis revealed dedicated query kernels and non-disableable hot-run result caching. The analysis demonstrated that pgrust functioned as a static harness for ClickBench's 43 queries rather than a general-purpose query engine.

Embedded Physical Plans and Hardcoded Query Kernels Inspection of the `pgrust` v0.4-preview binary and public v0.3 source code confirmed that the engine bypassed general query planning by embedding static physical plans tailored exclusively to ClickBench. The compiled `aarch64` release binary contains direct source references ranging from `crates/gravity/backend/irprobe/src/q00.rs` through `q36.rs`, along with `tpch_q01.rs`. These files act as hardcoded IR probe kernels for specific benchmark queries.

In the v0.3 source, the benchmark SQL file `queries-clickbench.sql` is compiled into the executor crate via `include_str!`, referencing hand-lowered physical plans stored in `clickbench.ron`. Instead of evaluating queries through a general execution path, plan matchers inspect incoming SQL against fixed ClickBench patterns, emitting explicit error messages named after benchmark queries when structural deviations occur.

  • Queries Q0, Q1, Q2, Q3, Q6, Q19, and Q29 execute as `MetadataAnswer` phases, bypassing table scans entirely.
  • Query Q19 reconstructs result lines directly from predicate constants.
  • Query Q29 uses an algebraic rewrite (`SUM(x+k) = SUM(x) + k*COUNT`) resolved via metadata statistics.
  • Query Q18 requires a persisted derived column with baked bit widths of `26/6/20`.
  • Query Q28 uses a single-entry lookup table (`HOST_EXTRACT_PATTERN`) to evaluate regular expressions.

Hot-Run Verdict Replay and Un-Disableable Caching ClickBench rules mandate that query result caches and near-end pipeline caches must be disabled, and entries must be marked as untuned (`tuned: no`). `pgrust` violated these rules by caching filter verdicts and replaying previous run answers during warm execution phases.

Plan notes for Q20, Q21, and Q23 state that warm runs perform a pure fold over cached verdicts with zero columns scanned. On a 96-core Graviton (Neoverse-V2) test machine, `EXPLAIN` output on the v0.4-preview binary reported `SQE: engine (family: windowreplay)` for Q20, while Q1 executed under `family: metaanswer` with the underlying scan marked as never executed.

  • Query Q20 (`URL LIKE '%google%'`) required 10.8 seconds on a cold start, dropping to 11 ms on the second run and 5 ms on the third run.
  • Altering the predicate constant (`'%rambler%'`) yielded a 283 ms cold start, followed by 12 ms and 4 ms warm runs.
  • Disabling the configuration flags `pgrust.gravity_entry = off` and `pgrust.condition_cache = off` failed to disable the cache, with repeated `LIKE` operations continuing to return in 3 to 4 ms.

Absence of General Columnar Execution The engine lacks a general execution path for columnar tables backed by `pgrcolumnar2`, which does not support write operations (`INSERT`, `UPDATE`, or `DELETE`). When executing SQL queries that fall outside the 43 predefined ClickBench shapes, `pgrust` refuses execution rather than falling back to a slower general scan.

During validation, the binary rejected 20 out of 67 near-benchmark probes and 3 out of 49 perturbed query variants. Unserved shapes fail with `ERROR: sqe: statement over columnar relation "hits" is not served`. Rejections included basic SQL operations such as `SELECT MIN(URL) FROM hits`, `SELECT AVG(UserID + 1) FROM hits`, subqueries, and non-standard `HAVING` expressions. While the 43 benchmark queries returned mathematically correct output compared to `clickhouse local`, the implementation measured a specialized benchmark lookup table rather than a usable database system.

Read the original article at github.com.