Decoupling Storage for LTAP: Inside the Lakebase Stateless Postgres Architecture
Databricks has introduced Lakebase, a serverless Postgres architecture that externalizes the write-ahead log and data files into independent cloud services called SafeKeeper and PageServer. By materializing transactional data into open columnar formats on cloud storage, Lakebase enables a Lakehouse Transactional Analytical Platform (LTAP) model where analytics engines and Postgres read from a single data copy without CDC pipelines.
The Monolithic Storage Bottleneck
Traditional database engines, including PostgreSQL, tightly couple the query processor, the write-ahead log (WAL), and data files onto a single machine. Under the standard design, write performance is optimized by appending changes sequentially to the WAL, while read performance relies on asynchronous background updates to the main data files.
This monolithic configuration introduces severe operational vulnerabilities. If the operating system or storage layer misreports flushes, commits can be lost silently during power failures. Furthermore, scaling reads or implementing high availability requires provisioned physical replicas that duplicate the entire dataset and continuously replay the WAL. Generating these clones is slow, resource-intensive, and risks degrading the primary database. Finally, executing complex analytical queries on the same hardware as transactional workloads inevitably causes resource contention, starving latency-sensitive OLTP tasks.
Decoupling Compute and Storage via SafeKeeper and PageServer
To resolve these architectural limitations, Lakebase makes the PostgreSQL compute layer stateless by offloading the log and data files to independent, elastic cloud services. This separation is handled by two specialized components:
- SafeKeeper: An externalized service dedicated to managing the write-ahead log. It handles durable write coordination, ensuring sequential write safety and high availability without requiring full physical standby clones.
- PageServer: An independent service that ingests the WAL from SafeKeeper, processes it to maintain up-to-date data pages, and serves read requests back to the stateless Postgres compute engine.
By externalizing these functions, the compute layer can scale up or down instantly. This decoupling also enables zero-copy branching and virtually unlimited storage capacity without introducing meaningful network latency.
Storage-Level Unification via LTAP
The Lakehouse Transactional Analytical Platform (LTAP) architecture builds upon this decoupled storage engine to bridge the gap between transactional and analytical workloads. Traditionally, Hybrid Transactional/Analytical Processing (HTAP) architectures attempted to execute both workloads within a single database engine, resulting in performance compromises. LTAP instead unifies the system at the storage layer while maintaining specialized engines for each execution path.
Lakebase automatically materializes incoming transaction data into open columnar formats, such as Parquet on S3 cloud storage. Both the Postgres engine and external Lakehouse engines can read directly from this single, shared storage format. This approach completely eliminates the need for complex Change Data Capture (CDC) pipelines, ETL workflows, and duplicate data volumes. Analytical query engines gain real-time access to fresh transactional data without imposing any compute or memory overhead on the primary OLTP engine.