跳到正文
Databricks:Blog·· 1 天前AI 评分32

什么是 HTAP?混合事务/分析处理简明指南

What is HTAP? A plain-English guide to hybrid transactional/analytical processing

AI 导读

HTAP(混合事务/分析处理)由 Gartner 于 2014 年提出,指在同一集成数据库系统内融合 OLTP 与 OLAP,让分析查询直接读取当前运营数据,无需等待批量 ETL 搬运。常见架构分为共用单一引擎的 Converged HTAP 与分离事务、分析引擎的 Composed HTAP,二者在性能、隔离性、数据一致性与新鲜度上各有取舍。文章还对比了 HTAP 与 LTAP,并给出选型考量。

正文

Consider a payment system processing a purchase. Before approving the transaction, a fraud model may need to evaluate it against the account's recent activity. But if that transactional data must first move from the transactional database to a data warehouse for analysis, even a short delay can make the result less useful.

Hybrid Transactional/Analytical Processing (HTAP) brings transaction processing and analytics closer together to address this problem. An HTAP database lets you run analytical queries on operational data while it is still current, without waiting for a batch data pipeline to move it into a separate analytical system.

In this guide, you'll learn how HTAP works, the architectural tradeoffs involved, and where it fits in a modern data architecture. You'll also see how different HTAP systems approach transactional and analytical workloads, how HTAP compares with Lake Transactional/Analytical Processing (LTAP), and what to consider when evaluating and adopting either approach.

TL;DR

  • HTAP brings transactional and analytical processing closer together so applications can run analytics on fresh transactional data.
  • Different HTAP architectures involve different trade-offs in performance, isolation, data consistency, and freshness.
  • Choose the architecture based on your workload. LTAP provides greater separation between transactional and analytical compute while both operate on the same data.

What is HTAP?

HTAP stands for Hybrid Transactional/Analytical Processing. Gartner coined the term in 2014 to describe architectures that bring online transaction processing (OLTP) and online analytical processing (OLAP) together within an integrated database system. Gartner's original framing of hybrid transaction/analytical processing argued that combining the two would create opportunities for dramatic business innovation, because decisions could be OLTP handles high volumes of small, frequent transactional operations that keep applications running, such as creating an order, processing a payment, or updating an account. OLAP processes complex queries over large datasets to calculate metrics, identify patterns, and answer analytical questions.made on the latest transactions rather than on yesterday's extract.

image2.png

HTAP brings these workloads closer together, giving analytical queries access to current operational data without depending entirely on a separate batch ETL pipeline. For many operational analytics use cases, HTAP systems can reduce or remove the need for ETL processes, cut down the number of data copies teams maintain, and move closer to a single source of truth for both operational and analytical systems. In the most integrated designs, OLTP and OLAP capabilities live in a single system rather than in two products stitched together.

Bringing OLTP and OLAP workloads closer together does not mean they have to run in exactly the same way. HTAP architectures therefore differ in how they combine OLTP and OLAP capabilities while preserving the characteristics each workload needs. Two common approaches are:

  • Converged HTAP: Runs transactional and analytical workloads through a single database engine. The workloads may use different indexes, storage formats, or execution paths, but they share much of the same database.
  • Composed HTAP: Uses separate transactional and analytical engines within an integrated architecture. The engines may share a storage layer or propagate changes through replication while remaining optimized for their respective workloads.

A converged engine avoids the need to synchronize independent query engines, but it must prevent analytical operations from interfering with latency-sensitive transactions. A composed architecture provides greater freedom to optimize and isolate each workload, but it must coordinate data versions and keep the analytical path sufficiently fresh.

In both cases, the central HTAP challenge is the same: provide analytical access to operational data quickly enough to be useful without compromising the performance and reliability of transactional workloads.

Why OLTP and OLAP became separate systems

OLTP and OLAP have different access patterns. Transactional queries typically read or update a small number of complete rows. OLTP databases therefore rely on row-based storage, selective indexes, efficient concurrency control, and predictable latency, often measured in milliseconds. Most transactional databases are relational databases tuned to keep that latency low even under heavy concurrency.

Analytical queries, by contrast, often scan millions of rows across a subset of columns to calculate metrics or identify patterns. OLAP databases use columnar storage because a column store improves compression and reduces the amount of unnecessary data read during large scans and aggregations. OLAP systems can tolerate longer latency for complex queries, trading seconds of response time for the ability to analyze far more data.

Optimizing OLTP and OLAP systems separately therefore made practical sense, but it introduced a new problem: the delay caused by data movement between them. Operational data typically had to be copied out of the OLTP system and into a data warehouse or other analytical database before it could be analyzed. Transactions and analytics processing ran on different schedules, on different hardware, against different copies of the data. Traditional architectures that separate OLTP and OLAP systems in this way build latency into every analytical answer.

As requirements for fresher data grew, data teams started assembling their own custom systems to close the gap. Change data capture feeds, stream processors, cloud data warehouses, cloud data lakes, and real-time query layers became the building blocks of today's disaggregated data stack. Each component solved a specific problem, but every additional hop added cost, operational effort, and another place where data consistency could break.

HTAP addresses that delay while preserving the performance benefits of workload-specific optimization.

How HTAP architecture works

An HTAP architecture provides paths for both transactional and analytical workloads while keeping operational data available for analysis with minimal delay. The exact design varies by system, but the architecture generally includes transaction processing, analytical processing, workload-specific data representations, and a mechanism for keeping those representations synchronized, as shown below.

image1.png

Transactional processing

The transactional path handles frequent application reads and writes, such as creating orders, updating accounts, or recording payments. These transactional workloads typically operate on small numbers of rows and require low latency, efficient concurrency control, and reliable transaction guarantees.

HTAP systems commonly use row-oriented storage engines, indexes, caches, and a write-ahead log designed to serve these operational access patterns efficiently and durably.

Analytical processing

The analytical path handles analytical queries that scan, filter, join, and aggregate larger amounts of data. Columnar representations are commonly used because analytical queries often need a subset of columns across many rows, and they are what make fast OLAP queries possible on operational data.

Depending on the HTAP design, analytical processing may run in the same database engine as transactions or on separate analytical compute. Separating the compute paths allows each workload to use execution resources optimized for its access patterns, so the system can add analytical capabilities without slowing OLTP systems down. Many HTAP systems also use intelligent query optimization, where the optimizer decides whether a query should read from the row store, the column store, or both.

Storage and data representation

HTAP systems may maintain more than one representation of the same data. A row-oriented storage engine can serve transactional access while a columnar storage engine supports analytical scans and aggregations.

These representations may exist within one database system or across separate transactional and analytical components. Some architectures also decouple compute from storage and use object storage alongside caches, logs, or other services.

Synchronization and freshness

When transactional and analytical processing use different storage engines or components, committed changes need a path from the transactional side to the analytical side. Systems can use mechanisms such as change data capture (CDC), replication of the write-ahead log, shared logs, or shared storage to propagate those changes.

The result is an analytical view that can incorporate recently committed transactions without requiring the traditional process of periodically extracting an entire dataset into a separate analytical system. How quickly those changes become visible depends on the architecture and its consistency guarantees.

Challenges of HTAP databases

Bringing transactional and analytical workloads closer together introduces tradeoffs in how data is stored and processed. Where those costs appear depends on the architecture, but HTAP databases struggle with the same core problem: optimizing for both OLTP and OLAP workloads at once. Four challenges are common.

Write amplification

Maintaining analytical data alongside frequent transactional writes creates additional work. Because many HTAP systems store data in two formats, updates may generate deltas or require refresh and compaction, increasing compute, I/O, and storage usage even when that work stays off the transaction path.

Resource contention

When OLTP workloads and OLAP workloads share resources, large analytical queries can push transaction latency from milliseconds to seconds. Separate compute provides stronger isolation, but you still need to account for shared bottlenecks such as storage, networking, and coordination services.

Consistency and freshness

Transactional and analytical paths may not advance at the same rate. You need to decide how quickly committed changes must become available for analysis and what data consistency guarantees the workload requires. Greater freshness typically requires tighter coordination between the two paths, and every HTAP architecture manages some tension between synchronization and isolation rather than eliminating it.

Operational complexity

Supporting mixed workloads can mean managing multiple data representations, execution paths, resource policies, and synchronization mechanisms. Query optimizers must also choose good execution plans across two storage formats. You need to account for that complexity when monitoring, scaling, troubleshooting, and recovering the system.

Evaluating HTAP solutions: Checklist

When evaluating an HTAP solution, test transactional and analytical workloads together under conditions that reflect your production environment. Measure whether transaction latency, analytical performance, and data freshness remain within your requirements as both workloads run concurrently.

CriterionWhat to test
OLTP p99 latency under mixed loadCheck whether p99 (99th-percentile) transaction latency stays within your target as analytical load increases.
Analytical performance at scaleTest scan and aggregation performance as your data volume, query complexity, and concurrency grow.
Freshness and consistencyMeasure how quickly committed changes become queryable and whether the consistency guarantees meet your requirements, including when the analytical path falls behind.
Cost under mixed loadMeasure compute, storage, caching, replication, network, and data-transfer costs at traffic levels representative of your workload.
Failure and recoveryTest how queries behave during failures, whether correctness is maintained under degraded conditions, how backlogs grow, and how quickly the system recovers.
Operational overheadConsider how much effort you will need for tuning, monitoring, capacity planning, workload management, and incident response.

Use production-like data, query patterns, volumes, and concurrency wherever possible. Include bursty writes, skewed data, traffic spikes, and demanding analytical queries so you can see how the system behaves under the conditions it will actually face.

Real-world HTAP databases and implementations

You can see the different ways HTAP systems separate transactional and analytical workloads in systems such as TiDB, AlloyDB, and Databricks Lakebase. Each takes a different approach to transaction processing, analytical processing, and the data path between them.

TiDB: Separate row and column engines

TiDB illustrates a composed HTAP architecture. In 2020, PingCAP published a VLDB paper describing TiDB as a Raft-based HTAP database, and its decoupled architecture integrates row and column storage for hybrid workloads. Transactional workloads use TiKV, a row-based storage engine, while TiFlash maintains a columnar representation for analytical queries. TiDB's optimizer can route queries to TiKV, TiFlash, or both, with the two paths running on separate nodes for greater workload isolation.

TiFlash receives replicas asynchronously through the Raft log but checks whether it has reached the data version required by a query to provide snapshot isolation. When evaluating this approach, consider how replication lag affects analytical latency and throughput during sustained transactional writes.

AlloyDB: One database with a columnar path

AlloyDB takes a more converged approach. It operates with a transparent in-memory columnar engine that accelerates analytical operations such as scans, joins, and aggregations. The engine can run on the primary instance or on read-pool instances for greater workload isolation. Google states that AlloyDB delivers up to 100x faster analytical queries than standard PostgreSQL.

When evaluating this approach, consider how much of your workload can use the columnar path. Unsupported operations or data unavailable in the columnar engine may fall back to the row-oriented path, while frequent updates can require columnar data to be refreshed. Test columnar coverage and performance using representative query and update patterns.

Lakebase and LTAP: Separate compute over a shared data layer

Lakebase with LTAP takes the separation between transactional and analytical workloads further. Rather than serving both through the same database engine, it keeps their compute independent while connecting them through a shared data layer. Postgres compute can remain optimized for low-latency transactions, while analytical compute processes the same operational data without placing analytical workloads on the primary database.

This separation also allows the two sides to scale independently. Transactional capacity can follow application traffic, while analytical capacity can follow query demand, without requiring one workload to provision resources for the other. For live reads, analytical compute reads the bulk of data from the columnar copy in object storage, fetches only the current log position from Postgres, and merges any very recent changes from the pageserver to provide a transactionally consistent view, without placing analytical workload on the primary Postgres instance.

HTAP in modern data architecture: When is it the right choice?

HTAP has been pitched as the next big DB architecture more than once. In practice, it is a good fit when the value of your analysis depends on access to fresh operational data. Start by asking how much delay your workload can tolerate. If data becomes less useful while moving from a transactional system to a separate analytical system, reducing that delay may justify an HTAP architecture.

That question matters more as emerging workloads such as AI agents and real-time personalization need current state and historical context at the same time. It is also why the conversation has shifted away from tightly coupled warehouses toward modular lakes built on open formats, where transactional and analytical systems can share data without forcing every workload into one engine.

Common use cases include:

Use caseWhy fresh data mattersWhat to evaluate
Real-time business intelligenceYou need dashboards and metrics to reflect recent operational activity.Freshness, analytical concurrency, and transaction latency
Operational analyticsYou use analytical results to inform an active workflow or decision.Consistency, query latency, and workload isolation
AI applications and agentsYour application needs current operational state alongside historical context to make decisions or take actions.Serving latency, freshness, consistency, and transaction impact
Feature servingYou need fresh features alongside historical features for predictions or personalization.Serving latency, freshness, consistency, and update frequency
Fraud detectionYou need to combine new events with recent and historical activity before making a decision.End-to-end decision latency, freshness, and availability

Choose the architecture based on how much freshness and workload isolation you need. Converged HTAP can work well when transactional and analytical workloads can safely share an operating environment. Composed HTAP provides stronger compute isolation when synchronization can still keep analytical data current enough for your workload.

When to keep OLTP and OLAP separate

Keep OLTP and OLAP separate when your analytical workloads can tolerate some delay. Historical analysis, model training, advanced analytics, and regulatory reporting can often remain on a dedicated OLAP database or in cloud data warehouses, especially when you need to protect transactional latency and availability through stronger workload isolation.

Consider your existing architecture as well. If your CDC or ETL data processing already meets your freshness, performance, reliability, and cost requirements, moving to HTAP may introduce migration risk and operational complexity without enough benefit. HTAP is most valuable when reducing data delay solves a specific application or business problem.

HTAP migration and adoption playbook

Once you have validated that an HTAP architecture can meet your workload requirements, adopt it incrementally. Set clear success criteria at each stage and make sure you can return to the existing path if those criteria are not met.

A practical migration sequence is:

  1. Map your workloads and dependencies: Identify the applications, queries, pipelines, and downstream consumers involved. Separate transactional and analytical workloads from those that genuinely need both.
  2. Set your guardrails: Define acceptable thresholds for transaction latency, analytical performance, freshness, correctness, availability, and cost. Decide in advance what would cause you to pause or roll back.
  3. Start with analytical reads: Where possible, run the existing and new analytical paths in parallel. Compare results, freshness, and performance before the new path affects application behavior.
  4. Shift traffic gradually: Move suitable workloads in stages while monitoring the new path and its effect on transactional performance.
  5. Test failure and recovery: Verify what happens when synchronization falls behind, components fail, or traffic returns to the previous path. Test these procedures before you depend on them in production.
  6. Migrate writes only if needed: If adoption changes your system of record, first validate durability, schema evolution, downstream dependencies, and rollback behavior.

Use the migration to review your schemas, indexes, and access patterns. HTAP can change how you store and query data, but it cannot compensate for a data model that does not fit your workload.

Wrapping up

HTAP can reduce the gap between transactions and analytics, but the right architecture depends on your workload. Start with the transaction latency, analytical performance, freshness, consistency, and isolation your application requires, then test those requirements together under representative mixed load.

If transactional and analytical compute need to scale independently while remaining connected through a shared data layer, Databricks Lakebase provides a Postgres-compatible transactional foundation for the lakehouse and the data lake beneath it. Explore Lakebase to see how Databricks approaches LTAP and evaluate where it meets your workload requirements.

Frequently asked questions

What is HTAP?

HTAP stands for Hybrid Transactional/Analytical Processing. It is a database architecture that brings online transaction processing (OLTP) and online analytical processing (OLAP) together, allowing applications to analyze operational data while it is still current.

What is an HTAP database?

An HTAP database supports both transactional and analytical workloads. Converged systems handle both through a single database engine, while composed systems use separate transactional and analytical engines connected through shared storage, replication, or other synchronization mechanisms.

What is HTAP in databases?

In databases, HTAP reduces the traditional separation between operational and analytical systems. It gives analytical queries access to fresher transactional data without relying entirely on moving that data through a separate batch ETL pipeline first.

What is the difference between OLTP and OLAP?

OLTP databases process high volumes of small transactions using row-oriented storage and low-latency access. OLAP databases run complex queries over large datasets using columnar storage, accepting longer latency in exchange for efficient scans and aggregations.

What is LTAP?

LTAP stands for Lake Transactional/Analytical Processing. It serves both transactional and analytical workloads from a unified storage layer in the lake, under one governance model, removing the CDC and replication pipelines teams traditionally maintain to keep separate systems in sync. This allows each workload to optimize and scale independently while operating on a common data foundation.

来源:Databricks:Blog · databricks.com