YugabyteDB 2.20 Unveiled: Elevating Change Data Capture for Cloud Native Applications
December 11, 2023
Introducing YugabyteDB 2.20, the latest release of the modern, distributed database for cloud native applications! This new release enhances our distributed PostgreSQL offering, broadening YugabyteDB’s applicability to a wider range of applications that require advanced data consistency, predictable performance at scale, and the familiarity of PostgreSQL behavior for an easy lift-and-shift.
YugabyteDB 2.20 elevates change data capture (CDC) by ensuring transactional consistency, boosting data integrity, and fueling reliable downstream applications. It also delivers OIDC-based authentication, optimized performance, new wait-on-conflict concurrency controls, enhanced data protection, and several of key, behind-the-scenes improvements.
Keep reading to learn more about the features in our latest stable, production-ready version.
Expand Application Reach with Transactionally Consistent Change Data Capture (CDC)
Event and streaming data are quickly becoming an essential use case for modern database architectures. It is crucial to capture data changes from a system of record (SOR), such as YugabyteDB, and apply them to downstream systems. These systems can range from operational databases like Oracle to business intelligence tools, real-time alerting systems, recommendation engines, supply chain management (SCM) systems, and more.
While many databases support CDC, business-critical applications need more. They need transactional consistency to ensure that the captured data accurately reflects the database’s actual state, even when dealing with concurrent transactions.
To achieve this, the database must identify transactionally consistent changes and be able to determine the Data Manipulation Language (DML) operations that make up a single transaction. Release 2.20 enhances YugabyteDB’s CDC capabilities by introducing transactionally consistent CDC streams.
YugabyteDB’s transactionally consistent CDC ensures users benefit from:
- Accurate representation of data changes
- Reproducible data lineage
- Data integrity in downstream systems
- Compliance with regulations
- Resilience in complex systems
This new feature broadens YugabyteDB’s support to an even wider range of applications and data processing scenarios, including:
- Real-time data analysis
- Business-critical event-driven applications
- Data auditing and compliance
- Master data management (MDM)
- Real-time customer experience management
- Data synchronization across systems
It seamlessly integrates with various downstream systems, including Oracle, Snowflake, and popular databases on AWS, Azure, and Google Cloud.
Accelerate Performance at Scale with Performance Enhancements
YugabyteDB 2.20 delivers a couple of key enhancements that improve performance across a variety of operations. Two major updates to highlight are:
- Catalog metadata caching: Catalog metadata caching significantly reduces the amount of time it takes to execute queries by storing frequently accessed metadata in memory. New Catalog Metadata Caching in the core YugabyteDB database notably boosts YSQL connection scalability and performance with faster warm-up times. Currently in early access, users are welcome to test this feature and provide feedback.
- Faster local bootstrap in self-managed YugabyteDB (YugabyteDB Anywhere): Key enhancements to the self-managed control plane decrease the time needed for local bootstrapping of tablets. Local bootstrapping involves pre-downloading (or pre-fetching) data and metadata to a node before it joins the distributed database cluster. This decreases join times and minimizes risk by restoring the cluster to full health faster if a node needs to go offline for maintenance. When the node goes offline, the overall cluster remains available and applications continue to operate as expected (assuming a replication factor of at least three, RF=3).
Simplify Lift-and-Shift with Wait-on-Conflict Concurrency Control
PostgreSQL users looking to evolve to a modern, horizontally scalable database with built-in resilience often prefer a straightforward lift-and-shift approach for their applications. To facilitate this transition smoothly, the new database must not only support PostgreSQL syntax and features but also deliver the default isolation levels with the same behavior. YugabyteDB 2.20 now makes that need a reality with new wait-on-conflict controls.
Wait-on-conflict concurrency control (WCC) is a concurrency control protocol where transactions wait for other transactions to finish before they proceed. This contrasts with other concurrency control protocols, such as lock-based concurrency control (LBC), which prevent transactions from accessing data locked by other transactions. WCC offers increased concurrency, reduced deadlock, and improved fairness compared to LBC.
YugabyteDB 2.20 also debuts an early access feature of wait-on conflict with pg-locks in its production-ready release train. When combined with Read Committed Isolation, this feature helps minimize or eliminate any application re-architecture for PostgreSQL users who want to lift-and-shift an application to YugabyteDB. It replicates the behavior of Read Committed isolation for standard PostgreSQL, but in a distributed environment.
This new feature is particularly valuable to users who have workloads with many multi-tablet statements and may have multiple sessions contending concurrently for the same locks. It helps prevent deadlock scenarios and ensures fairness so a session is not starved by new sessions. Our Wait-on-Conflict documentation provides additional details and an example scenario.
pg_locks View
In addition to the wait-on-conflict concurrency control, YugabyteDB 2.20 introduces a pg_locks view. This view provides users a tool for determining the reasons behind statement blockages in a WCC environment. To minimize overhead, pg_locks displays locks held or requested by only the longest-running transactions in the cluster, rather than displaying all actively held locks in the cluster.
Specifically, the view will only show locks held or requested by transactions running for longer than the session variable yb_locks_min_txn_age_ms. This variable is configurable and defaults to 1 second.
Enhance Security with OIDC-Based Authentication
OpenID Connect (OIDC) is an authentication layer on top of the OAuth 2.0 protocol. It allows customers to verify user identities and access basic user profile information from an OpenID Provider (OP). OIDC-based authentication works using JSON Web Tokens (JWTs).
YugabyteDB 2.20 delivers OIDC-based authentication for YSQL, enhancing security by building on top of the widely adopted OAuth 2.0 protocol. It streamlines development costs through a standard authentication method and improves user experience with a single sign-on (SSO) capability.
As we continue to develop this feature, our goal is to deliver an end-to-end solution that includes database access using OIDC tokens, setting authentication rules in YugabyteDB config files, creating user-name maps, and providing an audit trail for all related user and admin actions.
Increase Application Uptime with Online Point in Time Recovery (PITR)
First introduced in YugabyteDB 2.18, point-in-time recovery (PITR) in YugabyteDB enables recovery from user or software errors, while minimizing recovery point objective (RPO), recovery time objective (RTO), and the overall impact to the cluster. PITR in YugabyteDB combines flashback capability with periodic distributed snapshots.
Online PITR systems are a critical component of any modern database system, helping users protect their data from various threats and minimizing downtime in the event of an error.
YugabyteDB 2.20 further enhances the PITR capabilities of YugabyteDB Anywhere (our self-managed platform) by supporting online PITR without the need to stop any workloads.
Get Started Today
YugabyteDB 2.20 is available today! All core YugabyteDB features are available for free in the 100% open source version, and the orchestration engine enhancements are available to commercial costumes of our self-managed DBaaS (YugabyteDB Anywhere).
YugabyteDB 2.20 is also under the umbrella of our new long-term support (LTS) release. This means it will receive maintenance updates for at least two years (730 days) from the first release date of the minor release. Visit our release support policy to fully understand the implications for your business.
Visit the following resources for more information:
