The Distributed SQL Blog

Thoughts on distributed databases, open source, and cloud native

August 14, 2018

Benchmarking an 18 Terabyte YugabyteDB Cluster with High Density Data Nodes

For ever-growing data workloads such as time series metrics and IoT sensor events, running a highly dense database cluster where each node stores terabytes of data makes perfect sense from a cost efficiency standpoint. If we are spinning up new data nodes only to get more storage-per-node, then there is a significant wastage of expensive compute resources. However, running multi-terabyte data nodes with Apache Cassandra as well as other Cassandra-compatible databases (such as DataStax Enterprise) is not an option.

…

Getting Started With YugabyteDB on Pivotal Container Service (PKS)

August 10, 2018

By Karthik Ranganathan

Kubernetes Pivotal Container Service (PKS)

Getting Started With YugabyteDB on Pivotal Container Service (PKS)

Pivotal Container Service (PKS) simplifies the deployment and management of enterprise-grade Kubernetes on both public and private clouds. The PKS APIs and its command line interface make it easy to create and manage Kubernetes clusters without cloud-vendor lock in.

Getting started with PKS is easy

…

Apache Cassandra Architecture Fundamentals

August 9, 2018

By Jimmy Guerrero

Apache Cassandra How It Works

Apache Cassandra Architecture Fundamentals

What is Apache Cassandra?

Apache Cassandra is a distributed open source database that can be referred to as a “NoSQL database” or a “wide column store.” Cassandra was originally developed at Facebook to power its “Inbox” feature and was released as an open source project in 2008. Cassandra is designed to handle “big data” workloads by distributing data, reads and writes (eventually) across multiple nodes with no single point of failure.

…

How Does the Raft Consensus-Based Replication Protocol Work in YugabyteDB?

August 8, 2018

By Sid Choudhury

ACID Transactions Distributed SQL Google Spanner How It Works

How Does the Raft Consensus-Based Replication Protocol Work in YugabyteDB?

Editor’s note: This post was originally published August 8, 2018 and has been updated as of May 28, 2020.

As we saw in ”How Does Consensus-Based Replication Work in Distributed Databases?”, Raft has become the consensus replication algorithm of choice when it comes to building resilient, strongly consistent systems. YugabyteDB uses Raft for both leader election and data replication. Instead of having a single Raft group for the entire dataset in the cluster,

…

YugaByte Company and Database Update – Aug 3, 2018

August 3, 2018

By Kannan Muthukkaruppan

Company News Google Cloud Platform Performance Benchmarks

YugaByte Company and Database Update – Aug 3, 2018

$16 Million Funding Round

In case you missed the news earlier this Summer, YugaByte raised an additional $16M of funding from Dell Technologies Capital and our previous investor Lightspeed Venture Partners. With the additional funding, we are accelerating investments in engineering, sales, and customer success to scale our support for enterprises building business-critical applications in the cloud. So, as you’d expect…

We are Hiring!

…

How Does Consensus-Based Replication Work in Distributed Databases?

August 2, 2018

By Sid Choudhury

ACID Transactions Databases Distributed SQL How It Works

How Does Consensus-Based Replication Work in Distributed Databases?

Editor’s note: This post was originally published August 2, 2018 and has been updated as of May 26, 2020.

Whether it be a WordPress website’s MySQL backend or Dropbox’s multi-exabyte storage system, data replication is at the heart of making data durable and available in the presence of hardware failures such as machine crashes, disk failures, network partitions, and clock skews. The basic idea behind replication is very simple: keep multiple copies of data on physically isolated hardware so that one hardware failure does not impact the others;

…

A Quick Guide to Secondary Indexes in YugabyteDB

August 1, 2018

By Karthik Ranganathan

How It Works How To

A Quick Guide to Secondary Indexes in YugabyteDB

When creating a Cassandra-compatible YCQL table in the YugabyteDB database, you are required to create a primary key consisting of one or more columns of the table. Primary key based retrievals are efficient because YugabyteDB automatically indexes/organizes the data by the primary key. However, there are many use cases where you may need to retrieve data using columns that are not a part of the primary key. This is where secondary indexes help.

…

New to Google Cloud Databases? 5 Areas of Confusion That You Better Be Aware of

July 25, 2018

By Sid Choudhury

Google Cloud Platform Google Spanner

New to Google Cloud Databases? 5 Areas of Confusion That You Better Be Aware of

After billions of dollars in capital expenditure and reference customers in every major vertical, Google Cloud Platform has finally emerged as a credible competitor to Amazon Web Services and Microsoft Azure when it comes to enterprise-ready cloud infrastructure. While Google Cloud’s compute and storage offerings are easier to understand, making sense of its various managed database offerings is not for the faint-hearted. This post introduces app developers to the major Google Cloud database services,

…

Implementing Distributed Transactions the Google Way: Percolator vs. Spanner

July 23, 2018

By Sid Choudhury

ACID Transactions CockroachDB Distributed SQL Google Cloud Platform Google Spanner

Implementing Distributed Transactions the Google Way: Percolator vs. Spanner

Our post 6 Signs You Might be Misunderstanding ACID Transactions in Distributed Databases describes the key challenges involved in building high performance distributed transactions. Multiple open source ACID-compliant distributed databases have started building such transactions by taking inspiration from research papers published by Google. In this post, we dive deeper into Percolator and Spanner, the two Google systems behind those papers, as well as the open source databases they have inspired.

…

July 22, 2018

By Sid Choudhury

ACID Transactions How It Works

6 Signs You Might be Misunderstanding ACID Transactions in Distributed Databases

As described in A Primer on ACID Transactions, first generation NoSQL databases dropped ACID guarantees with the rationale that such guarantees are needed only by old school enterprises running monolithic, relational applications in a single private datacenter. And the premise was that modern distributed apps should instead focus on linear database scalability along with low latency, mostly-accurate, single-key-only operations on shared-nothing storage (e.g. those provided by the public clouds).

Application developers who blindly accept the above reasoning are not serving their organizations well.

…

The Distributed SQL Blog

Learn More to Accelerate Your Retail Business