Thoughts on distributed databases, open source, and cloud native
Time for another update from the engineering team at YugaByte! It has been a month since the last update, so let’s dive right in.
On the conference front, YugaByte was at Google Next towards the end of July. YugabyteDB was already very well integrated into the Google Cloud Compute ecosystem, we additionally announced support for Google Container Engine (GKE).
…
As we reviewed in “Docker, Kubernetes and the Rise of Cloud Native Databases”, Kubernetes has benefited from rapid adoption to become the de-facto choice for container orchestration. This has happened in a short span of only 4 years since Google open sourced the project in 2014. YugabyteDB’s automated sharding and strongly consistent replication architecture lends itself extremely well to containerized deployments powered by Kubernetes orchestration. In this post we’ll look at the various components involved in getting YugabyteDB up and running as Kubernetes StatefulSets.
…
For ever-growing data workloads such as time series metrics and IoT sensor events, running a highly dense database cluster where each node stores terabytes of data makes perfect sense from a cost efficiency standpoint. If we are spinning up new data nodes only to get more storage-per-node, then there is a significant wastage of expensive compute resources. However, running multi-terabyte data nodes with Apache Cassandra as well as other Cassandra-compatible databases (such as DataStax Enterprise) is not an option.
…
Pivotal Container Service (PKS) simplifies the deployment and management of enterprise-grade Kubernetes on both public and private clouds. The PKS APIs and its command line interface make it easy to create and manage Kubernetes clusters without cloud-vendor lock in.
Getting started with PKS is easy
…
Apache Cassandra is a distributed open source database that can be referred to as a “NoSQL database” or a “wide column store.” Cassandra was originally developed at Facebook to power its “Inbox” feature and was released as an open source project in 2008. Cassandra is designed to handle “big data” workloads by distributing data, reads and writes (eventually) across multiple nodes with no single point of failure.
…
Editor’s note: This post was originally published August 8, 2018 and has been updated as of May 28, 2020.
As we saw in ”How Does Consensus-Based Replication Work in Distributed Databases?”, Raft has become the consensus replication algorithm of choice when it comes to building resilient, strongly consistent systems. YugabyteDB uses Raft for both leader election and data replication. Instead of having a single Raft group for the entire dataset in the cluster,
…
In case you missed the news earlier this Summer, YugaByte raised an additional $16M of funding from Dell Technologies Capital and our previous investor Lightspeed Venture Partners. With the additional funding, we are accelerating investments in engineering, sales, and customer success to scale our support for enterprises building business-critical applications in the cloud. So, as you’d expect…
…
Editor’s note: This post was originally published August 2, 2018 and has been updated as of May 26, 2020.
Whether it be a WordPress website’s MySQL backend or Dropbox’s multi-exabyte storage system, data replication is at the heart of making data durable and available in the presence of hardware failures such as machine crashes, disk failures, network partitions, and clock skews. The basic idea behind replication is very simple: keep multiple copies of data on physically isolated hardware so that one hardware failure does not impact the others;
…
When creating a Cassandra-compatible YCQL table in the YugabyteDB database, you are required to create a primary key consisting of one or more columns of the table. Primary key based retrievals are efficient because YugabyteDB automatically indexes/organizes the data by the primary key. However, there are many use cases where you may need to retrieve data using columns that are not a part of the primary key. This is where secondary indexes help.
…
After billions of dollars in capital expenditure and reference customers in every major vertical, Google Cloud Platform has finally emerged as a credible competitor to Amazon Web Services and Microsoft Azure when it comes to enterprise-ready cloud infrastructure. While Google Cloud’s compute and storage offerings are easier to understand, making sense of its various managed database offerings is not for the faint-hearted. This post introduces app developers to the major Google Cloud database services,
…