YugabyteDB Community Update, Tricks and Tips – Dec 13, 2019
Welcome to this week’s community update where we recap a few interesting questions that have popped up in the last week or so on the YugabyteDB Slack channel, Forum, GitHub or Stackoverflow. We’ll also review upcoming events, new blogs and documentation. Ok, let’s dive right in:
How best to configure clusters across deployment types
Ava over on StackOverflow asked how to best setup configurations for different deployment models like single AZ, multi-AZ, multi-region and multi-cloud.
For documentation on configuring clusters on GCP, Azure and AWS, check out: “Deploying YugabyteDB on Public Clouds.” The documentation includes a discussion about prerequisites, plus how to configure data drives, file systems, networking, YugabyteDB specific configuration files and more.
On multi-AZ, multi-region and multi-cloud deployments we offer a variety of geo distribution configurations for tservers and masters to help you best optimize your deployment given your requirements. A simple optimization on these clusters is to put all leaders in the same availability zone so write transactions don’t have to travel across multiple zones. You can do this by specifying
set_preferred_zones via the yb-admin tool.
Wait, what’s a tserver?
The YB-TServer (short for Yugabyte Tablet Server) is the process that does the actual IO for end user requests. Data for a table is split/sharded into tablets. Each tablet is composed of one or more tablet-peers depending on the replication factor. And each YB-TServer hosts one or more tablet-peers.
What’s a master server?
The YB-Master is the keeper of system meta-data/records, such as what tables exist in the system, where their tablets live, what users/roles exist, the permissions associated with them, and so on. It is also responsible for coordinating background operations (such as load-balancing or initiating re-replication of under-replicated data) and performing a variety of administrative operations such as create/alter/drop of a table. Note: that the YB-Master is highly available as it forms a Raft group with its peers, and it is not in the critical path of IO against user tables.
How encryption works in YugabyteDB
James over on StackOverflow asks how encryption at rest works in YugabyteDB and if it can be enabled on the fly.
Yes, encryption at rest can be enabled after the fact on an existing YugabyteDB cluster. New incoming data will start getting encrypted right away. Old data will get encrypted lazily – when scheduled compactions occur. As a general rule, for write heavy workloads, you can expect the overhead to be under 5%. For read heavy workloads, especially those where the hot data set is in cache, the impact should be negligible.
The yb-tserver(s), if restarted, will learn of the current key from yb-master(s) using RPC calls. To protect this exchange from an attack, you can use the on-the-wire encryption feature, which makes sure that all server-to-server communication happens securely. If a yb-master is restarted, it’ll learn of the current key from other yb-masters. However, if all the masters go down at once, none of the masters will have this in memory state, and you will have to use yb-admin tool to give the key to all the masters.
YSQL & YCQL API compatibility
Ali over on stackoverflow asks how compatible are YSQL (fully-relational) & YCQL (semi-relational) with each other.
Today, the YugabyteDB APIs are independent from one another. This means that the data inserted or managed by one API cannot be queried by the other API. Additionally, there is no common way to access the data across the APIs (external frameworks such as Presto can help for simple cases.) In the future, we will allow YCQL tables to be “accessed” from YSQL (which is fully PostgreSQL compatible) as a foreign table using foreign data wrappers (FDW).
How encryption works in an asynchronous Two Data Center (2DC) deployment
Sergiy over on Slack asks how encryption works in a 2DC deployment.
Currently, we support encryption in transit (between datacenters) and encryption at rest for data on disk. We can also use encryption in transit to secure communication from client servers. Using this technique you can have clusters in a multi-datacenter asynchronous replication fully secured.
New Documentation, Blogs, Tutorials, and Videos
- YugabyteDb Engineering Update – Dec 9, 2019
- Getting Started with PostgreSQL’s Row-Level Security in YugabyteDB
- Using Stored Procedures in Distributed SQL Databases
- Announcing YugabyteDB on Crossplane, the Open Source Multicloud Control Plane
- What is Distributed SQL?
- How Plume Handled Billions of Operations Per Day Despite an AWS Zone Outage
- Getting Started with PostgreSQL Authentication and Authorization in YugabyteDB
- Getting Started with PostgreSQL’s Row-Level Security in YugabyteDB
- Designing a Change Data Capture and Two Data Center Architecture for a Distributed SQL Database
- Distributed Databases Deconstructed: CockroachDB, TiDB and YugabyteDB
- We have uploaded over a dozen videos from this year’s Distributed SQL Summit. You can links to the presentations, slides and a recap of the highlights by checking out, “2019 Distributed SQL Summit Recap and Highlights.”
- Encryption at rest
- Configuring encryption at rest with the yb-admin tool
- Using PostgreSQL’s JDBC Driver
- YugabyteDB configuration reference documentation for TServer, Master nodes and port assignments
- Two data center (2DC) deployment instructions
Upcoming Meetups and Conferences
- Jan 16: Silicon Valley PostgreSQL Meetup
Distributed SQL Webinars
- Dec 18: Distributed SQL Database Administration 101
- Jan 15: Developing Cloud-Native Spring Microservices with a Distributed SQL Backend
We are Hiring!
YugaByte is growing fast and we’d like you to help us keep the momentum going! Check out our currently open positions:
- Software Engineer – Cloud Infrastructure – Sunnyvale, CA
- Software Engineer – Core Database – Sunnyvale, CA
- Software Engineer – Full Stack – Sunnyvale, CA
- Developer Advocate – Sunnyvale, CA
- Senior Technical Writer – Remote
Our team consists of domain experts from leading software companies such as Facebook, Oracle, Nutanix, Google and LinkedIn. We have come a long way in a short time but we cannot rest on our past accomplishments. We need your ideas and skills to make us better at every function that is necessary to create the next great software company. All while having tons of fun and blazing new trails!
Ready to start exploring YugabyteDB features? Getting up and running locally on your laptop is fast. Visit our quickstart page to get started.
- Compare YugabyteDB in depth to databases like CockroachDB, Google Cloud Spanner and MongoDB.
- Get started with YugabyteDB on macOS, Linux, Docker, and Kubernetes.
- Contact us to learn more about licensing, pricing or to schedule a technical overview.