Apache Airflow DAGs for Migrating PostgreSQL Data to YugabyteDB
In this blog, we’ll build an Apache Airflow workflow that will migrate data between PostgreSQL and YugabyteDB.
In this blog, we’ll build an Apache Airflow workflow that will migrate data between PostgreSQL and YugabyteDB.
Choosing YugabyteDB for Apache Airflow deployments streamlines database management and enhances system reliability. Its high availability and fault tolerance features negate the need for complex failover strategies associated with PostgreSQ. In this blog you will lean how to configure Airflow to use YugabyteDB.
Microsoft’s Azure Kubernetes Service (AKS) offers a highly available, secure, and fully managed Kubernetes service for developers looking to host their applications on containers in the cloud. AKS features elastic provisioning, an integrated developer experience for rapid application development, enterprise security features, and the most available regions of any cloud provider.
YugabyteDB is a natural fit for AKS because it was designed to support cloud native environments since its initial design.
…
The glory days of the heavy-weight hypervisor are slowly fading away, and in the last few years, containerization of applications and services is the new reality. With containerization, enterprises can prototype, deploy, and meet scale demands more quickly. To systematically and efficiently manage these large-scale deployments, enterprises have bet on technologies like Kubernetes (aka k8s), a powerful container orchestrator, to get the job done. Kubernetes was originally developed by Google, but it has been open sourced since 2014 and is today developed by a large community of contributors.
…
As we reviewed in “Docker, Kubernetes and the Rise of Cloud Native Databases”, Kubernetes has benefited from rapid adoption to become the de-facto choice for container orchestration. This has happened in a short span of only 4 years since Google open sourced the project in 2014. YugabyteDB’s automated sharding and strongly consistent replication architecture lends itself extremely well to containerized deployments powered by Kubernetes orchestration. In this post we’ll look at the various components involved in getting YugabyteDB up and running as Kubernetes StatefulSets.
…
Results from the 2018 Kubernetes Application Usage Survey should put to rest concerns enterprise users have had around the viability of Docker containers and Kubernetes orchestration for running stateful services such as databases and message queues. Its exciting to see that nearly 40% of respondents are running databases (SQL and/or NoSQL) using Kubernetes. This number will continue to grow in the months ahead.
…
Kubernetes, the open source container orchestration engine that originated from Google’s Borg project, has seen some of the most explosive growth ever recorded in an open source project. The complete software development lifecycle involving stateless apps can now be executed in a more consistent, efficient and resilient manner than ever before. However, the same is not true for stateful apps — containers are inherently stateless and Kubernetes did not do anything special in the initial days to change that.
…