In this blog, we explore how to stream data from YugabyteDB’s Change Data Capture feature to Snowflake using SnowflakeSinkConnector on Confluent cloud via Kafka connect.
In this blog, we’ll discover how to publish data to a Kafka topic and then read those messages to another sink, in this case a PostgreSQL database.
Kubernetes has become widely adopted in the Fortune 500. Many companies are now using the platform to run stateless and stateful applications on-premises or as hybrid cloud deployments in production. Of course, with any new technology, there are growing pains when running resilient Kubernetes workloads. But most executives and developers agree that the benefits far outweigh the challenges.
Data on the Kubernetes ecosystem is evolving rapidly with the rise of stateful applications. However,
Log aggregation is an integral part of a distributed system. As the name suggests, a distributed system will have multiple processes across multiple machines, and each process will generate a lot of data. Looking at the data in silos is time-consuming and wouldn’t yield important information as the data sets still need to be correlated. But aggregating the logs is a huge productivity booster that helps to transform the raw log data into insightful information.
When building modern web applications, developers often find data modeling and data access to be productivity bottlenecks. Rather than moving towards a schema-less database solution, many find using an ORM (object-relational mapping) tool with SQL to be their preferred option. The Node.js community has long been supportive of the Sequelize ORM, with Prisma being a newer option for those looking to model, migrate, and query their data.
In this blog, we’ll get acquainted with Prisma and how it interfaces with Node.js and YugabyteDB.
Change Data Capture (CDC) is a technique to capture changes in a source database system in real-time. The goal is to stream those changes as events through a data processing pipeline for further processing.
CDC enables many use cases, especially in modern microservices-based architecture that involves a lot of bounded services. It is the de-facto choice for use cases such as search indexes, in-memory data cache, real-time notifications, data sync between sources,
In this blog post, we will stream data to downstream databases leveraging YugabyteDB’s Change Data Capture (CDC) feature introduced in YugabyteDB 2.13. This will publish the changes to Kafka and then stream those changes to databases like MySQL, PostgreSQL, and Elasticsearch.