Unlocking the Power of Event Streaming with YugabyteDB: Real-World Use Cases
October 5, 2023
The value of real-time event streaming is undeniable. Whether it’s vehicle telemetry, consumer WiFi management, or financial market data, companies are leveraging streaming data to make decisions quicker, take advantage of market opportunities faster, and drive revenue growth. So let’s dive into how three companies—General Motors, Plume, and Xignite—are transforming and realizing unprecedented benefits through the use of YugabyteDB as their storage for their event processing systems.
General Motors: Driving Innovation through Vehicle Telemetry
General Motors (GM), with its rapidly expanding fleet of over 20 million vehicles, is a data powerhouse, generating staggering volumes of telemetry data. This data not only serves other business units but also drives significant revenue for GM through both internal and external partnerships. The scale of their system is immense, handling up to 3 million writes/second and storing over 70 terabytes of data in hot storage before offloading to cold storage.
Initially, they were using Apache Cassandra and faced issues with performance and Day 2 operations like upgrades and scaling. By switching to YugabyteDB, they can now easily scale during peak times and can scale ten times faster. What used to take a week now takes a day. Performance has also improved over 10x. The data density advantage of YugabyteDB also enabled GM to reduce its hardware footprint, leading to significant cost savings that outweighed its licensing costs.
Architecture Insights
GM constructed its system, known as Vehicle Data Factor (VDF), with four main pillars.
- Messaging Tier: Using Apache Kafka as a messaging bus, this tier ingests telemetry data from their connected vehicles, including critical data used for their 911 emergency service systems.
- Transformation Tier: Built on Apache Spark Streaming, this layer transforms ingested data through various stages before storing it in YugabyteDB, which serves as the system of record. This tier also handles data aggregations and calculations.
- Hot Storage: The data is stored temporarily in YugabyteDB for 90 days before being offloaded to cold storage in Hadoop.
- Edge Applications: Since it holds the most recent hot data, YugabyteDB serves as the system of record for GM’s mobile applications* and various other business units that deliver services to internal and external partners.
*For instance, if you have GM’s mobile application and you want to lock your doors, that will generate an event that is then streamed back to the vehicles. So it’s not just vehicle data that’s driving these events—it’s also interactions with their customers.
Plume: Managing Smart Homes with Data-Driven Intelligence
Plume enables ISPs to offer various smart home services to their customers. The Plume platform is cloud-based and features a fully integrated suite of products including hardware, analytics, integrations, services, and mobile tools.
The Plume management plane serves connected homes and the WiFi/smart devices found therein, handling billions of operations per day across a data set of around 35 terabytes (and growing).
Initially using MongoDB and Cassandra, Plume struggled to scale and meet data sovereignty requirements. After adopting YugabyteDB, they successfully achieved their business goals by geo-partitioning their data to keep it in the locality that it needed to be in.
Xignite: Powering the Financial Industry with Real-Time Market Data
Xignite handles real-time stock ticker and financial market data as a backend for major U.S. financial companies. They had stringent Service Level Objectives (SLOs) and Service Level Agreements (SLAs) as well as requirements for very low latency and high transaction volumes.
They initially deployed their services on AWS RDS and Redis as their database solution for streaming data, but they were struggling to meet their performance and Day 2 operational requirements. They even tried to implement some caching solutions to help offset some of the performance issues, but the results were still not acceptable. By switching to YugabyteDB, Xignite achieved a 50% cost savings compared to AWS RDS and met all their operational and performance targets.
The Way Forward
Companies looking to extract more value from real-time events should consider database solutions that can offer massive write scaling, high write throughput, strong consistency, and high availability.
YugabyteDB was designed to meet these very challenges, providing scale, throughput, and geo-distribution natively. As the examples of GM, Plume, and Xignite demonstrate, the platform can handle varied and complex use cases, making it an ideal choice for any organization looking to leverage the full potential of event streaming data.
