Organizations in numerous industries are increasingly interested in and are attempting to build, real-time systems that far exceed the limited capabilities of the software systems of the recent past. The issues that these systems need to address impact internal operations and customer experiences, and also extend beyond the walls of the individual organization to change the expected capabilities of the industry, and even the health of the planet.
The next generation of real-time systems come into play in extremely diverse uses:
- Safety and security: Delivering new levels of health and public safety in smart buildings that automatically detect and stop the spread of disease,
- Retail: Enabling new personalized proximity marketing experiences in physical retail environments,
- Emergencies: Detecting floods and other emergency situations and then automatically triggering evacuation protocols
In all these scenarios – there can be no compromise in terms of responsiveness, reliability, and scalability. This demands that those in charge of development embrace a more modern way of thinking about the way these high-performance real-time systems should be architected.
When the Database First Way of Creating Real-Time Systems Fails — How Many Elevators Can You Actually Monitor — Before It Breaks?
A modern super-city might have buildings with hundreds of thousands of elevators – all of which require constant monitoring to detect situations of interest such as security and safety concerns. The best way to address this kind of ‘smart building’ challenge is through real-time stream processing that can handle data analytics at scale and deliver consistent and timely situational awareness.
Development would likely start with information from a single elevator with analysis done in a simple time-series database and small batch queries. But it would be incorrect to assume that what works for one should work for hundreds and then thousands.
The flaw in this assumption is that the database queries will be able to handle the explosion of data without massive loss in performance speed. This approach works as expected with a small number of elevators, but the whole system fails when the amount of data (elevators) grows beyond the capabilities of the database
Regardless of placing other real-time capabilities around the periphery of the traditional database in this system, the use of a database itself is what inherently breaks the system at scale.
The solution to creating a robust scalable system is to perform the analytics of anomaly detection in memory, and then move information to the database for historical purposes. The database is the last step, not the first, in a modern real-time system.
The Three Kinds of Real-Time Systems
While there is growing interest in real-time systems, there is accompanying noise, confusion, and misinformation about the different kinds and capabilities of real-time systems, as well as the relevance (or not) of databases to their ability to scale and perform as required. There are three types of real-time systems, each of which is relevant for solving a different class of problems.
- ‘Hard’ Real-Time Systems – hardware-based,
- Micro-Batch Real-Time Systems – ‘soft’ real-time systems that use more traditional data processes and queries,
- Event-Driven Real-Time Systems – ‘soft’ real-time systems that use stream or event processing.
1. “Hard’ Real-Time Systems
These types of systems are needed to solve problems that cannot tolerate any missing ‘deadlines,’ with performance speed measured in a few milliseconds. No database could ever deliver on this kind of performance, and in addition, all hardware equipment and computing need to be done on-premise. High precision automated robotic assembly lines require the rigor of this type of real-time system.
2. Micro-Batch Real-Time Systems
This approach to real-time systems is most appropriate for problems that only require some real-time processing with latencies in the hundreds of milliseconds (or even seconds,) and that require little scaling. eCommerce ordering systems can be a good match for this.
Traditional approaches to data processing are performed against small amounts of data (micro-batches) at a fast ‘duty cycle.’ Ground zero for creating fatal problems is found in attempts to scale the system and diminish the latency between batches to make these systems function, similar to event-driven real-time systems.
As the number of batches increases linearly, the compute overhead and cost to continually run the queries in the growing volume of micro-batches increases exponentially (up to the square of the database size.) At some point, the law of physics kicks in, and it becomes impossible to make the data analysis layer of the system perform in the defined ‘real-time’ at high volume. Ultimately, a database will never be as fast as event processing.
3. Event-Driven Real-Time Systems
This is the ‘goldilocks’ solution for applications that require action within a very short time period in the 1-10 millisecond range. A recommendation system is an appropriate use of this kind of real-time system – such as in eCommerce or in industrial automation.
In-memory processing, not a database, is the driving force in this system. Information (from IoT sensors, embedded AI, event brokers, etc.) is processed in flight using stream analytics, and it can then be sent to a database for historical purposes.
As the amount of data increases — the compute work scales linearly — not exponentially – as in the case of the micro-batch approach.
Finding and Avoiding the Performance and Scale Choke Points in Real-Time Systems
The analysis of the three types of real-time systems shows us that systems that use a traditional database storage model can never be scalable in real-time, even if the ingestion was real-time.
It takes time to perform queries, and query performance degrades as a database grows – which is exactly what happens when you try to scale a system. In our earlier elevator example, ingestion was real-time, but accessing and performing queries on the information stored in the database was NOT real-time.
The performance of that system was ultimately gated by the worst performing part of the entire system – the database.
In designing the next generation of real-time systems, it is essential to consider the time frame in which different information must be accessed and understood and the scale to which you ultimately want to grow your system.
It’s Not an Either-Or Decision — Next-Generation Real-Time Systems Will Need to Be Hybrid
There is not a one-size-fits-all approach to real-time systems. But it’s always important to start with understanding which information needs to be stored over longer periods of time in a database for historical reporting, deeper analytics, and pattern recognition.
Next, as opposed to the information that requires immediate action (on the order of milliseconds) for real-time event processing. The best systems will be those that combine the different models of data processing to take advantage of the benefits each offer.
Featured Image Credit: Natã Romualdo; Pexels; Thank you!
