A Novel Architecture for Scaling Distributed Data Systems: Analysis of High-Throughput, Low-Latency Transaction Processing
Keywords:
distributed systems, stream processing, real-time analytics, fault tolerance, scalability, consensus algorithms.Abstract
Scaling distributed data systems to manage millions of transactions per second while ensuring sub-millisecond latency presents significant challenges in consistency, availability, and real-time analytics. This study explores the architectural innovations required to meet these demands,particularly focusing on multi-region active-active replication, event-driven data ingestion with Kafka, and high-performance analytics using Druid.
References
Bernstein, P. A., & Newcomer, E. (2009). Principles of transaction processing for the systems professional. Morgan Kaufmann Publishers.
Brewer, E. A. (2000). Towards robust distributed systems. ACM SIGACT News, 31(2), 7-10. https://doi.org/10.1145/351180.351189
Cao, X., Zhang, Y., & Guo, H. (2019). Scalable and consistent distributed data systems: A comprehensive survey. Journal of Computer Science and Technology, 34(4), 755-774. https://doi.org/10.1007/s11390-019-1914-4