AI in Optimizing Big Data Pipelines and Storage Systems
Keywords:
Artificial Intelligence Optimization, Big Data Pipelines, Intelligent Storage Management, Reinforcement Learning Scheduling, Explainable AI GovernanceAbstract
Data center infrastructure faces escalating complexity as global data volumes grow exponentially,rendering rule-based management insufficient for modern enterprise platforms. This article presents the Governance-Aware AI Optimization Framework (GAAOF), an integrated architecture unifying pipeline orchestration, anomaly detection, storage optimization, and explainable governance for enterprise big data systems.
References
David Reinsel et al., "The Digitization of the World From Edge to Core," Seagate, 2018. [Online].
Available: https://www.seagate.com/files/www-content/our-story/trends/files/idc-seagate-dataage
whitepaper.pdf
Michael Armbrust et al., "A View of Cloud Computing," Communications of the ACM, 2010.
[Online]. Available: https://dl.acm.org/doi/10.1145/1721654.1721672
