Enhancing Cloud Scalability with AI-Driven Resource Management

Amit Choudhury; Yuvaraj Madheswaran

doi:10.55524/ijirem.2024.11.5.5

Abstract

This research paper aims at analyzing the factors that can help improve scalability of cloud by incorporating different machine learning algorithms in management of resources. Since controlling and managing cloud resources is becoming more challenging with compounded base requirements, the majority of conventional resource management solutions may not prove adequate. This research assesses the performance of five state-of-art machine learning techniques namely Reinforcement Learning, Long Short-Term Memory, Gradient Boosting Machines, Autoencoders and Neural Architecture Search in minimizing operational cost and enhancing resource utilization and overall system efficiency for improving business outcomes. The findings reveal that the use of RL-based approaches to optimize operational cost reduction and minimizing provisioning delay by 20% and 30% respectively and LSTM network to increase the accuracy of demand forecasting by 12% and overall efficiency of resource utilization by 22%. The use of GBM models in forecasts results in 30% error reduction in costs that drop by 20% while service improves by 25%. Using autoencoders, the models achieve 97% accuracy in detecting anomalies and infinityai1411@gmail.com in turn increasing the efficiency of allocation by 15 percent. The NAS-optimized models yield increased accuracy by a percentage point of 18 % as well as a 25% faster computational speed. Altogether, these theoretical developments demonstrate the ability of AI-based methodologies to enhance the cloud scalability promising and provide practical recommendations for improving resource management approaches in the cloud environment.

Keywords

Cloud scalability, AI-driven Resource Management, Machine Learning Algorithms, Reinforcement Learning, Deep Q-Learning, Long Short-Term Memory Networks, LSTM Forecasting, Gradient Boosting Machines, XGBoost

References

[1] S. Kanungo, "AI-driven resource management strategies for cloud computing systems, services, and applications," World Journal of Advanced Engineering Technology and Sciences, vol. 11, no. 2, pp. 559-566, 2024. Available From : https://doi.org/10.30574/wjaets.2024.11.2.0137

[2] S. Iqbal and A. Heng, "AI-driven resource management in cloud computing: Leveraging machine learning, IoT devices, and edge-to-cloud intelligence," 2023. Available From : http://dx.doi.org/10.13140/RG.2.2.28383.27049

[3] Q. Liang, W. A. Hanafy, A. Ali-Eldin, and P. Shenoy, "Model-driven cluster resource management for AI workloads in edge clouds," ACM Transactions on Autonomous and Adaptive Systems, vol. 18, no. 1, pp. 1-26, 2023. Available From : https://doi.org/10.1145/3582080

[4] M. J. Goswami, "Leveraging AI for cost efficiency and optimized cloud resource management," International Journal of New Media Studies: International Peer Reviewed Scholarly Indexed Journal, vol. 7, no. 1, pp. 21-27, 2020. Available From: https://www.researchgate.net/publication/381280852_Leveraging_AI_for_Cost_Efficiency_and_Optimized_Cloud_Resource_Management

[5] R. K. Navandar, "Enhancing cloud computing environments with AI-driven resource allocation models," Advances in Nonlinear Variational Inequalities, vol. 27, no. 3, pp. 541-557, 2024. Available From: https://internationalpubls.com/index.php/anvi/article/view/1418

[6] P. D. A. S. Rao, "Orchestrating efficiency: AI-driven cloud resource optimization for enhanced performance and cost reduction," International Journal of Research Publication and Reviews, 2023. Available From: https://www.semanticscholar.org/paper/Orchestrating-Efficiency%3A-AI-Driven-Cloud-Resource-Rao/5780cec20018cdd99dc713febcd1f43938b9b9a3

[7] A. Boudi, M. Bagaa, P. Pöyhönen, T. Taleb, and H. Flinck, "AI-based resource management in beyond 5G cloud native environment," IEEE Network, vol. 35, no. 2, pp. 128-135, 2021. Available From: https://doi.org/10.1109/MNET.011.2000392

[8] G. K. Walia, M. Kumar, and S. S. Gill, "AI-empowered fog/edge resource management for IoT applications: A comprehensive review, research challenges and future perspectives," IEEE Communications Surveys & Tutorials, 2023. Available From: https://doi.org/10.1109/COMST.2023.3338015

[9] C. Seo, D. Yoo, and Y. Lee, "Empowering sustainable industrial and service systems through AI-enhanced cloud resource optimization," Sustainability, vol. 16, no. 12, p. 5095, 2024. Available From: https://doi.org/10.3390/su16125095

[10] M. Abouelyazid and C. Xiang, "Architectures for AI integration in next-generation cloud infrastructure, development, security, and management," International Journal of Information and Cybersecurity, vol. 3, no. 1, pp. 1-19, 2019. Available From : https://publications.dlpress.org/index.php/ijic/article/vi

[11] I. Horrocks, "Transforming IoT security: Harnessing AI and cloud systems for optimal resource management," 2023.

[12] S. Priyadarshini, T. N. Sawant, G. B. Yadav, J. Premalatha, and S. R. Pawar, "Enhancing security and scalability by AI/ML workload optimization in the cloud," Cluster Computing, pp. 1-15, 2024. Available From: https://doi.org/10.1007/s10586-024-04641-x

[13] B. Kumar, "Challenges and solutions for integrating AI with multi-cloud architectures," International Journal of Multidisciplinary Innovation and Research Methodology, vol. 1, no. 1, pp. 71-77, 2022. Available From: https://ijmirm.com/index.php/ijmirm/article/view/76

[14] U. M. R. Inkollu and J. K. R. Sastry, "AI-driven reinforced optimal cloud resource allocation (ROCRA) for high-speed satellite imagery data processing," Earth Science Informatics, vol. 17, no. 2, pp. 1609-1624, 2024. Available From : https://doi.org/10.1007/s12145-024-01242-5

[15] K. Lin, Y. Li, Q. Zhang, and G. Fortino, "AI-driven collaborative resource allocation for task execution in 6G-enabled massive IoT," IEEE Internet of Things Journal, vol. 8, no. 7, pp. 5264-5273, 2021. Available From : https://doi.org/10.1109/JIOT.2021.3051031

[16] J. Sekar and L. L. C. Aquilanz, "Autonomous cloud management using AI: Techniques for self-healing and self-optimization," Journal of Emerging Technologies and Innovative Research, vol. 11, pp. 571-580, 2023. Available From : https://www.researchgate.net/publication/382205673_AUTONOMOUS_CLOUD_MANAGEMENT_USING_AI_TECHNIQUES_FOR_SELF-_HEALING_AND_SELF-OPTIMIZATION

Cites this article as

A. Choudhury, Y. Madheswaran, "Enhancing Cloud Scalability with AI-Driven Resource Management", International Journal of Innovative Research in Engineering & Management (IJIREM), Vol-11, Issue-5, Page No-32-39, 2024. Available from: https://doi.org/10.55524/ijirem.2024.11.5.5

Corresponding Author

Amit Choudhury

Department of Information Technology, Dronacharya College of Engineering, Gurgaon, Department of Information Technology, Dronacharya College of Engineering, Gurgaon, Department of Information Technology, Dronacharya College of Engineering, Gurgaon, Gurugram, India