Articles Open Access
Scalable AI Infrastructure for Next-Generation Cloud-Based Enterprise Automation and Intelligence Solutions
Associate Professor, Department of Computing (CCI), College of Computing and Informatics, Universiti Tenaga Nasional, Malaysia
Abstract
Modern enterprises generate and manage enormous volumes of structured, semi-structured, and unstructured information across databases, data warehouses, data lakes, cloud storage systems, enterprise applications, collaboration platforms, and document repositories. Conventional enterprise knowledge-management approaches often struggle to provide timely and contextually relevant access to this distributed information. Retrieval-Augmented Generation (RAG) provides an approach for combining information retrieval with generative artificial intelligence, enabling language models to generate responses based on enterprise-specific knowledge retrieved from authorized data sources. This study examines the integration of modern data-engineering practices with RAG architectures across cloud platforms for enterprise knowledge management. The research focuses on data ingestion, transformation, metadata management, document processing, embedding generation, vector search, retrieval, prompt construction, response generation, and governance. A conceptual methodology is adopted to analyze how cloud-native data pipelines can support reliable and scalable RAG systems while maintaining data quality, security, lineage, and access control. The proposed framework considers heterogeneous enterprise data as a continuously evolving knowledge ecosystem rather than a static repository. It examines the role of batch and streaming pipelines, lakehouse architectures, vector databases, metadata catalogs, and application programming interfaces in establishing an integrated retrieval environment. Particular attention is given to challenges involving data freshness, semantic search accuracy, hallucination, access permissions, privacy, interoperability, and multi-cloud complexity. The study provides a conceptual foundation for implementing enterprise RAG systems through disciplined data engineering and cloud-based infrastructure
Keywords
DOI
PDF Download
Publication Information
| Journal | International Research Journal of Innovative Engineering |
|---|---|
| Volume | 10 |
| Number | 4 |
| Pages | 19339-19347 |
| Published | 2026-08-27 |
| Issue | Vol. 10 No. 4 (2026): International Research Journal of Innovative Engineering (IRJIE) |
References
1. Nicolas-Plata, A., Gonzalez-Compean, J. L., & Sosa-Sosa, V. J. (2024). A service mesh approach to integrate processing patterns into microservices applications. Cluster Computing, 27, 7417–7438.
2. Meshram, A. K., & Sikarwar, V. (2026, March). Big Data-Driven AI Models for Network Anomaly Detection and Cyber Threat Forecasting. In 2026 14th International Symposium on Digital Forensics and Security (ISDFS) (pp. 1-6). IEEE.
3. Polamarasetty, V. K. (2024). Workday-based enterprise benefits management: Configuration, automation, and operational optimization. International Journal of Research Publications in Engineering, Technology and Management (IJRPETM), 7(5), 11347–11352.
4. Bandaru, P. K. (2022). Hardware-in-the-loop testing for connected vehicles: Enhancing software reliability through continuous validation. International Journal of Engineering & Extended Technologies Research (IJEETR), 4(2), 4645–4651.
5. Raja, G. V. (2022). AI Enabled Cloud and Data Engineering Frameworks for Secure, Scalable, and Intelligent Cyber Physical Analytics Systems. International Journal of Research and Applied Innovations, 5(4), 7395–7409.
6. Jain, R. (2021). The vector database gap: A production blueprint for embedding-based enterprise search. International Journal of Future Innovative Science and Technology (IJFIST), 4(4), 6706–6713.
7. Kagga, S. R., Ayyagari, V., & Rekulapalli, K. (2026, June). Federated Learning for Dialysis Outcome Prediction Across Hospitals. In 2026 5th International Conference on Computer Networks, Big Data and IoT (ICCBI) (pp. 533-538). IEEE. Bottom of Form
8. Vemireddy, S. (2024). Secure and scalable intelligent service architectures for next-generation enterprise applications. International Journal of Engineering & Extended Technologies Research (IJEETR), 6(4), 8177-8182.
9. Pothuri, M. K. (2026). AI-Optimized Symmetry Episode Analytics for Early Detection of High-Utilizers: A Claims-Based Predictive Modelling Framework Using Advanced Machine Learning Models. International Journal of Artificial Intelligence, Data Science, and Machine Learning, 7(1), 279-287.
10. Bhuiyan, T., Tasnim, M., Khan, M. A. N., Onik, T. A., Nabil, M. A., Alam, A., & Himeluzzaman, M. (2025). Machine learning-based prediction of diabetes using patient health records. Vascular and Endovascular Review, 8(10s), 446–454.
11. Ahuja, D. (2026, March). Performance Analysis of a Cloud-Native Web Application Deployed on Kubernetes. In 2026 14th International Symposium on Digital Forensics and Security (ISDFS) (pp. 01-06). IEEE.
12. Khare, A., Bhatnagar, A., & Singh, K. (2025, October). Adoption of SAP SuccessFactors in Talent Analytics and Workforce Forecasting. In 2025 2nd International Conference on Recent Trends in Electrical, Electronics and Computing Technologies (ICRTEECT) (pp. 1-6). IEEE.
13. Anand, L. (2023). Machine Learning Enabled Enterprise Integration through Intelligent API Governance Secure Cloud Infrastructure and Automated Operations. International Journal of Research and Applied Innovations, 6(3), 5972-5979.
14. Manda, P. (2026). Database modernization - Sybase to SQL Server migration. International Journal of Engineering & Extended Technologies Research, 8(1), 252–258.
15. Agarwal, S. (2025). Observability in stateful workloads: Strategies for monitoring persistent services in dynamic cloud environments. International Journal of Computer Technology and Electronics Communication, 8(5), 11631–11636.
16. Hashmi, H., & Srikanth, V. (2023, November). Combining Neural Networks to Recognize Offline Digits & Words by Training the Model Using Keras. In 2023 3rd International Conference on Advancement in Electronics & Communication Engineering (AECE) (pp. 694-697). IEEE.
17. Vedula, J. (2023). Operational resilience by design: A continuity assurance model for modernizing critical energy applications. International Journal of Applied Engineering & Technology, 5(S2), 299–309.
18. Jayabalan, K. (2026). Model predictive control in smart energy systems. In Intelligent Control in Smart Energy Systems (pp. 111-131). Elsevier.
19. Reddy, K. V. (2025). Layered Verification Strategies for AI Accelerator Hardware: Matrix Engines and SRAM Buffers. Journal of Computer Science and Technology Studies, 7(5), 786-795.
20. Sandhu, Y. S. (2025). Scalable pipeline orchestration through intelligent dependency-aware recovery and performance optimization. International Journal of Future Innovative Science and Technology (IJFIST), 8(5), 15712–15722.
21. Badam, L. R. (2024). AI-based early warning system for financial scams targeting consumers. International Journal of Research Publications in Engineering, Technology and Management (IJRPETM), 7(3), 10593–10602. https://doi.org/10.15662/IJRPETM.2024.0703016
22. Bitragunta, S. L. V. (2022, November 20). High level modeling of high-voltage gallium nitride (GaN) power devices for sophisticated power electronics applications. Journal of Artificial Intelligence, Machine Learning and Data Science, 1(1), 2011–2015.
23. Gopinathan, V. R. (2023). Modernizing Enterprise Applications Using Intelligent API Frameworks Machine Learning and Cloud Native Computing. International Journal of Science, Research and Technology, 6(5), 10690-10697.
24. Rahul Rao Juvvadi. (2024). Large Language Models in FP&A: An Architecture for Grounded Variance Commentary and Driver-Based Narrative Generation. Enterprise Development and Microfinance, 34(1), 71–84.
25. Chidambaranathan, S., Mudunuri, L. N. R., Banu, S. S., Anitha, V., Praveen, R., & Golla, S. (2024, November). Effects of Water Quality Deterioration and the Application of Artificial Intelligence and Blockchain Technology. In 2024 International Conference on Advances in Computing, Communication and Materials (ICACCM) (pp. 1-6). IEEE.
26. Selvarajan, K. (2022). Cloud-native architectures for high-throughput distributed data processing. International Journal of Emerging Trends in Engineering and Management Research (IJETEMR), 7(5), 12538–12548.
27. Lenin, S. T., & Venkatasalam, K. (2025). Effective classification of heart disease using convolutional neural networks. Circuits, Systems, and Signal Processing, 44(2), 911-935.
28. Manda, P. (2026). Database modernization - Sybase to SQL Server migration. International Journal of Engineering & Extended Technologies Research, 8(1), 252–258.
29. Vimal Raja, G. (2024). Intelligent data transition in automotive manufacturing systems using machine learning. International Journal of Multidisciplinary and Scientific Emerging Research, 12(2), 515-518.
30. Ramasamy, M. (2022). Architecting scalable intent-based networking platforms for enterprise automation. International Journal of Emerging Trends in Engineering and Management Research (IJETEMR), 7(3), 11812–11823.
31. Bhuiyan, T., Tasnim, M., Khan, M. A. N., Onik, T. A., Nabil, M. A., Alam, A., & Himeluzzaman, M. (2025). Machine learning-based prediction of diabetes using patient health records. Vascular and Endovascular Review, 8(10s), 446–454.
32. Soundappan, S. J. (2023). Empowering Secure Enterprise Digital Transformation using Artificial Intelligence for Predictive Analytics in Cloud Computing. International Journal of Emerging Trends in Engineering and Management Research, 8(5), 14373.
33. Matrouk, K., V, S., Kumar, S., Bhadla, M. K., Sabirov, M., & Saadh, M. J. (2023). Deep Learning–based Dynamic User Alignment in Social Networks. ACM Journal of Data and Information Quality, 15(3), 1-26.
34. Kargeti, H. (2026, February). Automating Enterprise Vulnerability Exposure: SCAP-Based Asset-CVE Linkage with Social Signal Triage for Cyber Defense. In 2026 International Conference on Artificial Intelligence, Computer, Data Sciences and Applications (ACDSA) (pp. 1-6). IEEE.
35. Padmanabham, S. (2024). Intelligent security architecture for risk and compliance. International Journal of Science, Research and Technology, 7(1), 11373-11380.
36. Ravichandran, S., & Kandasamy, V. (2025). Optimized Attention Augmented Residual Convolutional Neural Network with Fa-Resnet for Fabric Defect Detection. Journal of Control Engineering and Applied Informatics, 27(4), 3-15.
37. Jain, R. (2021). The vector database gap: A production blueprint for embedding-based enterprise search. International Journal of Future Innovative Science and Technology (IJFIST), 4(4), 6706–6713.
38. Jayabalan, K., Gupta, S., Kaur, U., Shubneet, Yadav, A. R., & Talwandi, N. S. (2026, February). Developing Energy-Efficient Neural Network Models for Real-Time Edge AI in Resource-Constrained Environments. In International Conference On Innovative Computing And Communication (pp. 519-529). Cham: Springer Nature Switzerland.
39. Guo, T., Yang, Q., Wang, C., Liu, Y., Li, P., Tang, J., Li, D., & Wen, Y. (2024). KnowledgeNavigator: Leveraging large language models for enhanced reasoning over knowledge graph. Complex & Intelligent Systems, 10, 7063–7076.
2. Meshram, A. K., & Sikarwar, V. (2026, March). Big Data-Driven AI Models for Network Anomaly Detection and Cyber Threat Forecasting. In 2026 14th International Symposium on Digital Forensics and Security (ISDFS) (pp. 1-6). IEEE.
3. Polamarasetty, V. K. (2024). Workday-based enterprise benefits management: Configuration, automation, and operational optimization. International Journal of Research Publications in Engineering, Technology and Management (IJRPETM), 7(5), 11347–11352.
4. Bandaru, P. K. (2022). Hardware-in-the-loop testing for connected vehicles: Enhancing software reliability through continuous validation. International Journal of Engineering & Extended Technologies Research (IJEETR), 4(2), 4645–4651.
5. Raja, G. V. (2022). AI Enabled Cloud and Data Engineering Frameworks for Secure, Scalable, and Intelligent Cyber Physical Analytics Systems. International Journal of Research and Applied Innovations, 5(4), 7395–7409.
6. Jain, R. (2021). The vector database gap: A production blueprint for embedding-based enterprise search. International Journal of Future Innovative Science and Technology (IJFIST), 4(4), 6706–6713.
7. Kagga, S. R., Ayyagari, V., & Rekulapalli, K. (2026, June). Federated Learning for Dialysis Outcome Prediction Across Hospitals. In 2026 5th International Conference on Computer Networks, Big Data and IoT (ICCBI) (pp. 533-538). IEEE. Bottom of Form
8. Vemireddy, S. (2024). Secure and scalable intelligent service architectures for next-generation enterprise applications. International Journal of Engineering & Extended Technologies Research (IJEETR), 6(4), 8177-8182.
9. Pothuri, M. K. (2026). AI-Optimized Symmetry Episode Analytics for Early Detection of High-Utilizers: A Claims-Based Predictive Modelling Framework Using Advanced Machine Learning Models. International Journal of Artificial Intelligence, Data Science, and Machine Learning, 7(1), 279-287.
10. Bhuiyan, T., Tasnim, M., Khan, M. A. N., Onik, T. A., Nabil, M. A., Alam, A., & Himeluzzaman, M. (2025). Machine learning-based prediction of diabetes using patient health records. Vascular and Endovascular Review, 8(10s), 446–454.
11. Ahuja, D. (2026, March). Performance Analysis of a Cloud-Native Web Application Deployed on Kubernetes. In 2026 14th International Symposium on Digital Forensics and Security (ISDFS) (pp. 01-06). IEEE.
12. Khare, A., Bhatnagar, A., & Singh, K. (2025, October). Adoption of SAP SuccessFactors in Talent Analytics and Workforce Forecasting. In 2025 2nd International Conference on Recent Trends in Electrical, Electronics and Computing Technologies (ICRTEECT) (pp. 1-6). IEEE.
13. Anand, L. (2023). Machine Learning Enabled Enterprise Integration through Intelligent API Governance Secure Cloud Infrastructure and Automated Operations. International Journal of Research and Applied Innovations, 6(3), 5972-5979.
14. Manda, P. (2026). Database modernization - Sybase to SQL Server migration. International Journal of Engineering & Extended Technologies Research, 8(1), 252–258.
15. Agarwal, S. (2025). Observability in stateful workloads: Strategies for monitoring persistent services in dynamic cloud environments. International Journal of Computer Technology and Electronics Communication, 8(5), 11631–11636.
16. Hashmi, H., & Srikanth, V. (2023, November). Combining Neural Networks to Recognize Offline Digits & Words by Training the Model Using Keras. In 2023 3rd International Conference on Advancement in Electronics & Communication Engineering (AECE) (pp. 694-697). IEEE.
17. Vedula, J. (2023). Operational resilience by design: A continuity assurance model for modernizing critical energy applications. International Journal of Applied Engineering & Technology, 5(S2), 299–309.
18. Jayabalan, K. (2026). Model predictive control in smart energy systems. In Intelligent Control in Smart Energy Systems (pp. 111-131). Elsevier.
19. Reddy, K. V. (2025). Layered Verification Strategies for AI Accelerator Hardware: Matrix Engines and SRAM Buffers. Journal of Computer Science and Technology Studies, 7(5), 786-795.
20. Sandhu, Y. S. (2025). Scalable pipeline orchestration through intelligent dependency-aware recovery and performance optimization. International Journal of Future Innovative Science and Technology (IJFIST), 8(5), 15712–15722.
21. Badam, L. R. (2024). AI-based early warning system for financial scams targeting consumers. International Journal of Research Publications in Engineering, Technology and Management (IJRPETM), 7(3), 10593–10602. https://doi.org/10.15662/IJRPETM.2024.0703016
22. Bitragunta, S. L. V. (2022, November 20). High level modeling of high-voltage gallium nitride (GaN) power devices for sophisticated power electronics applications. Journal of Artificial Intelligence, Machine Learning and Data Science, 1(1), 2011–2015.
23. Gopinathan, V. R. (2023). Modernizing Enterprise Applications Using Intelligent API Frameworks Machine Learning and Cloud Native Computing. International Journal of Science, Research and Technology, 6(5), 10690-10697.
24. Rahul Rao Juvvadi. (2024). Large Language Models in FP&A: An Architecture for Grounded Variance Commentary and Driver-Based Narrative Generation. Enterprise Development and Microfinance, 34(1), 71–84.
25. Chidambaranathan, S., Mudunuri, L. N. R., Banu, S. S., Anitha, V., Praveen, R., & Golla, S. (2024, November). Effects of Water Quality Deterioration and the Application of Artificial Intelligence and Blockchain Technology. In 2024 International Conference on Advances in Computing, Communication and Materials (ICACCM) (pp. 1-6). IEEE.
26. Selvarajan, K. (2022). Cloud-native architectures for high-throughput distributed data processing. International Journal of Emerging Trends in Engineering and Management Research (IJETEMR), 7(5), 12538–12548.
27. Lenin, S. T., & Venkatasalam, K. (2025). Effective classification of heart disease using convolutional neural networks. Circuits, Systems, and Signal Processing, 44(2), 911-935.
28. Manda, P. (2026). Database modernization - Sybase to SQL Server migration. International Journal of Engineering & Extended Technologies Research, 8(1), 252–258.
29. Vimal Raja, G. (2024). Intelligent data transition in automotive manufacturing systems using machine learning. International Journal of Multidisciplinary and Scientific Emerging Research, 12(2), 515-518.
30. Ramasamy, M. (2022). Architecting scalable intent-based networking platforms for enterprise automation. International Journal of Emerging Trends in Engineering and Management Research (IJETEMR), 7(3), 11812–11823.
31. Bhuiyan, T., Tasnim, M., Khan, M. A. N., Onik, T. A., Nabil, M. A., Alam, A., & Himeluzzaman, M. (2025). Machine learning-based prediction of diabetes using patient health records. Vascular and Endovascular Review, 8(10s), 446–454.
32. Soundappan, S. J. (2023). Empowering Secure Enterprise Digital Transformation using Artificial Intelligence for Predictive Analytics in Cloud Computing. International Journal of Emerging Trends in Engineering and Management Research, 8(5), 14373.
33. Matrouk, K., V, S., Kumar, S., Bhadla, M. K., Sabirov, M., & Saadh, M. J. (2023). Deep Learning–based Dynamic User Alignment in Social Networks. ACM Journal of Data and Information Quality, 15(3), 1-26.
34. Kargeti, H. (2026, February). Automating Enterprise Vulnerability Exposure: SCAP-Based Asset-CVE Linkage with Social Signal Triage for Cyber Defense. In 2026 International Conference on Artificial Intelligence, Computer, Data Sciences and Applications (ACDSA) (pp. 1-6). IEEE.
35. Padmanabham, S. (2024). Intelligent security architecture for risk and compliance. International Journal of Science, Research and Technology, 7(1), 11373-11380.
36. Ravichandran, S., & Kandasamy, V. (2025). Optimized Attention Augmented Residual Convolutional Neural Network with Fa-Resnet for Fabric Defect Detection. Journal of Control Engineering and Applied Informatics, 27(4), 3-15.
37. Jain, R. (2021). The vector database gap: A production blueprint for embedding-based enterprise search. International Journal of Future Innovative Science and Technology (IJFIST), 4(4), 6706–6713.
38. Jayabalan, K., Gupta, S., Kaur, U., Shubneet, Yadav, A. R., & Talwandi, N. S. (2026, February). Developing Energy-Efficient Neural Network Models for Real-Time Edge AI in Resource-Constrained Environments. In International Conference On Innovative Computing And Communication (pp. 519-529). Cham: Springer Nature Switzerland.
39. Guo, T., Yang, Q., Wang, C., Liu, Y., Li, P., Tang, J., Li, D., & Wen, Y. (2024). KnowledgeNavigator: Leveraging large language models for enhanced reasoning over knowledge graph. Complex & Intelligent Systems, 10, 7063–7076.
Statistics
88
Views
49
Downloads