Comparative Evaluation of Open-Source ETL Frameworks in Building Cost-Effective Business Intelligence Infrastructure
Keywords:
ETL Framework, Business Intelligence, Open-Source Data Pipeline, Data Warehouse IntegrationAbstract
The high licensing costs of proprietary Extract, Transform, Load (ETL) tools are driving organizations to turn to cost-effective open-source solutions for building Business Intelligence (BI) infrastructure. This study aims to evaluate and compare four popular open-source ETL frameworks—Pentaho, Talend Open Studio, Apache NiFi, and Apache Airflow—to identify the best data pipeline solution for BI systems. This study uses a literature review and comparative analysis based on each tool's official documentation, evaluating five key criteria: Ease of Setup, GUI Support, Community Support, Cloud Integration, and License Type—in the end-to-end data architecture from Source Database to Data Warehouse to BI Dashboard. The analysis shows that Pentaho and Apache NiFi excel in their drag-and-drop visual interfaces and ease of initial configuration, making them ideal for creating visual-based pipelines. Conversely, Apache Airflow excels in cloud integration and community support thanks to its code-as-configuration approach despite having a higher learning curve, while Talend delivers balanced performance across all criteria. In conclusion, framework selection largely depends on the team's technical capabilities; Visual-based tools are suitable for rapid implementation, while code-based platforms are suitable for complex cloud architectures. This tool-based approach eliminates proprietary licensing costs, with suggestions for further research on real-time streaming data performance testing.
Downloads
References
[1] I. Ahmed, “An Experimental Evaluation of ETL-Driven Business Intelligence Architectures for Enhancing AI-Based Cybersecurity Performance,” Am. J. Data Sci. Anal., vol. 7, no. 03, pp. 45–81, Mar. 2026, doi: 10.63125/1w976q53.
[2] D. Mahmud and M. Z. Ikbal, “THE ROLE OF ETL (EXTRACT-TRANSFORM-LOAD) PIPELINES IN SCALABLE BUSINESS INTELLIGENCE: A COMPARATIVE STUDY OF DATA INTEGRATION TOOLS,” ASRC Procedia Glob. Perspect. Sci. Scholarsh., vol. 2, no. 1, pp. 89–121, Apr. 2022, doi: 10.63125/1spa6877.
[3] C. Erlingsson and P. Brysiewicz, “A hands-on guide to doing content analysis,” Afr. J. Emerg. Med., vol. 7, no. 3, pp. 93–99, Sep. 2017, doi: 10.1016/j.afjem.2017.08.001.
[4] F. P. E. Putra, D. T. Agustina, T. S. K. Khotimah, and T. Ramadhanty, “Analisis Kinerja Jaringan 5G dalam Meningkatkan Konektivitas Internet of Things (IoT),” J. Inform. Dan Tekonologi Komput. JITEK, vol. 5, no. 1, pp. 56–62, Mar. 2025, doi: 10.55606/jitek.v5i1.5836.
[5] A. Sorour and A. S. Atkins, “Big data challenge for monitoring quality in higher education institutions using business intelligence dashboards,” J. Electron. Sci. Technol., vol. 22, no. 1, p. 100233, Mar. 2024, doi: 10.1016/j.jnlest.2024.100233.
[6] O. Oloruntoba, D. Oyeyemi, and O. Omolayo, “Designing Scalable ETL Pipelines for Multi-Source Graph Database Ingestion,” J. Comput. Anal. Appl., vol. 34, pp. 236–258, Jul. 2025, doi: 10.5281/zenodo.19355312.
[7] A. Nambiar and D. Mundra, “An Overview of Data Warehouse and Data Lake in Modern Enterprise Data Management,” Big Data Cogn. Comput., vol. 6, no. 4, p. 132, Dec. 2022, doi: 10.3390/bdcc6040132.
[8] F. P. E. Putra, R. M. Ilhamsyah, S. A. Efendy, and A. Rizki, “Implementation And Evaluation Of Zerotier-Based Virtual Network For Device Connectivity,” Brill. Res. Artif. Intell., vol. 5, no. 1, pp. 281–290, Mar. 2025, doi: 10.47709/brilliance.v5i1.5966.
[9] F. P. E. Putra, suhdi, A. Ramadhani, and Marzuq, “Integrasi Teknologi Kuantum dan fiber Optik untuk Meningkatkan Keamanan dan Efisiensi Jaringan Masa Depan,” J. Ilm. Ilk. - Ilmu Komput. Inform., vol. 8, no. 2, pp. 151–163, Jul. 2025, doi: 10.47324/ilkominfo.v8i2.342.
[10] C. T. Gonçalves, M. J. A. Gonçalves, and M. I. Campante, “Developing Integrated Performance Dashboards Visualisations Using Power BI as a Platform,” Information, vol. 14, no. 11, p. 614, Nov. 2023, doi: 10.3390/info14110614.
[11] I. Lokaadinugroho, A. S. Girsang, and B. Burhanudin, “Tableau Business Intelligence Using the 9 Steps of Kimball’s Data Warehouse & Extract Transform Loading of the Pentaho Data Integration Process Approach in Higher Education,” Eng. Math. Comput. Sci. J. EMACS, vol. 3, no. 1, pp. 1–11, Jan. 2021, doi: 10.21512/emacsjournal.v3i1.6816.
[12] F. P. E. Putra, S. M. Dewi, Maugfiroh, and A. Hamzah, “Privasi dan Keamanan Penerapan IoT Dalam Kehidupan Sehari-Hari : Tantangan dan Implikasi,” J. Sistim Inf. Dan Teknol., pp. 26–32, Jul. 2023, doi: 10.37034/jsisfotek.v5i2.232.
[13] K. C. Nagabhyru, “Bridging Traditional ETL Pipelines with AI Enhanced Data Workflows: Foundations of Intelligent Automation in Data Engineering,” Online J. Eng. Sci., vol. 1, no. 1, pp. 82–93, Nov. 2022, doi: 10.31586/ojes.2022.1345.
[14] A. Adewusi, U. Okoli, E. Adaga, T. Olorunsogo, O. F. Asuzu, and D. Daraojimba, “BUSINESS INTELLIGENCE IN THE ERA OF BIG DATA: A REVIEW OF ANALYTICAL TOOLS AND COMPETITIVE ADVANTAGE,” Comput. Sci. IT Res. J., vol. 5, pp. 415–431, Feb. 2024, doi: 10.51594/csitrj.v5i2.791.
[15] F. P. E. Putra, Iksan, and N. Saadah, “Interaktif dan Personalisasi Peningkatan Pembelajaran IoT di Sekolah,” J. Sistim Inf. Dan Teknol., pp. 175–181, Jul. 2023, doi: 10.37034/jsisfotek.v5i2.236.
[16] H. Li, X. Wang, Y. Feng, Y. Qi, and J. Tian, “Integration Methods and Advantages of Machine Learning with Cloud Data Warehouses,” Int. J. Comput. Sci. Inf. Technol., vol. 2, no. 1, pp. 348–358, Mar. 2024, doi: 10.62051/ijcsit.v2n1.36.
[17] S. Kusumba, “Cloud-Optimized Intelligent ETL Framework for Scalable Data Integration in Healthcare–Finance Interoperability Ecosystems,” Int. J. Res. Appl. Innov., vol. 5, no. 3, pp. 7056–7065, May 2022, doi: 10.15662/IJRAI.2022.0503004.
[18] S. E. Bibri and S. K. Jagatheesaperumal, “Harnessing the Potential of the Metaverse and Artificial Intelligence for the Internet of City Things: Cost-Effective XReality and Synergistic AIoT Technologies,” Smart Cities, vol. 6, no. 5, pp. 2397–2429, Oct. 2023, doi: 10.3390/smartcities6050109.
[19] P. Patel, “Infrastructure Economics of Sparse Mixture-of-Experts in Cloud-Native NLP: Benchmarking Cost, Accuracy, and Performance,” Glob. Bus. Econ. J., Jun. 2025, doi: 10.70924/f83n6wqz/vboj68ls.
[20] F. Dottori, L. Mentaschi, A. Bianchi, L. Alfieri, and L. Feyen, “Cost-effective adaptation strategies to rising river flood risk in Europe,” Nat. Clim. Change, vol. 13, no. 2, pp. 196–202, Feb. 2023, doi: 10.1038/s41558-022-01540-0.
[21] R. Adepu, “MODERNIZING LEGACY DATA CENTERS THROUGH VIRTUALIZATION AND SOFTWARE-DEFINED INFRASTRUCTURE,” Int. J. Res. Appl. Innov., vol. 4, no. 4, pp. 17–36, Aug. 2021, doi: 10.15662/2by66f36.
[22] D. Kapitan, F. Heddema, A. Dekker, M. Sieswerda, B.-J. Verhoeff, and M. Berg, “Data Interoperability in Context: The Importance of Open-Source Implementations When Choosing Open Standards,” J. Med. Internet Res., vol. 27, p. e66616, Apr. 2025, doi: 10.2196/66616.
[23] A. Dapkute, V. Siozinys, M. Jonaitis, M. Kaminickas, and M. Siozinys, “Digital Twin Data Management: Framework and Performance Metrics of Cloud-Based ETL System,” Machines, vol. 12, no. 2, p. 130, Feb. 2024, doi: 10.3390/machines12020130.
[24] Y.-X. Liu et al., “EasyAmplicon: An easy-to-use, open-source, reproducible, and community-based pipeline for amplicon data analysis in microbiome research,” iMeta, vol. 2, no. 1, p. e83, 2023, doi: 10.1002/imt2.83.
[25] M. Schmitt, “Automated machine learning: AI-driven decision making in business analytics,” Intell. Syst. Appl., vol. 18, p. 200188, May 2023, doi: 10.1016/j.iswa.2023.200188.
[26] X. Xie, N. Moretti, J. Merino, J. Y. Chang, P. Pauwels, and A. K. Parlikad, “Enabling building digital twin: Ontology-based information management framework for multi-source data integration,” IOP Conf. Ser. Earth Environ. Sci., vol. 1101, no. 9, p. 092010, Nov. 2022, doi: 10.1088/1755-1315/1101/9/092010.
[27] N. G. Maswanganyi, N. M. Fumani, J. K. Khoza, B. A. Thango, and L. Matshaka, “Evaluating the Impact of Database and Data Warehouse Technologies on Organizational Performance: A Systematic Review,” Oct. 02, 2024, Business, Economics and Management. doi: 10.20944/preprints202410.0059.v1.
[28] M. C. Solano and J. C. Cruz, “Integrating Analytics in Enterprise Systems: A Systematic Literature Review of Impacts and Innovations,” Adm. Sci., vol. 14, no. 7, p. 138, Jul. 2024, doi: 10.3390/admsci14070138.
[29] E. Kristoffersen, P. Mikalef, F. Blomsma, and J. Li, “The effects of business analytics capability on circular economy implementation, resource orchestration capability, and firm performance,” Int. J. Prod. Econ., vol. 239, p. 108205, Sep. 2021, doi: 10.1016/j.ijpe.2021.108205.
[30] F. P. Eka Putra, A. Baidawi, A. A. Mubarok, and Frediyanto, “Merancang Jaringan Sensor Nirkabel dan IoT untuk Kota Pintar Pamekasan,” J. Inf. Dan Teknol., pp. 138–145, Jul. 2023, doi: 10.37034/jidt.v5i2.331.
[31] U. Gain and V. Hotti, “Low-code AutoML-augmented Data Pipeline – A Review and Experiments,” J. Phys. Conf. Ser., vol. 1828, no. 1, p. 012015, Feb. 2021, doi: 10.1088/1742-6596/1828/1/012015.
[32] F. P. Eka Putra, I. N. Sudana Degeng, S. Ulfa, and W. Kamdi, “The Evolution of Quality Education: Impacts and Challenges of Using Open Educational Resources (OER) and Open Educational Practices (OEP) in the Conceive - Design - Implement - Operate (CDIO) Framework.,” TEM J., vol. 13, no. 1, p. 386, Feb. 2024, doi: 10.18421/TEM131-40.
[33] N. Syed, A. Anwar, Z. Baig, and S. Zeadally, “Artificial Intelligence as a Service (AIaaS) for Cloud, Fog and the Edge: State-of-the-Art Practices,” ACM Comput Surv, vol. 57, no. 8, p. 211:1-211:36, Mar. 2025, doi: 10.1145/3712016.
[34] M. G. Kahn et al., “Migrating a research data warehouse to a public cloud: challenges and opportunities,” J. Am. Med. Inform. Assoc., vol. 29, no. 4, pp. 592–600, Mar. 2022, doi: 10.1093/jamia/ocab278.
[35] F. Olan, E. Ogiemwonyi Arakpogun, J. Suklan, F. Nakpodia, N. Damij, and U. Jayawickrama, “Artificial intelligence and knowledge sharing: Contributing factors to organizational performance,” J. Bus. Res., vol. 145, pp. 605–615, Jun. 2022, doi: 10.1016/j.jbusres.2022.03.008.
[36] S. N. Annam, “Optimizing IT Infrastructure for Business Continuity,” Stallion J. Multidiscip. Assoc. Res. Stud., vol. 1, no. 5, pp. 31–42, Oct. 2022, doi: 10.55544/sjmars.1.5.7.
[37] F. P. E. Putra, M. A. Mahmud, and I. S. Maqom, “Pengembangan Sistem Pemantauan Lingkungan Berbasis Internet of Things (IoT) di Kampus,” Digit. Transform. Technol., vol. 3, no. 2, pp. 996–1001, 2023, doi: 10.47709/digitech.v3i2.3457.
[38] Fauzan Prasetyo Eka Putra, Noviyani Dwi Saputri, Fathur Rosi, and Rohilia Loati, “Optimalisasi Infrastruktur Cloud Networking melalui Integrasi SDN, NFV, dan Multi-Cloud,” J. Inform. Dan Tekonologi Komput. JITEK, vol. 5, no. 1, pp. 118–125, Mar. 2025, doi: 10.55606/jitek.v5i1.6099.
[39] M. Picciani et al., “Oktoberfest: Open-source spectral library generation and rescoring pipeline based on Prosit,” PROTEOMICS, vol. 24, no. 8, p. 2300112, 2024, doi: 10.1002/pmic.202300112.
[40] F. Eka Putra, M. Mustafida, R. Alfadili, and A. Nahriyah, “Perancangan Jaringan Nirkabel Berbasis Mesh untuk Menunjang Aplikasi Smart City,” J. Inform. Dan Tekonologi Komput. JITEK, vol. 5, pp. 84–92, Mar. 2025, doi: 10.55606/jitek.v5i1.5934.
[41] J. Sreemathy, R. Brindha, M. Selva Nagalakshmi, N. Suvekha, N. Karthick Ragul, and M. Praveennandha, “Overview of ETL Tools and Talend-Data Integration,” in 2021 7th International Conference on Advanced Computing and Communication Systems (ICACCS), Mar. 2021, pp. 1650–1654. doi: 10.1109/ICACCS51430.2021.9441984.
[42] J. Tagliabue and C. Greco, “Reproducible data science over data lakes: replayable data pipelines with Bauplan and Nessie,” in Proceedings of the Eighth Workshop on Data Management for End-to-End Machine Learning, in DEEM ’24. New York, NY, USA: Association for Computing Machinery, Jun. 2024, pp. 67–71. doi: 10.1145/3650203.3663335.
[43] F. P. E. Putra, A. B. Tamam, R. W. Efendi, and Z. Muim, “Optimasi Keamanan DNS: Eksplorasi Optimal dengan Implementasi DNS Security Extensions (DNSSEC),” REMIK Ris. Dan E-J. Manaj. Inform. Komput., vol. 8, no. 1, pp. 349–358, Jan. 2024, doi: 10.33395/remik.v8i1.13398.
[44] Y. Alghofaili, A. Albattah, N. Alrajeh, M. A. Rassam, and B. A. S. Al-rimy, “Secure Cloud Infrastructure: A Survey on Issues, Current Solutions, and Open Challenges,” Appl. Sci., vol. 11, no. 19, p. 9005, Jan. 2021, doi: 10.3390/app11199005.
[45] B. Bent et al., “The digital biomarker discovery pipeline: An open-source software platform for the development of digital biomarkers using mHealth and wearables data,” J. Clin. Transl. Sci., vol. 5, no. 1, p. e19, 2021, doi: 10.1017/cts.2020.511.
[46] N. P. Rana, S. Chatterjee, Y. K. Dwivedi, and S. Akter, “Understanding dark side of artificial intelligence (AI) integrated business analytics: assessing firm’s operational inefficiency and competitiveness,” Eur. J. Inf. Syst., vol. 31, no. 3, pp. 364–387, May 2022, doi: 10.1080/0960085X.2021.1955628.
[47] R. Somayajula, R. R. Pai, N. Sajanraj, and K. Shah, “Zero-ETL Architectures for AI Workloads Direct ML Model Access on Cloud-Native OLAP Systems,” in 2025 International Conference on Computing Technologies (ICOCT), Jun. 2025, pp. 1–7. doi: 10.1109/ICOCT64433.2025.11118928.
[48] Fauzan Prasetyo Eka Putra, Maktsuful Ghummah, Moh. Amrullah, and Rafli Hidayatullah, “Studi Kinerja Mesh Network untuk Penerapan Internet of Things (IoT) di Lingkungan Perkotaan,” J. Inform. Dan Tekonologi Komput. JITEK, vol. 5, no. 1, pp. 63–73, Mar. 2025, doi: 10.55606/jitek.v5i1.5895.
[49] I. Naseer, “AWS Cloud Computing Solutions: Optimizing Implementation for Businesses,” Stat. Comput. Interdiscip. Res., vol. 5, no. 2, pp. 121–132, Dec. 2023, doi: 10.52700/scir.v5i2.138.
[50] F. P. E. Putra, Fauzan, Syirofi, M. Mursidi, D. Wahid, and A. Nuraini, “Sistem Pengendali Lingkungan Pertanian Dengan Wireless Sensor Network Untuk Mengoptimalkan Budidaya Hidroponik,” Digit. Transform. Technol., vol. 3, no. 2, pp. 931–937, 2023, doi: 10.47709/digitech.v3i2.3461.
Published
Issue
Section
License
Copyright (c) 2026 Yuris Ikrar RABBANI, Wildan Ramadhani (Penulis)
This work is licensed under a Creative Commons Attribution 4.0 International License.
