Energy Measurement System for Data Lake: An Initial Approach - Université de Pau et des Pays de l'Adour
Communication Dans Un Congrès Année : 2024

Energy Measurement System for Data Lake: An Initial Approach

Résumé

Data Lakes are increasingly deployed as a solution for Big Data analytics. Recent improvements in Data Lake technology have focused on improving data access, governance, and discoverability. How-ever, the energy consumption of data operations, a non-trivial issue for eco-conscious organizations, is currently overlooked. Furthermore, existing monitoring tools do not adequately address the complexities of Data Lake architectures. This paper presents the initial phase of developing a system for measuring energy in Data Lake pipeline operations. The novelty of our solution lies in the fact that we define four measures to assess the power usage of crucial hardware components in a Data Lake context: CPU, RAM, NIC, and storage devices. To validate our approach, we developed a monitoring tool grounded in real-world datasets from a Data Lake benchmark.
Fichier principal
Vignette du fichier
ACIIDS_2024_paper_63-1.pdf (678.74 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04634197 , version 1 (03-07-2024)

Identifiants

  • HAL Id : hal-04634197 , version 1

Citer

Alvarez-Valera Hernan Humberto, Maurice Alexandre, Franck Ravat, Jiefu Song, Philippe Roose, et al.. Energy Measurement System for Data Lake: An Initial Approach. 16th Asian Conference on Intelligent Information and Database Systems (ACIIDS 2024), Apr 2024, Ras Al Khaimah, United Arab Emirates. à paraître. ⟨hal-04634197⟩
159 Consultations
46 Téléchargements

Partager

More