Designing a metadata framework for bigdata models in Cloudera Data Lakes across AWS, Azure, and GCP
Author(s):
Ramamurthy Valavandan, Balakrishnan Gothandapani, Savitha Ramamurthy, Jagathambal Subramanian, Kanagalakshmi Subramaian, Valavandan Valavandan, Bharani, Dharani
Keywords:
Cloud Platforms AWS Azure GCP , Big Data Environments, Cloudera, Metadata Migration, PySpark, SparkSQL
Abstract
his research presents an innovative metadata framework design for big data models in Cloudera Data Lakes across AWS, Azure, and GCP cloud platforms. The study focuses on migrating metadata using Data Vault data models, utilizing PySpark and SparkSQL for analysis. As big data environments grow in complexity, accurate metadata migration becomes crucial. This study explores best practices and automation tools for efficient metadata migration in large-scale environments. The research evaluates unique features of AWS, Azure, and GCP, including data storage, processing, security, and cost-effectiveness. It also assesses scalability and usability for managing big data in Cloudera Data Lakes with Data Vault data models. Findings show that AWS offers extensive services and tools, while Azure and GCP provide cost-effective options. AWS benefits from a large partner and developer network, aiding in managing big data in Cloudera Data Lakes with Data Vault models. This study provides innovative insights into metadata framework design and the capabilities of AWS, Azure, and GCP for big data management in Cloudera Data Lakes, aiding organizations in selecting the appropriate cloud platform.
Article Details
Unique Paper ID: 159872

Publication Volume & Issue: Volume 9, Issue 12

Page(s): 536 - 551
Article Preview & Download


Share This Article

Join our RMS



Conference Alert

NCSEM 2024

National Conference on Sustainable Engineering and Management - 2024

Last Date: 15th March 2024

Latest Publication

Call For Paper

Volume 11 Issue 1

Last Date for paper submitting for Latest Issue is 25 June 2024

About Us

IJIRT.org enables door in research by providing high quality research articles in open access market.

Send us any query related to your research on editor@ijirt.org

Social Media

Google Verified Reviews

Contact Details

Telephone:6351679790
Email: editor@ijirt.org
Website: ijirt.org

Policies

Important Links

Browse Rsearch papers