Dynamic slot allocation for mapreduce workload
Author(s):
M.S.Harini Lakshmi, Fathima nisha S, Manjula D, A. Sheelavathi
Keywords:
Map reduces Hardtop, Fair Scheduler, Dynamic Scheduling, Slots allocation
Abstract
MapReduce is a popular parallel Computing paradigm, for large-scale data processing in cluster and data centers. However, the slot utilization can be low, especially when Hadoop Fair scheduler is used, due to the pre-allocation of slots among map and reduce tasks, and the order that map tasks followed by reduce task in a typical MapReduce environment. To address this problem, we propose to allow slots to be dynamically allocated to either map or reduce tasks depending on their actual requirement. Specifically, we have proposed two types of Dynamic Hadoop Fair scheduler (DHFS), for two different levels of fairness (i.e., cluster and pool level).The experimental results show performance significantly while guaranteeing the fairness.
Article Details
Unique Paper ID: 144281

Publication Volume & Issue: Volume 3, Issue 10

Page(s): 58 - 61
Article Preview & Download


Share This Article

Join our RMS

Conference Alert

NCSEM 2024

National Conference on Sustainable Engineering and Management - 2024

Last Date: 15th March 2024

Call For Paper

Volume 10 Issue 10

Last Date for paper submitting for March Issue is 25 June 2024

About Us

IJIRT.org enables door in research by providing high quality research articles in open access market.

Send us any query related to your research on editor@ijirt.org

Social Media

Google Verified Reviews