A Novel Approach of Fair Scheduling to Enhance Performance of Hadoop Distributed File System
Abstract:
Nowadays, big data is one of the most challenging issues for managing big amounts of data more effectively and efficiently. It widely used in E-commerce, social media, online business and such types of applications. Hadoop Distributed File System (HDFS) is one of the widely used frameworks which can easily handle and store large amounts of data set frequently. For HDFS job scheduling is more challenging because it plays an important role in time optimization in big data. For resolving this issue in this paper, we introduce a job scheduling algorithm which is more time efficient and accurate than existing fair job scheduling algorithm. We optimize the time cycle of fair scheduling by minimizing iteration. We have accelerated with the different number of jobs in the existing algorithm and proposed an algorithm for experimentally proving the time complexity and time measurement. It is observed that the proposed method is computationally efficient than the existing one and our algorithm has reduced the number of iterations and improved the time efficiency on average 26.719%.
Authors: Rubayet Hussain ; Mostafijur Rahman ; Khawja Imran Masud ; Sheikh Md Roky ; Md. Nasim Akhtar ; Tanjila Akter Tarin
Source: 2nd International Conference on Electrical, Computer and Communication Engineering, ECCE 2019
Link:
https://ieeexplore.ieee.org/abstract/document/8679252/authors#authors
