Please use this identifier to cite or link to this item: http://localhost:80/handle/Hannan/170396
Title: Efficient and fair scheduler of multiple resources for MapReduce system
Authors: Jianxin Liao;Lei Zhang;Tonghong Li;Jingyu Wang;Qi Qi
subject: data locality|efficient and dominant resource held time fairness scheduler|multiple resources|fair scheduler|EHTF scheduler|coarse-grained fairness doctrine|MapReduce system|resources utilisation
Year: 2016
Publisher: IEEE
Abstract: Scheduling tasks close to their data and optimising resources utilisation are both crucial for the efficiency of MapReduce system. On the other hand, there is a conflict between fairness and efficiency. In this study, an efficient and dominant resource held time fairness (EHTF) scheduler is presented, in which the efficient utilisation of resources, data locality and fairness are addressed simultaneously. In EHTF scheduler, the authors introduce the concept of `coarse-grained fairness' to improve the efficiency of MapReduce system. For each scheduling, several tasks from different jobs can be assigned to the free slot without violating the coarse-grained fairness doctrine. To determine the best task from these several tasks in each scheduling step, a score model is proposed by taking into consideration both resources utilisation and data locality. The authors describe the design and implementation of EHTF scheduler. The authors' experimental results show that EHTF achieves more fairness and better throughput than Fair and Quincy schedulers.
URI: http://localhost/handle/Hannan/170396
ISSN: 1751-8806
1751-8814
volume: 10
issue: 6
More Information: 182
188
Appears in Collections:2016

Files in This Item:
File Description SizeFormat 
7775138.pdf844.85 kBAdobe PDFThumbnail
Preview File
Title: Efficient and fair scheduler of multiple resources for MapReduce system
Authors: Jianxin Liao;Lei Zhang;Tonghong Li;Jingyu Wang;Qi Qi
subject: data locality|efficient and dominant resource held time fairness scheduler|multiple resources|fair scheduler|EHTF scheduler|coarse-grained fairness doctrine|MapReduce system|resources utilisation
Year: 2016
Publisher: IEEE
Abstract: Scheduling tasks close to their data and optimising resources utilisation are both crucial for the efficiency of MapReduce system. On the other hand, there is a conflict between fairness and efficiency. In this study, an efficient and dominant resource held time fairness (EHTF) scheduler is presented, in which the efficient utilisation of resources, data locality and fairness are addressed simultaneously. In EHTF scheduler, the authors introduce the concept of `coarse-grained fairness' to improve the efficiency of MapReduce system. For each scheduling, several tasks from different jobs can be assigned to the free slot without violating the coarse-grained fairness doctrine. To determine the best task from these several tasks in each scheduling step, a score model is proposed by taking into consideration both resources utilisation and data locality. The authors describe the design and implementation of EHTF scheduler. The authors' experimental results show that EHTF achieves more fairness and better throughput than Fair and Quincy schedulers.
URI: http://localhost/handle/Hannan/170396
ISSN: 1751-8806
1751-8814
volume: 10
issue: 6
More Information: 182
188
Appears in Collections:2016

Files in This Item:
File Description SizeFormat 
7775138.pdf844.85 kBAdobe PDFThumbnail
Preview File
Title: Efficient and fair scheduler of multiple resources for MapReduce system
Authors: Jianxin Liao;Lei Zhang;Tonghong Li;Jingyu Wang;Qi Qi
subject: data locality|efficient and dominant resource held time fairness scheduler|multiple resources|fair scheduler|EHTF scheduler|coarse-grained fairness doctrine|MapReduce system|resources utilisation
Year: 2016
Publisher: IEEE
Abstract: Scheduling tasks close to their data and optimising resources utilisation are both crucial for the efficiency of MapReduce system. On the other hand, there is a conflict between fairness and efficiency. In this study, an efficient and dominant resource held time fairness (EHTF) scheduler is presented, in which the efficient utilisation of resources, data locality and fairness are addressed simultaneously. In EHTF scheduler, the authors introduce the concept of `coarse-grained fairness' to improve the efficiency of MapReduce system. For each scheduling, several tasks from different jobs can be assigned to the free slot without violating the coarse-grained fairness doctrine. To determine the best task from these several tasks in each scheduling step, a score model is proposed by taking into consideration both resources utilisation and data locality. The authors describe the design and implementation of EHTF scheduler. The authors' experimental results show that EHTF achieves more fairness and better throughput than Fair and Quincy schedulers.
URI: http://localhost/handle/Hannan/170396
ISSN: 1751-8806
1751-8814
volume: 10
issue: 6
More Information: 182
188
Appears in Collections:2016

Files in This Item:
File Description SizeFormat 
7775138.pdf844.85 kBAdobe PDFThumbnail
Preview File