Distributed Join-the-Idle-Queue for Low Latency Cloud Services

Abstract

Low latency is highly desirable for cloud services. To achieve low response time, stringent timing requirements are needed for task scheduling in a large-scale server farm spanning thousands of servers. In this paper, we conduct an in-depth analysis for distributed Join-the-Idle-Queue (JIQ), a promising new approximation of an idealized task-scheduling algorithm. In particular, we derive semi-closed form expressions for the delay performance of distributed JIQ, and we propose a new variant of distributed JIQ that offers clear advantages over alternative algorithms for large systems.

10 pages, 10 figures. This work is inspired by Michael Mitzenmacher's recent work on distributed Join-Idle-Queue (arXiv:1606.01833) as well as Alexander Stolyar's recent work on centralized Join-Idle-Queue (arXiv:1512.07873 and arXiv:1407.6343)