This paper presents an investigation into the origin and breakdown of server-induced latency with the goal of understanding how latency-optimization techniques can be made more effective. Using two servers with different architectures, we analyze the latency behavior under various loads and find that a phenomenon we call {it service inversion} is responsible for much of the latency increase under load. We trace the roots of this problem to negative interactions between the server application and the locking and blocking mechanisms in the kernel. Using a modified server that avoids these problems, we demonstrate a qualitatively different change in the latency profiles, generating more than an order of magnitude reduction in latency.

We also find that locking and blocking in the operating system artificially increases the burstiness of the load presented to the application. By eliminating such delays, not only do we observe a much smoother level of activity at the server, but we also eliminate most of the motivation for connection scheduling. In effect, we show that connection scheduling opportunities at the application level appear to be largely artifact of the server implementation, rather than a broadly-applicable technique. Our measurements on the modified server show much lower service inversion, suggesting that when the coarse-grained blocking is removed, the existing scheduling inside the networking stack adequately performs fine-grained connection scheduling.