Servers--for the web, databases, mail, files, or many other resources--are commonly structured as a collection of parallel tasks, each of which runs code to process a request. Threads, processes, or event handlers underlie these systems' software architecture because these mechanisms provide the control independence and dynamic scheduling to hide high latency operations such as I/O and communication. Unfortunately, many servers run poorly on modern processors. Considerable empirical evidence shows that these systems run at a fraction of processors' potential performance. In part, poor performance is attributable to this software architecture, which frequently switches between unrelated pieces of code, thereby destroying the program locality that is a prerequisite for effective caches, TLB, and branch predictors.

This work propose a software solution to this problem. Cohort Scheduling is a technique that increases code and data locality by consecutively executing logically related operations across different server requests. To support this technique, we developed a programming model, called Staged Computation, that offers an abstraction to group related operations and mechanisms to implement this scheduling policy. These techniques have been implemented in a library called StagedServer. Measurements on realistic systems show that Cohort Scheduling can improve both processor performance and server throughput. (Joint work with Michael Parkes.)