Paper Title: Performance Guarantees for Web Server End-Systems: A Control Theoretical Approach

Author(s): Tarek F. Abdelzaher, Kang G. Shin, Nina Bhatti

1) Is the paper technically correct?
 [X] Yes
 [ ] Mostly (minor flaws, but mostly solid)
 [ ] No

2) Originality
 [X] Very good (very novel, trailblazing work)
 [ ] Good 
 [ ] Marginal (very incremental)
 [ ] Poor (little or nothing that is new)

3) Technical Depth
 [ ] Very good (comparable to best conference papers)
 [X] Good (comparable to typical conference papers)
 [ ] Marginal depth
 [ ] Little or no depth

4) Impact/Significance
 [ ] Very significant
 [X] Significant
 [ ] Marginal significance.
 [ ] Little or no significance.

5) Presentation
 [X] Very well written
 [ ] Generally well written
 [ ] Readable
 [ ] Needs considerable work
 [ ] Unacceptably bad

6) Overall Rating
 [ ] Strong accept (very high quality)
 [X] Accept (high quality - would argue for acceptance)
 [ ] Weak Accept (marginal, willing to accept but wouldn't argue for it)
 [ ] Weak Reject (marginal, probably reject)
 [ ] Reject (would argue for rejection)

7) Summary of the paper's main contribution and rationale
   for your recommendation. (1-2 paragraphs)

This paper presents a novel approach to Quality of Service (QoS) guarantees for web server systems using control theory. A control theoretic formulation of how to model web server systems for performance control is provided by detailing the design of sensors, actuators, feedback loops, and targets from the point of view of web server systems. The goal is to achieve pre-determined response time and throughput targets on a per class basis of workloads, while avoiding server overload. The QoS of each workload has it's guarantees met by providing performance isolation, service differentiation, and excess capacity sharing. A linear feedback control mechanism is used to regulate the system. The "actuator" takes the control variable m, representing the service level, and outputs the fraction of clients to be served at each level. The "feedback" measure is calculated from the "sensor", which is resource utilization. Resource utilization is based on the request rates, bandwidth, and the fraction of requests being admitted. The target utilization is a pre-determined value of 0.58 which is a result from previous work.

I recommend "Accept" based on the novelty and usefulness of the proposed approach and the quality of presentation of the paper.



8) List 1-3 strengths of the paper.  (1-2 sentences each,
identified as S1, S2, S3.)

S1. Novel and intuitive approach to QoS for web servers (Control Theory).

S2. Thorough discussion of design, implementation, and experimentation



9) List 1-3 weaknesses of the paper (1-2 sentences each,
identified as W1, W2, W3.)

W1. Proposed approach is for static-content systems. With the current trends in web technology, the approach will need to be extended for dynamic content (as mentioned in the paper) to be of significant use.

W2. Deployment of the proposed approach requires a lot of web server specific tuning, and relies heavily on the accuracy of the attained values. It seems that characterizing the "capacity" of a web server would be more difficult than the paper suggests.

W3. The paper assumes the web server has only a single bottleneck resource. The claim is that this is the case in "the great majority of cases", but it is not clear that this is a safe assumption in general.


10) Detailed comments for authors.

This is a very interesting and useful approach to performance guarantees for web server systems. The paper is well written and is thorough in it's discussions. A couple of notes:

- Should there be a distinction between "Capacity Planning" and "Capacity Verification"? The paper claims the ability to do capacity planning, however, it does not seem to have the ability to actually plan the required capacity given the workload (and web server) characteristics. Instead, the system verifies if capacity is sufficient for a given workload specification and web server system. These are not the same, as determining how much capacity to add (to an under-provisioned system) is still a problem left to the administrator.

- Is it really safe to assume that the web server has only a single bottleneck resource? It seems that in the general case, a web server may have different bottlenecks at different times, even for the same set of workloads (perhaps this is more of an issue in the dynamic-content case, something to consider if this work is to be extended in that direction).