CS848 Paper Review Form - Fall 2006

Paper Title: Managing the performance impact of administrative utilities

Author(s):

1) Is the paper technically correct?
 [X] Yes
 [ ] Mostly (minor flaws, but mostly solid)
 [ ] No

2) Originality
 [ ] Very good (very novel, trailblazing work)
 [X] Good 
 [ ] Marginal (very incremental)
 [ ] Poor (little or nothing that is new)

3) Technical Depth
 [ ] Very good (comparable to best conference papers)
 [X] Good (comparable to typical conference papers)
 [ ] Marginal depth
 [ ] Little or no depth

4) Impact/Significance
 [ ] Very significant
 [X] Significant
 [ ] Marginal significance.
 [ ] Little or no significance.

5) Presentation
 [X] Very well written
 [ ] Generally well written
 [ ] Readable
 [ ] Needs considerable work
 [ ] Unacceptably bad

6) Overall Rating
 [ ] Strong accept (very high quality)
 [X] Accept (high quality - would argue for acceptance)
 [ ] Weak Accept (marginal, willing to accept but wouldn't argue for it)
 [ ] Weak Reject (marginal, probably reject)
 [ ] Reject (would argue for rejection)

7) Summary of the paper's main contribution and rationale
   for your recommendation. (1-2 paragraphs)

This paper proposes a policy based throttling mechanism for controlling and 
restricting the performance impact of running administrative utilities on 
production systems within the specified degradation units. The paper 
introduces self-imposed sleep(SIS) as a mechanism to throttle the resource 
consumption of utilities and employs an online estimation scheme with a 
feedback loop to translate policies to appropriate throttling settings. 
An experimental evaluation showing the feasibility of such a mechanism 
is provided at the end.

This paper aims to address a very important issue faced by administrators 
in maintaining the integrity of real-world, high-availability production 
systems i.e. manageability. The proposed policy-based mechanism for regulating 
the execution of utilities can significantly reduce the management burden of 
the system.

8) List 1-3 strengths of the paper.  (1-2 sentences each,
identified as S1, S2, S3.)

S1: The proposed mechanism is simple yet effective and practical 
in nature; the work has been written and presented well.

S2: The solution provides a high-level interface for the administrators, 
making control fairly independent of specific performance metrics.

S3: The proposed solution can be implemented purely at the application level 
and thus does not require any changes to the OS.

S4: The applications of the presented architecture are not limited to DBMS systems.


9) List 1-3 weaknesses of the paper (1-2 sentences each,
identified as W1, W2, W3.)

W1: The solution can only be implemented into new utilities by developers or old 
utilities would have to be modified to make it work which may not be possible at 
all for legacy systems.

W2: The results presented are based on a single utility of a particular DBMS and 
thus provide no clue about the general usefulness of this approach.

10) Detailed comments for authors.

There can be many administrative utilities running on a system; the proposed solution 
calculates a single throttling value for all of them. As rightly hinted in the paper, 
this may not be an efficient solution. Many of the utilities will actually be system 
and workload dependent. Now if we try to calculate separate throttling values for each 
utility the obvious question would be that ‘Can the performance overhead of doing utility 
specific control calculations be justified?’ I think there is a need to extend the 
proposed design and the experimental study to answer such questions. In the last section, 
it has been mentioned that future work will include automation of determining controller 
parameters, since the space was not an issue in this paper, it would have been nice to 
actually see some thoughts/ideas that the authors might have at this point to tackle this 
problem.