CS846 Paper Review Form - Winter 2012

Reviewer: Sharon Choy

Paper Title: Defining Domain-Specific Modeling Languages to Automate Product Derivation: Collected Experiences

Author(s): Tolvanen, J.-P. and Kelly S

1) Is the paper technically correct?
 [ ] Yes
 [X] Mostly (minor flaws, but mostly solid)
 [ ] No

2) Originality
 [ ] Very good (very novel, trailblazing work)
 [ ] Good 
 [X] Marginal (very incremental)
 [ ] Poor (little or nothing that is new)

3) Technical Depth
 [ ] Very good (comparable to best conference papers)
 [ ] Good (comparable to typical conference papers)
 [X] Marginal depth
 [ ] Little or no depth

4) Impact/Significance
 [ ] Very significant
 [X] Significant
 [ ] Marginal significance.
 [ ] Little or no significance.

5) Presentation
 [ ] Very well written
 [X] Generally well written
 [ ] Readable
 [ ] Needs considerable work
 [ ] Unacceptably bad

6) Overall Rating
 [ ] Strong accept (award quality)
 [X] Accept (high quality - would argue for acceptance)
 [ ] Weak Accept (borderline, but lean towards acceptance)
 [ ] Weak Reject (not sure why this paper was published)


7) Summary of the paper's main contribution and rationale for your recommendation. (1-2 paragraphs)


The main contribution of this paper is to identify and categorize  approaches used for defining DSM languages by analyzing cases in which DSM languages were created to support model-based software development (for the purposes of automating product variant creation).  The paper categorizes approaches into four main groups: a domain expert’s or a developer’s concepts, generation output, look and feel of the system built, and variability space.  The motivation for this work is that there are few studies that have investigated the process of language creation, refinement and evolution. Thus, the significance and contribution of this work is that it attempts to amalgamate different methods/approaches applied for DSM language creation.

The originality of the paper stems from collecting use-cases for DSM languages, and from outlining the advantages and disadvantages of these different approaches.  With regards to technical correctness, the paper does not explain why only four categories were used (e.g. could the categories provided been more granular?).  Though there is no new technical innovation, a survey work, such as this one, allows readers to determine which approach for creating a DSM language would be appropriate. Thus, the novelty of this paper is not the creation of a new invention, but rather the categorization and summary of existing approaches and works. Additionally, this paper is easy to read, and it provides figures for easy comprehension and uptake. Thus, individuals reading this paper can determine which approach to take when creating a DSM language.  Therefore, because of the presentation of this paper and the implications it may have on individual companies, a rating of accept is granted.


8) List 1-3 strengths of the paper.  (1-2 sentences each, identified as S1, S2, S3.)

S1.  The use of figures and examples in this paper allows for easy understanding and visualization of the different DSM language categories.

S2. The paper logically outlines DSM definitions and its categorization approach.  The organization of this paper was well done.


9) List 1-3 weaknesses of the paper (1-2 sentences each, identified as W1, W2, W3.)

W1.  The analysis of this paper could have been more in-depth as only a couple of sentences were provided in the analysis section for each approach.  Perhaps the paper could have discussed increase in productivity when using different approaches, or suggested certain approaches for certain problems.

W2.  Though the organization and writing of the paper was done well, I believe a chart (one that indicates the benefits, drawbacks, and potential use-cases) for each approach would allow for simpler comparison and be even easier to read.

W3.  I believe the abstract did not clearly outline the motivation for the paper (e.g. answering the question “what is the purpose for identifying approaches that are applied for defining languages that enable automated variant derivation”?).