HOW TO GET YOUR PAPER ACCEPTED AT OOPSLA (reprinted from the OOPSLA '91 Proceedings, Appendix, p. 359-363) OOPSLA'96 Call for Student Volunteers INTRODUCTION The primary goal of the program committee of a major conference like OOPSLA is to put together a high quality program of technical papers. The program is created by soliciting contributed papers from authors and then selecting the best papers according to some criteria. The process is reactive: the authors submit, the committee selects from those submissions. Because of time constraints, it is generally not possible for the committee to interact with an author to improve the quality of a paper that fails to meet the acceptance criteria in its submitted form. Although the reviewers may offer helpful comments to the author of a rejected paper, the author must find some other forum for the paper (hopefully in an improved form). As program chair for the 1991 OOPSLA conference, I decided to address this situation proactively by writing this article. The purpose of this article is to provide advice to prospective OOPSLA paper authors, based on my past experience. If you are a prospective author of an OOPSLA paper, I hope you will be able to use this advice to make improvements to your paper before submitting it to the program committee, thus improving its chances for acceptance. This article will also offer some suggestions to help you decide whether to submit a paper to OOPSLA at all. Members of the program committee spend a considerable amount of time reviewing a large number of papers. You should avoid unnecessarily adding to this workload, by not submitting a paper that is clearly inappropriate. CRITERIA The 1991 OOPSLA conference is looking for papers in two basic categories. Research papers are papers that describe work whose purpose is to advance the state of the art of object technology. Experience papers are papers that describe work in which object technology has been applied for some purpose. The evaluation criteria for these papers are somewhat different; these differences are noted in the remaining discussion. Tutorial papers and surveys are not solicited for the 1991 conference; they should not be submitted. (This policy could change for future conferences; check the call for papers.) The primary requirement is that your paper must contribute to our understanding of object technology. It must have something new to say, and its message must be of sufficient importance and interest to warrant the attention of the OOPSLA community. If your paper is a research paper, it should describe a new idea or a new technique. It must describe original work. Your paper should present supporting evidence, not just conjecture. Idea papers should be backed up by a convincing analysis. If your paper is an experience paper, it should present new data based on actual experience that demonstrates the effectiveness (or ineffectiveness) of object technology, describes problems encountered, and makes suggestions for improvements. It should provide new evidence, either positive or negative, to evaluate existing ideas or techniques. It should help provide direction for future research, as well as provide new insights of value to other practitioners using object technology. The distinction between research and experience papers is not absolute. It is possible to gain experience in a research setting, and it is possible to develop new ideas or techniques while applying a technology. However, if you focus on one of these two aspects, you will write a better paper. Your paper should make its contribution abundantly clear. It is not the job of the program committee to ferret out this information. One aspect of identifying the contribution is to cite and make appropriate comparisons with previous work. A research paper should compare and contrast the work with prior work, demonstrating novelty. An experience paper should compare its results with other papers that present similar or opposing data. An experience paper that merely confirms that which is well known has opposed to that which is widely believed) is of little value. Obviously, your paper must not have been published elsewhere in the same or similar form. Less obviously, your paper must not be under consideration elsewhere in the same or similar form. While the desire for authors to "hedge their bets" by submitting the same or similar paper to multiple conferences and/or journals is understandable, this practice places an unnecessary burden on the peer review process. It can also cause embarrassment or confusion if the same paper is accepted at more than one conference. If you believe that unusual circumstances warrant simultaneous submission to more than one forum, you MUST notify ALL the program chairs and editors involved; simultaneous submission without notice is considered highly unethical. (Note that archival journals frequently accept versions of papers that have already appeared in conferences, but usually it is in your best interest to get feedback from the conference presentation before submitting a polished and revised version to a journal.) It is also considered inappropriate to submit multiple papers to the same conference that cover substantially similar or overlapping material. (It may be appropriate, however, for multiple papers to be submitted concerning distinct aspects of a single project, particularly if the various papers have different authors.) If you have published previous papers on the same subject, your paper should clearly indicate the relationship between your new paper and the previous papers. Finally, your paper must be well written. It must clearly communicate its message to the intended audience. PURPOSE You should answer some key questions before submitting a paper: The first question you should answer is "Why am I writing this paper?" If the answer is of the form "to document what I have been doing for the past two years", then you are in danger of writing a bad paper. It may be a shock to your pride, but most people are not interested in what you have done for the past two years. If you need to document your efforts, write a memo or a tech report. Another poor answer is "to help build my case for tenure". Tenure may be your initial motivation for writing a paper, but it should not be the only motivation. The purpose of your paper should be to communicate something to someone. So, the next questions are "What is my paper trying to say?" and '"Who is the audience for my paper?" If you cannot clearly answer these questions, then the paper is likely to be poor. A focused paper is better than a scattered paper. Resist the temptation to describe every great idea you had while working on your project. Pick a primary message and communicate it well. After deciding what the paper is trying to say, the next question to answer is "Is it worth saying" Is it a new message, or just a rehash of an old message? Is the message of value, or potential value, or is it trivial? Is it conjecture, or have you demonstrated the soundness of your conclusions? After deciding who the audience is, the next question to answer is "'Is OOPSLA a good way to reach my audience?" Does my message have value to researchers and developers of object technology? Does my message have value to practitioners using object technology? IS IT PREMATURE? Many papers are rejected because they are "premature". This characterization means that the work appears to be interesting, but it has not progressed far enough to be worth reporting in a conference paper. The paper may have more conjectures or opinions than results. Perhaps there are ideas that look promising, but they have not been worked out in enough detail. Perhaps more analysis of the issues is needed. Perhaps the proposed technique sounds interesting, but its value cannot be determined until it has been implemented. An experience paper may be called premature if it offers conjectures about expected results rather than reporting observed results. For example, if you are just about to release a product, it is premature to claim improvements in software maintenance until the product undergoes a maintenance cycle. If you have just created a "reusable" library, it is premature to declare it reusable until actual reuse occurs. One area of difficulty is portability. Experience has shown that designing a portable system of any kind can be surprisingly difficult. One cannot convincingly claim to have designed a portable system unless the portability of that system has been tested in at least two environments. The decision to accept or reject a paper that is premature is a judgment call by the program committee. A committee may choose in some cases to accept a paper that presents early work of a profound or provocative nature. However, you should honestly evaluate the maturity of your work before deciding to submit a paper on it. You may be able to write a much stronger paper for next year's conference! IS IT SOUND? Your paper should convince the program committee that your work is technically correct. The burden of proof is on you, the author. It is not the job of the program committee to prove your paper incorrect. If the correctness of your work is in doubt, your paper will probably be rejected. Soundness of ideas or techniques can often be demonstrated by the depth and clarity of the analysis, or by reference to a working implementation. Questions of soundness often arise for papers that present algorithms or proofs (see the next two sections). ALGORITHMS It is self-evident that complex algorithms are difficult to get right. Garbage collection algorithms in particular have gained a degree of notoriety amongst members of previous OOPSLA program committees. In general, a description of a complex algorithm will not by itself be sufficient to convince the program committee that your algorithm is correct. The committee will be much more likely to accept your algorithm paper if it contains at least one of the following elements: a formal proof of correctness (but see the next section), reference to a working implementation, or (marginally) a formal analysis of its complexity. PROOFS A formal proof is of value only if it is convincing. While a reviewer may be able to spot an error in a faulty proof, one cannot expect a reviewer to validate a proof. Therefore, any sloppiness in the formalism is grounds for suspicion (and likely rejection of the paper). It is better to avoid formality than to misuse it. In addition to being convincing, a proof must prove something worth proving. It is not worth anyone's time to read a paper that proves an irrelevant result. Be careful about including a proof in an effort to make your paper more "prestigious". This approach may backfire, as a sloppy or unmotivated proof can easily cause a paper to be rejected that otherwise might have been accepted. GENERALITY The audience at OOPSLA includes people from many communities. You will improve your paper if you can address a broad audience, wherever possible. While specific examples can be given, the problem and solution should be presented in general terms. I will explore this question in the context of object-oriented programming languages, although it applies to other areas as well. Many papers submitted to OOPSLA are written in the context of a specific object-oriented programming language. In many cases, the papers are actually addressing more general issues. As a hypothetical example, suppose you want to write a paper on "extending the Foo programming language to support distributed computing". A paper on this subject could easily address many issues that are not specific to the Foo programming language, although some might be. Such general issues might include naming, storage management, concurrency, and distributed schema (type) management. Your paper would be much stronger, and much more likely to be accepted, if it addresses these issues in general terms wherever possible, using the Foo-specific work as examples. If your paper is inherently Foo-specific, then you should consider submitting it to a more appropriate forum, such as the Foo Conference, the Foo Journal, or a meeting of the Foo Users Group. A gray area arises if your paper deals with a distinctive aspect of the Foo language. A paper that can demonstrate the value (or disadvantage) of a language-specific feature could be of great interest to all language researchers. WHAT ABOUT APPLICATIONS? Suppose you've written an application using object technology. Should you submit a paper to OOPSLA about it? That depends. What is interesting about it? Is it the novelty of the application? That might not be sufficient. For example, a novel application for allowing users to find library books might better be reported at a conference on library technology. Is the novelty of the application directly enabled by using object technology? That could be a good OOPSLA paper! Is the interesting aspect your experience in using object technology? Then concentrate on reporting your experience; don't devote an undue amount of verbiage to describing the application itself. DON'T BE ISOLATED Object technology overlaps many established areas of computer science, such as programming languages, databases, and distributed computing. If you are writing a research paper, it is important that you be familiar with the larger area, and not isolate yourself to the narrower domain of object technology. For example, if you are writing a paper on object-oriented distributed computing, you should think of yourself as writing a paper on distributed computing (that happens to involve an "objects" approach). The fact that your approach is object-oriented does not excuse you from relating the work to the existing distributed computing literature. Similar comments could be made about many different areas. WRITING Effective communication is important for a successful paper. A paper has little value if its intended audience cannot understand it. An incomprehensible paper cannot even be reviewed. Most authors will benefit from having their paper reviewed by a skilled writer. If your native language is not English, you have an extra burden. If at all possible, try to have your paper reviewed by a native or fluent speaker of English. FINAL ADVICE In my experience, most papers are substantially improved by getting feedback from other people. Giving a talk to a small group is an excellent way to get feedback and to force your self to organize your thoughts. Also, have your paper reviewed by your colleagues before submitting it to OOPSLA. After all, you can change your paper based on their feedback; if the program committee rejects it, there is no second chance (for that conference). Do not hesitate to attach a cover letter to your paper if there is additional information that would be useful to the program committee. For example, if you have published previous papers on the same subject, you might attach a cover letter to explain in more detail the novel contributions of the paper you are submitting. Finally, as has been noted elsewhere [Wegman], the conference review process is necessarily imperfect. The reviewers operate under strict time constraints, and the committee must make quick decisions. A paper will not receive the careful attention that it would from a journal. Furthermore, the committee may need to satisfy other constraints in putting together a successful program. As a result, some good papers will be rejected. Authors should carefully consider any reviewer comments and get opinions from experienced colleagues before deciding whether to abandon the effort or to revise the paper and submit it elsewhere. Additional insight can be obtained from the excellent paper by Smith [Smith] that describes the review process from the point of view of the referee. ACKNOWLEDGMENTS This article has benefited from the collective experience of the current and five previous OOPSLA program committees. REFERENCES [Smith] Alan Jay Smith. "The Task of the Referee". Computer (April 1990),65-71. [Wegman] Mark N. Wegman. "What it's like to be a POPL Referee, or How to write an extended abstract so that it is more likely to be accepted". Sigplan Notices 21:5 (May 1986),91-95. SEE ALSO The OOPSLA'93 Panel on "How to Get a Paper Accepted at OOPSLA". Alan Snyder Hewlett-Packard Laboratories P.O. Box 10490 Palo Alto CA 94303-0969 Back to OOPSLA Homepage This information last updated by ayers@zti.com Mon Jan 1 12:05PM 1995