Daniel M. Berry

Cheriton School of Computer Science
University of Waterloo
Waterloo, ON, Canada

Abstract:

Context and Motivation
This talk notes the advanced state of the natural language (NL) processing art and considers four broad categories of tools for processing NL requirements documents. These tools are used in a variety of scenarios. The strength of a tool for a NL processing task is measured by its recall and precision.

Question/Problem
In some scenarios, for some tasks, any tool with less than 100% recall is not helpful and the user may be better off doing the task entirely manually.

Principal Ideas/Results
The talk suggests that perhaps a dumb tool doing an identifiable part of such a task may be better than an intelligent tool trying but failing in unidentifiable ways to do the entire task. It considers how recall; precision; a weighted F-measure; and a new measure, summarization, should be used to evaluate such a tool.

Contribution
Perhaps a new direction is needed in research for RE tools.

Joint work with Ricardo Gacitua, Pete Sawyer, and Sri Fatimah Tjong