Every software project arises out of a business problem. Requirements-gathering and analysis try to identify the business problem to be solved and probable characteristic a software product needs to have as a solution to the business problem. Requirements are the foundation on which software is built. Gathering and managing requirements is one of the biggest challenges faced in a project, and a robust requirement management process is one of the stepping-stones to a successful project.
From the perspective of a tester, requirements are used to determine whether the system exhibits proper behavior for each test case. Thus the quality of a requirements specification has a great impact on the quality of the finished software. Because of this, a requirements specification should be complete, correct, consistent, and unambiguous. Otherwise, defects may remain undetected, resulting in the delivery of a faulty software product to the users.
In this article, I have attempted to bring together ideas on inspecting and analyzing requirements based on reading techniques as procedural approaches encompassing common checklist methods to achieve better defect detection rates as early as possible in the development lifecycle.
A reading technique is a sequence of steps guiding the individual analysis of a text in order to achieve the goals of a particular inspection task (e.g., to identify defects in a requirements document).
Reading technique approaches can be identified under:
A very common method is the checklist-based reading technique (CBR). The inspector is given a checklist consisting of several questions. These questions should be answered during the inspection. The checklist serves as a tracker of which parts are to be checked and what aspects the reader should think of. CBR provides more aid and advice to the inspectors than ad-hoc reading and is therefore a very common technique.
The scenario-based reading technique provides guidance to the inspector by presenting him a scenario which he sticks to. Usually scenarios focus on certain details or parts and not the whole document. Therefore, to cover the whole document, different scenarios are provided to each inspector. The underlying theory is that a focused inspector finds more defects than one who inspects the whole document – thus the teams’ effectiveness should be increased. Scenario-based reading is rather a family of inspection techniques than a concrete one. Defect-based and perspective-based reading techniques are the most popular members of the scenario-based family.
Through their work in the SEL, Victor R. Basili and Forrest Shull have evolved their understanding of reading technologies via experimentation. Their research efforts focus on the development of families of reading techniques, based on empirical evaluation.
Reading for analysis is aimed at answering the following question: Given a document, how do I assess various qualities and characteristics? To emphasize defect detection, the researchers referenced above have focused on the requirements phase for this purpose. They have generated two families of reading techniques, by creating operational scenarios which require the reader to first create an abstraction of the product, and then answer questions based on analyzing the abstraction with a particular emphasis or role that the reader assumes. Each reading technique in a family can be based upon a different abstraction and question set. The choice of abstraction and the types of questions may depend on the document being read, the problem history of the organization, or the goals of the organization.
The goal of DBR is to detect defects in a requirements document with focus on defect classes: data type inconsistency, incorrect functions, and ambiguity or missing information.
The goal of PBR is to detect defects in a requirements document with focus on product consumers or perspectives, e.g., reading from the perspective of the software designer, the tester, the end-user, the maintainer, the hardware engineer. The analysis questions are generated by focusing predominantly on various types of requirements errors (e.g., incorrect fact, omission, ambiguity, and inconsistency) by developing questions that can be used to discover those errors from the one perspective assumed by the reader of the document (e.g., the questions for the tester perspective lead the reader to discover those requirement errors that could be found by testing the final product).
The efficiency of PBR reading technique in detecting defects is determined by the scenarios. It could be argued that scenarios are another kind of checklist. A checklist is a set of questions or statements without any hint or procedural description of how to read a document.
For a given perspective (designer, tester, or customer), an operational procedure is created by first identifying an abstraction of the document (Abstraction Procedure: answering the question of what information should be important to the inspector). This is followed by a model of how the information can be used to detect defects (Use Procedure: deciding which quality aspects are of interest in the document and understanding how the abstraction addresses those aspects). The abstraction procedure gives the user the right information; the use procedure should provide a step-by-step way to check that information for defects.
There are several reasons why it is difficult to generate scenarios. There are several levels of abstraction in a requirement document (e.g. you can look at the whole system which is described by the requirement-document or you can only look at a function)
There exists no standard for documenting requirements. A requirement document can contain:
Within each role, several different methods or techniques for testing can be used.
We have to find, on the one hand, a set of questions and activities that can be used for a broad range of requirements documents and that has, on the other hand, enough detail that the reader knows what to do.
The questions that the inspector is asked about the abstraction correspond directly to the types of defects being sought. This requires an analysis of what types of issues are of most concern to the project and organization. For a requirements inspection, defects can fit into one or more category or class:
The effectiveness of PBR & DBR in comparison to less procedural review techniques have been assessed in a number of studies conducted by researchers in the U.S. The results found improved rates of defect detection for both individual reviewers and review teams for unfamiliar application domains. Two experiments conducted at the University of Maryland compared defect-based reading to ad hoc reading and checklist-oriented reading. The conclusions were:
The results of the experiments carried out by Basili & Shull on Perspective Based Reading encouraged the hypothesis:
