Software Process Improvement Benefits of Formal Inspections

2/27/97

Since the early 1970s a peer review technique called Formal Inspections (FIs) has been used to find defects in software work products. FIs were originally developed and implemented by Dr. Michael Fagan while employed at IBM. Since those early days of FIs, their use has become widespread within many organizations, both commercial and DoD: NASA, Hewlett-Packard, Inc., Jet Propulsion Lab, and Honeywell Bull to name a few. There are several different names used synonymously for FIs such as Software Inspections, Formal Technical Reviews, and Structured Peer Reviews. Regardless of the name, FIs may be defined as follows:

"A defined, documented, structured, and disciplined methodology of

finding defects in all software development products over the entire

software life cycle. Formal Inspections are conducted by a software

engineering team of 4-7 people assigned to specific roles with specific

tasks to be performed during six specific phases, documented on specific

forms, over a specific period, taking an average of 28-40 total staff hours."

The original purpose of FIs was the detection and elimination of defects and defect propagation in software work products, the earlier in the software life cycle the better. However, in addition to finding defects, numerous other benefits have been reported frequently and consistently by people from many organizations who have used the FI Process on a wide variety of software work products and software applications. Here is a description of those benefits:

1. Increases pride of authorship - human beings have a tendency to produce a better product or service if they know in advance that a group of their peers are going to look at it in detail and attempt to find defects in the artifacts associated with the product and/or service. Software professionals are no different.

2. Improves communications among all parties involved - one of the fundamental keys to success in all human endeavor is communication. The FI process requires, when followed, structured, planned, and supportive communications to find the defects in the targeted software work products.

3. Improves knowledge of software products - unlike the ordinary project scenario wherein each staff member has a specific range of technical exposure in line with the tasks they perform and the work products for which they are responsible, FIs require that each FI Team member learn about the work product being inspected. Each FI participating member of the technical staff ends up with a broader and deeper knowledge of the software work products which were formally inspected.

4. Provides a mechanism for highlighting and prioritizing candidate areas for enhancement (especially for existing systems) - most software work products have problem areas that appear consistently in related software work products and/or many times in themselves. These issues are prime candidates for setting priorities based upon the frequency found and criticality of the issues. This information is provided automatically just by doing FIs.

5. Improves staff morale - a sense of camaraderie and teamwork is usually developed by the realization that everyone is in this together and have to go through the same process. It is also individually rewarding for the authors of the work products to be able to dramatically increase the quality of their work products in a short period of time by having others find the problems.

6. Improves formal reviews - the process of doing FIs precipitates the recording of defect and effort spent information on the FI Announcement and Report Form. This information often is a great aid to presentations during formal reviews to sponsors and other interested parties. A project manager is able to show that doing FIs finds and resolves defects at the rate of about 1.5 staff hours per defect vs. the dozens (sometimes hundreds) of staff hours that are spent to find, report, process, design, implement, test, integrate, certify, and re-deliver software after it has been fielded or reached Operation Evaluation (OPEVAL). This Return On Investment (ROI) is typical when using the FI Process.

7. Provides a means for technology transfer to others - very often people on similar projects or even the same project do not know what other people are doing or learning. Everyone on a FI team learns about the technology and the processes being used to develop, maintain and integrate into the work products.

8. Provides on-the-job training of standards, technology, and organizational culture - when someone new comes onto a project one of the quickest ways to bring them up to speed is to put them on a FI Team. They quickly become exposed to and start learning about their colleagues, the software processes, and the technology of the project. It is so much more stimulating and productive than putting them in a room with a stack of documents and saying "Here, read these and let me know when you are done."

9. Accelerates convergence to project standards of quality - part of the "inspection package" of materials to use to do a FI are the project standards; e.g., design standards, coding standards, etc. The FI process recommends their usage and where they are inadequate or non-existent will soon become apparent by common defects that are found for which a standard or guideline to circumvent it is either missing or needs enhancement.

10. Provides visible commitment of project to software product quality - a project is either doing FIs or they are not. Those projects doing FIs are clearly demonstrating their commitment to produce quality because FIs completely satisfy the SEI CMM Peer Review Level 3 KPA.

11. Focuses multiple viewpoints for building the right product - synergism is defined in Webster's Dictionary as "The action of two or more substances, organs, or organisms to achieve an effect of which each is individually incapable." With the team members as the organisms, FIs are very synergistic. People are frequently amazed how they found something as a team they would have not discovered by themselves.

12. Improves statistical measurement of quality - there are a variety of metrics that may be collected to measure product quality. Defect density is one such measurement and FIs find and predict defect density extremely well. Honeywell Bull performs FIs on every work product the company produces until such time their defect density reaches a level so low that the law of diminishing returns applies. Their defect density measurement tracking process is driven by the data supplied from their FI results.

13. Provides accountability for staff to perform the review of software products - traditionally, a contractor completes the draft of a software deliverable and gives it to the Government for review and comment. The Government employees are usually too busy doing crisis management or responding to the crisis "de juror" that they can't find the quality time necessary to rigorously review a software work product such as a 150 page Software Requirements Spec (SRS). So when the due date comes for the Government to be done there is little or no real review and comment completed. The FI process, by its very nature, puts peer pressure on every FI Team member to do their part. Those that don't will be embarrassed because it becomes apparent to everyone else that they didn't pull their weight. Nothing even needs to be said, it is just obvious. Project leaders like this situation because they don't have to micromanage or badger their staff to do the review, it just happens due to the FI process.

14. Provides focused analysis for author(s) vs. serial complaints - under the traditional review cycle of "give me your comments" the software product author(s) get a sporadic and disjointed barrage of feedback which often contradicts itself from one reviewer to another. FIs eliminate that problem. All of the defects are identified at one time and consensus is reached for each one of them before being given to the author(s) for resolution. One author said "I'm so happy to have it (the work product review) over with because people are no longer streaming into my office every day with questions and comments. I couldn't get any work done but with FIs the whole process is done and over in a short period of time.

15. Assists in generating movement and progress in SPI - there is a large movement in the DOD to get the software crisis under control. The major impetus of that effort is for software development and maintenance activities to become more mature, in other words, improve their software development processes. The primary tool for appraising current maturity and targeting SPI is SEI's CMM made up of Key Process Areas. All of the CMM KPAs are interrelated and interdependent on one another in some fashion. The use of FIs not only satisfies the Level 3 Peer Review KPA, but it also points out, on a project by project basis, those key processes that are not present or not working well in the project. Once identified, some action is necessary to service the KPA interrelationships. For example, if a FI on a design document finds a defect in the requirements, what process is available to go back and fix the requirement defect and then implement all subsequent changes needed to resolve any defect propagation caused by the requirement defect. It is not the role of the FI Process to do that. It is Configuration Management's and Project Planning's role to address such a situation but if those KPAs are not well defined and implemented there will be a breakdown.

That's the 15 advantages. So you might ask, what are the disadvantages? None! (In this author's opinion.) However, there are some obstacles which are described as follows:

The use of FIs requires a cultural change from the crisis management style to a planning and risk management style. Thus the management chain must make a commitment by providing training to their technical software staff and allow them to practice FIs to make the FI process work for them. That is, the staff of each project has a different mix of skills, interests, knowledge, and personalities. A proactive commitment or buy in by the staff is necessary to successfully implement the FI process. Any process can be made to be ineffective or fail if the users of the process are against it. The FI Process is no different in this regard. The best way to achieve buy in is for the prospective team members to receive high quality training and do a couple FIs so they can experience the advantages for themselves. After that, the FI process will sell itself.

FIs are so well established in some software cultures, that there is a free newsletter and a web page for FIs available from the Software Inspections and Review Organization .

In 1989, when Stan Rifkin was a software process specialist at the Carnegie Mellon University (CMU) SEI, he used to tell new SEPGs that if an organization were going to do just one of the CMM 13 level 2 and 3 KPAs, he recommended it be Peer Reviews using the FI process.