|
||
"Committed to Excellence in Software Testing"
Growth of maturity in the testing process Rodger Drabick 27 October 1999 In the early days of software, before “software engineering” was a gleam in the eyes of Dr. Roger Pressman and others, testing was generally an an-hoc activity performed at the end of the development life cycle. Effective testers (no test engineers in THOSE days) were generally skilled in “error guessing”. Then along came the 1970s; Glenford Myers published his seminal book, “The Art of Software Testing” in 1979. Though Myers described a number of excellent techniques for testing, along with his major paradigm shift of “test to FAIL, not to prove the software works”, testing was still considered an ART. Testing gurus were emerging, and suggesting the concept of developing a Test Plan at the FRONT of the life cycle, based on requirements. In 1972, Dr. Bill Hetzel convened the first formal conference on Software Testing at the University of North Carolina. In 1984, the Data Processing Management Association convened an annual software testing conference (in cooperation with the US Professional Development Institute) which continues to this day as the Testing Computer Software Conference. Bill Perry’s Quality Assurance Institute hosts a yearly testing conference (The International Software Testing Conference), and Dr. Edward Miller’s Software Research, Inc. hosts the yearly Quality Week (dedicated to software and system testing). Drs. Gelperin and Hetzel coordinate the yearly Software Testing Analysis and Review (STAR) conference both in the US and abroad. And there are other significant conferences that I have not mentioned. In 1981, Dr. “Bill” Hetzel was teaching a public seminar titled “Structured Software Testing”. Later his book, with the tongue-in-cheek title “The Complete Guide to Software Testing” was published. In June 1988, Drs. David Gelperin and Bill Hetzel broke more ground with their article in Communications of the ACM entitled “The Growth of Software Testing”. In this article they outlined the Systematic Testing and Evaluation Process (STEP), which was based on the concept of an early requirements-based Test Plan. STEP “defines a system of testing tasks, products, and roles for the consistent and cost-effective achievement of test objectives.” The authors even advanced the thought that development of the Test Plan (and communication of it’s contents to the development staff) was so significant that if schedule forced truncation of testing, the Test Plan alone could significantly reduce defects by giving the developers an objective perspective into what their system was intended to do. Since Glenford Myers book, many other valuable reference texts on software and system testing have become available; one has only to consult the pages of the Single Source catalog from Software Quality Engineering to see examples of these. Distinguished authors include, but are not limited to, Boris Beizer, Chin Kuei Cho, Vern Crandall, Michael Deutsch, Bill Hetzel. Bill Perry, Gordon Schulmeyer. Also, since the early days of ad-hoc testing, testing has found a home of its own in many organizations. Whereas testing was primarily performed by programmers as a sort of secondary function in the early 70’s, in the late 70’s and into the mid-80’s responsibility for testing resided within a Quality Assurance or Software Quality Assurance group. More recently, the focus of Software Quality Assurance groups has changed into more of a process monitoring function (including monitoring of testing), and separate Testing groups are proliferating. In the early days of software testing, there was little or no support available to the tester in the way of automated tools. CASE tools proliferated for the development staff, but we testers were forced to continue on with manual methods. Since the mid-80’s, testing tools have been growing in both quantity and quality. Issues of quality and testing magazines (e.g., Software QA Magazine, and Software Testing and Quality Engineering magazine) have provided us with a catalog of tools, ranging from coverage analyzers to test planning and test design aids that can help us do our jobs more efficiently and more cost effectively. Recently, other developments seem to reflect the emerging maturity of software and system testing. I have published articles on modeling the Formal Software Process in Software QA Magazine. This work, based on principles outlined by Watts Humprey in his book “Managing the Software Process” was first briefed at the Quality Week conference in 1992. The complete model was most recently briefed at the Testing Computer Software Conference in Washington, DC in 1996. In May 1996, David Gelperin and Aldin Hayashi published an article in Application Development Trends entitled “How to Support Better Software Testing”. This article described a Testability Maturity Model for evaluating the testing process. Good things seem to cluster, because in that same period, Dr. Illene Burnstein, Dr. C. Robert Carlson, and Taratip Suwannasart made a presentation on “Development of a Testing Maturity Model” at the 1996 Quality Week conference. Simultaneously, Susan Burgess and I were developing the Testing Capability Maturity Model for use in assessing the Maturity of the Testing Process at a large federal agency. These three maturity models for testing share the common background that they have been developed based on concepts established in the Capability Maturity Model (CMM) for software engineering from the Software Engineering Institute at Carnegie Mellon University. The maturity models for testing are all designed to complement the CMM. The prime purpose of the several testing maturity models is to provide a foundation for testing process improvement. Interestingly enough, the CMM with its five maturity levels and seventeen Key Process Areas (KPAs) doesn’t devote a single KPA to testing, though it has KPAs for Software Quality Assurance and Software Configuration Management identified within its Maturity Level 2. The Gelperin/Hayashi Testability Maturity Model has three “levels”:
The Testability Maturity Model has six Key Support Areas (KSAs):
The KSAs appear to span the model, and are not specifically related to a particular level, in contrast to the CMM and the other two models. In a telephone conversation with Dr. Gelperin in 1997, he noted that he has changed his perspective. He views his Testability Maturity Model as a “Support Model” rather than a “Maturity Model”. He now makes the following distinction. A Test Maturity Model (TMM) evaluates a group’s capacity to perform its mission, while a Testability Support Model (TSM) evaluates the degree of support provided to the test group by the environment in which that group is embedded. Therefore his model can be seen to evaluate the reality outside of test rather than the maturity of the test function itself. Obviously, successful testing depends on both support and maturity. We may see another article from Dr. Gelperin in the future that better defines these two models and their relationship. The Testing Maturity Model from Drs. Burnstein and Carlson, and Ph.D. candidate Suwannasart has five maturity levels, like the CMM. However, the levels are defined “differently”.
The Testing Maturity Model has a set of “Maturity Goals” associated with Levels 2 through 5:
The Burgess/Drabick I.T.I. Testing Capability Maturity Model (TCMM) has five maturity levels, like the CMM:
Levels 2 through 5 have a set of seventeen Key Process Areas (KPAs):
The TCMM has an associated checklist of assessment questions, a set of open-ended interview questions to use subsequent to administration of the assessment questionnaire, and a spreadsheet (with associated histograms) to aid in processing the assessment data. Thus we see that Testing has progressed from a primarily ad-hoc activity performed by ex-programmers near the end of the development life cycle into a discipline that spans the software development lifecycle, with dedicated conferences, many fine source books, with defined process models and methods of assessing process maturity. Though our maturity models have yet to converge into a single model with the rigor and reputation of the SEI CMM, we can hope that this will occur in the foreseeable future. Unfortunately, though the maturity of testing and many testing organizations had made significant progress in the last ten years, I’m afraid I cannot say the same for the majority of project managers. Even though techniques in Project Management are well defined, my recent experience suggests that management by crisis is still the most common method of operation. Even mature testing organizations can only do so much, when no integrated plans or schedules (showing the interrelationship of tasks from the entire project organization) are generated for the project. My thanks to my long time friend, Dr. David Gelperin, for providing comments relative to his Support Model. For those of us in the Testing field, the future is full of promise. Significant challenges remain ahead of us, as new systems are deployed in the new millenium ============================================================================== AUTHOR PROFILE Rodger Drabick is a nationally recognized expert in software process engineering, software quality assurance and testing management with over 35 years experience. Rodger can be reached on xkejaguar@aol.com ============================================================================== |
||