During the SDLC, the Software Development Life Cycle, time and resources need to be committed to application testing.  This is the testing that is done by the testing group, (i.e. the Black Box Testers.)  The allocation of time and resources is always a problem as some organizations aren’t quite sure how much resources should be assigned.  So the question that I often hear in my consulting and teaching assignments is Where Does Testing Belong?  Some say at the beginning, others say at the end.  Some even say nowhere as it doesn’t really add anything to the products reliability or success. After all, the developers have done the White Box Testing, so no additional testing needs to be done.  Nothing could be further from the truth.  Applications’ testing is one of the most misunderstood contributions to the success of the project and corporation. 

The need for Applications Testing (Black Box Testing)

Application Testing is not always a part of the project team, is not invited to the meetings to ensure everyone understands what the expectations and deliverables will be.  Yet without this involvement how can the test group effectively perform their jobs?

I have been the recipient of the following conversation.

“I need you to test this product but there is no hurry it isn’t going out until Friday.”

Question:  “What is it supposed to do?”

Answer: “Don’t worry about it, just test it.”

Question:  “If I don’t know what it is supposed to do, how will I be able to ensure that it will do what it was intended to do?”

Answer: “If you need to know what it is supposed to do, ask the programmer.”

Sound Familiar?

Ever been on the receiving end of that conversation?  Over ninety percent of programmers have never been to a customer’s site, have never seen the program run in the customer’s environment, yet will design, develop and ship the product.  Why, “We have always done it that way.”  How about this one – “It works on my machine?”  It may work on the developer’s machine, but if it doesn’t work on the deliverable machine it is of little value.

Few programmers speak the customer’s language and therefore, from the very beginning, there is this lack of understanding and communication with the customer to ensure everyone is on the right track. Without a complete understanding of what is required, each person could have their own concept of what they think the client meant to say.  There is an old saying, “I know you understood what I said.  What you fail to realize is that what I said is not what I meant.”  Sounds like double talk doesn’t it?  Yet we often fail to have a good understanding before we start the project.  We are sometimes in too much of a rush to do the “HOW” before we really understand the “WHAT.”  Putting the cart before the horse?

Testing is an integral part of the project

Testing needs to be just as involved as any other group in the project and throughout the entire SDLC.  It sometimes may take longer to write a test case than it did for the programmer to write the code.  When will the test group get the time to develop a Test Plan, write test scripts and test cases?  Where will the test group get the information on the customer’s needs? This information does not just come out of thin air.  Good tests require careful planning and a good understanding of the customer’s expectations.  How can this be accomplished if testing is not involved from the very beginning? In fact some organizations are developing code from test cases as opposed to the other way around.  If you have a better understanding of what is needed, you will have a better chance of achieving those objectives easier.  This will result in a savings of development costs, and in the long run, maintenance costs as well.  And we all want to do better with less.

Differences between Quality Control and Quality Assurance

Too often the job responsibilities of Quality Control and Quality Assurance are confusing.  Most organizations put the two responsibilities together and have a “QA/Tester” title.  This is a bad thing to do because they are two distinct and different jobs.  So what are the responsibilities of Quality Control and Quality Assurance? 

Quality Control - QC - Testers are there to demonstrate that the product will meet the user’s needs.  Not to demonstrate the program runs.  A program can be “right” meaning that it runs as designed, yet not be the “right program” meaning it doesn’t meet the user’s needs. Black Box testers are Quality Control.

What does a tester do when they encounter a defect?  Log it, write a problem report and move on. But unless someone performs a “ROOT CAUSE ANALYSIS”, we are doomed to continue to make the same mistakes.  A Root Cause Analysis identifies why the problem occurred in the first place and how it be prevented in the future. 

So who does the Root Cause Analysis?

Quality Assurance – QA - is chartered to ensure that processes have been defined, documented, and will be repeated each time.  One of the major jobs of the QA group is “lessons learned.”  How can we get better if we don’t learn from previous successes and or failures?   When do you want to do this?  As you go along, or wait and repeat the error over and over again as we continue.  “Lessons learned” should be an on-going process and improve the quality of this product as well as future products.  The sooner we can find out why the problem occurred, the cheaper it will be to repair and prevent in the future.  What costs more, fixing it now and preventing it from repeating or let it happen and fix it and fix it and fix it etc.?  Lessons learned are an ongoing process.  During the entire life cycle of development and maintenance we need to continuously monitor what is good and bad.  Repeat good things, and prevent reoccurrences of bad things.

Opposition to process

Ever heard the comment, we don’t have the time or money to do it right now, we will fix it when it comes back  What is the guarantee it will come back, and if it does it is will cost a lot more to fix than it would have cost to prevent.  Additionally, what about your image?  Do your customers want a buggy product, or would they like to have it correct the first time.  Also, what about the competition?  Do you think they might try to capitalize on the fact you send out buggy product?

If we leave testing to be at the end of the development cycle and we uncover problems, we may have to change some of the things that were previously tested.  Once we make changes we must retest – “Regression Testing.”  When that happens we often encounter the “Fix one – Break two” problem.  As we fix one error we create additional errors.  The size of the fix has nothing to do with the regression testing process.  You touch it – you must regression test.  The success of testing is not in the quantity of testing we do, but in the quality of the testing.

Development and Maintenance cost savings

The amount of time available for testing will be shorter depending on where in the product cycle testing is involved.  It will also affect the cost of the project resulting in delivering less than we originally planned to do.

Many of these problems can be prevented.  Prevention is much cheaper than cure.  But to achieve this success we need to improve the way things are currently being done.  A better and clearer requirements process is the beginning of the improvement.  A quality product will result when Software Testing is more involved with the team members, improved communication, documented and enforced standards, and a clearer understanding of each person’s responsibilities and contributions.   A Needs Analysis, i.e. separating the needs from the wants, accompanied with structured Walk-thrus, prior to the designing the code is essential to provide a quality product up front.  Minimize Scope Creep.  Design and develop more Testable Requirements prior to the design.  Testers will ask the types of questions relative to the product’s use and customer’s expectations.  This information is essential to be done before the design, not after.

Early tester involvement is crucial to the overall success of the project

Testing is an integral part of the project and as such needs to be more involved and supported.  When it comes to testing, earlier is better.  Meeting the customer’s needs is what testing is all about, not just putting out code.  Testing is a major contributor to the overall success of the project and we cannot accomplish this when most testers inherit whatever anyone else decides to give them.  The expectations are too high and cannot be achieved.