For the last few years I have been working for Consulting type companies.  At times it has been fun and others I wonder why I did this to myself.  Right now I'm out of my home town on a project and regretting the travel involved.  I do go home every weekend, but with the weather this time of year (December) in Colorado it can be a bit of a gamble.

Today there is a major snow storm in Colorado and I am wondering if I am going to be able to fly home tomorrow.  I will have to play the phone game with the airline and live on the "Standby" list for flights if my flight gets cancelled.  Oh joy!

So for anyone thinking about going into consulting/contracting you may have to travel for your work.  Just keep in mind that you will get "stuck" somewhere sometime, part of the personal price you pay for those "Consulting Dollars".

Recently I got burned by a job candidate, or more accurately by the unethical company that represented them.  In this case it was a company that had 'doctored' up the person's resume so that they would appear to be a highly qualified test professional.  The resume's front page in the Skills/Knowledge was loaded with the 'correct buzzwords' for the tools and technologies that caused it to land in my lap. 

I work for a consulting firm and we do work across the U.S., and we try to 'contract' local people if needed when we cannot get an FTE consultant to the client. We are a small shop so this does happen often.  Our internal recruiter contacted the person and setup a technical interview with me over the phone (we were not in the same state). When I spoke with this 'person' they were right on the money with knowledge and speaking ability, so they looked like a good candidate. I was ready to move forward, but wanted to check out a couple more candidates to cover my bases.

The next resume I got from our internal recruiter looked almost the same as the previous person I just spoke to. I did a side by side comparison and the first page had a knowledge section that was 95% the same between the two, there were only a few tweeks. I went back and talked with my recruiter to find out what was up. They replied it looked like the two candidates were probably from the same Contract Registry (do Corp. to Corp. contract) and that they had probably used a ringer for my phone interview.

I got fooled by a 'ringer' phone interviewing for a potential job candidate.  So I called the 'person' I talked with about an hour later. Guess what, not the same person (voice, speaking skills or knowledge)! Their resume and the other one immediately went into the trash can!
There are some places out there where they just doctor up the resume so they can get high on the hit list and then have a 'ringer' get the job for some dumb schmo!  I had heard about this bait and switch crap, but never thought I would get caught by it. I still want to find the company that did this and expose their fracking asses for this B.S. !!!!

Caveat emptor (buyer beware)

This last week (while I was out on vacation no less) Mercury Interactive was bought out by Hewlett Packard (HP), and is causing a lot of us in the QA/Testing profession to wonder what is going to happen next.  As Scott Barber said in his blog this week, we have seen some major players (Rational & Segue) in the Test Tool marketplace get acquired in the last couple of years and a couple of new entries (Microsoft (or a re-entry, anyone remember MS-Test?), Akimbi) that have caused quite a stir.  This is change, and whether we like it or not it does happen.  Sometimes for the better and other times not.  But this is what life is all about, change.  How we cope with it and learn to live with it.

This type of "Sea Change" has happened before, and before the Dot.com boom of the late 90's.  There have been other tools and companies that have come before and either merged, emerged, or died. Early on in the PC software world when DOS was dominant there were only a few test tools around (AutoTester, QES QA/Architect, etc.), and those companies all pretty much died off when the GUI market started to boom with Windows 3.0 and OS/2.  Companies like Mercury (XRunner & WinRunner), SQA (Robot), Segue (QA/Partner), Softbridge (Automated Test Facility or ATF), Compuware (QA/Run) and Microsoft (MS-Test) all took off and pushed us to the next generation of tools and automation.  Right before the Dot.com craze started some of the companies were bought out (SQA by Rational, Softbridge by Teradyne), sold off their products (Microsoft with MS-Test to Rational) or emerged as the main players (Mercury, Segue, Compuware, and Rational with acquisition of SQA).  This scared some of us then too.  I switched quickly to Windows and QA/Partner, when OS/2 started to go down in flames, after working with ATF for 4 years.  I learned other tools and survived.  I adapted to the change, and the last few years have been focused on the Mercury toolset.  Now I will need to adapt again possibly.  This is a change I will need to roll with.  Only time will tell what other changes are in store for us.

I just got my latest copy of Better Software magazine and found an article about the "worth" of testing (article is "Proving our Worth, Quantifying the Value of Testing" by Lee Copeland).  In it he discusses the "value" we as testers bring to the Software Development equation, and states that our main goal is to provide "information" to all the other parties associated to a project.  I agree totally with this statement.

As testers our main goal is to "find" things out about the system under test and to disemminate the information to the concerned parties on the project.  We are in essence the CSI of the software world; we use deductive reasoning and experiments (tests) to try to determine what is going on and what caused it to happen.  Then we tell the proper authorities about our findings.  As Lee stated in his article, the context and quality of our information is what lends value to the project and that is one of the factors that proves our worth in whole.

Which leads me to the premise of the title of this entry; we (testers) are in the middle of it all.  We are the central focal point, or "hub", on the software project wheel.  A lot of other groups revolve around us and rely on testing to "interconnect" them.  Information flows in and through the Testing/QA group during the project.  Now this may sound a bit self-absorbed and egotistical, but it is the truth.  And it is one of the reasons testing is one of the more politically hot areas on a project.  Testing/QA lives in the limelight and under the microscope, whether we like it or not.  Testing/QA is at the center of the process wheel, or eye of the storm as some would say, and we are responsible for holding together the rest of the groups via the information we provide.  If we are not good at doing that then the rest of the wheel is unstable and will break apart.  And even at times where testing is not responsible for something in the process we still get the blame and heat for it.  Price we pay for being in the middle of it all.

So in conjecture to the article, is it really worth all the headaches and pain associated to the work we do on the project and in the information we try to bring to the table?  Yeah, because we do a "worthy" job and provide "value" to the project.  And in some respects it is fun to be in the middle of it all.

I previously talked about a person's 'innate curiosity' and 'inquisitive nature', and how this trait lends itself to being in the QA/Testing field.  In response to this post I got a comment along the lines of "is this capability of a tester innate or learned?"  Which my response is going to be a non-committal 'well... yes and no'. 

Yes, it can be learned to a degree.  There are different lines of thought process that can be taught that relate to the tasks involved in testing work.  We experience this all the way through school in classes on Mathematics (Proof Theorums), hard sciences (Chemistry, Physics), Psychology, and Philosophy (Logic and Critical Thinking).  We learn how to 'pick apart' a system (be it biological or non-biological) and prove whether it works or not, why so, and how.  That is why I have met a lot of test professionals with non-CS degree's like Biology, Chemistry, Physics, Philosophy, and Psychology.  They were taught to be thinkers and researchers.  The skills were taught in these types of disciplines, thus it was a nurture type of situation.

Now... No it can't all be learned.  There has to be some innate mindset that needs to be present.  Personality characteristics are what help drive a person to go towards the disciplines that teach the additional skills and help refine them.  Speaking for myself I was always a curious child, I got into things (and a lot of trouble) and always enjoyed pulling things apart to see how they work (my fathers favorite lawnmower was a victim of this curiosity; got it apart alright, just couldn't get it back together correctly again ;-) ).  Also, I have a degree in Zoology and got into computers out of interest (and economic need).  I was always more interested in the analytical/mathematical side of the discipline (genetics, population biology, epidemiology) and didn't like the classification stuff (what is the difference between a bufo woodhousi and a rana pipiens, there both frogs and who cares).  So there was a 'natural' ability present in me.  When I first got into the software industry I was a programmer, doing a lot of maintenance work.  I debugged/tested and fixed other peoples code, and I naturally moved into the formal testing field quickly when an opportunity arose (I liked it more, and at the time saw it as a career move that would be a better fit for me).  Now I still like programming, I do a lot of automation work (functional and load/performance).  But I do it in combination with my overall work focus.  It is in my nature.

Now I know people will argue this, and I expect it.  Does it make me a better tester?  In my opinion it does because this line of work can be stressful and tedious, and it takes a certain personality and characteristics to be able to do it effectively.  I have met other 'testers' who didn't last long because they only learned the craft and didn't have a built in drive for it.  That has been my experience so far.

So is a good tester born or taught?  They are born (nature) and they are taught (nurture), which is the key factor.  As with anyone we all have natural talents, and it is how those talents are developed (through education) that define our capabilities.  As the saying goes "brains only gets you so far, the rest is hard work!"

In part 1 I talked about things such as self study, finding a mentor and taking a realistic approach to your work/job.  In part two I want to get more into the mind / thought processes employed when getting into testing.  This will talk about the 'inquisitive nature' a tester needs in order to perform the work.

An 'inquisitive nature' is one where a person looks at something and examines it to gain a better understanding of what it is and how it works.  Curiosity is another term used for this, and it can be used interchangeably here if you wish.  I'll stick with inquisitive for now.

People by nature are inquisitive; children are the best at this.  What is that?  Why does it do that?  How does it do that?  When will it do it?  Where will it do it?  These are basic cognitive questions that a child will ask.  These are the questions we as test professionals need to constantly ask, and answer.

What is that piece of functionality?  What does it do, and how?  Where does it do its function?  Where does it come from?  Why does it do this part of functionality at this time, a question of when.  When does it not do its functionality and why?  Who uses this functionality?  These are some of the basic questions of the system under test (SUT) for Who, What, Where, When, Why and How.  These may be easy, or not, questions to get answers for.  It is our job to formulate and get answers to these questions.  Because of that we need to also look at the questions and answers for the methods/processes we will use to get the answers of the SUT.

These questions will include; What information about the SUT do I need to learn in order to test it, where is the information at, who is the person that might know about the SUT, when will I need to get the information or perform the test itself, how will I perform the test, what am I trying to prove by doing the test, why should I test that component or not, etc. etc.  This is just a sample of the questions to be asked and answered.  Testing is about asking questions and getting answers to them.

But this all needs to be done with some sort of organization and plan.  And next time I will discuss that aspect of starting out in testing.  So come back later and let your 'inquisitive nature' get some answers to those questions.

One thing I have been interested in over the years is the question of determing when testing is done.  When enough testing has been completed and the product is ready to be shipped.  In the past I have used completion metrics such as Defect Resolution Rate (how fast are you fixing things and when are you fixing more than your finding) and Test Execution Status to aid in this task.  Test Execution Status can be the amount of coverage of the requirements (test cases covering requirements) and their pass/fail rate along with execution rate (tests planned and executed) of the tests.  This show the traditional growth curves when plotted.  It give you an idea of how much you have done.  Which helps to determine where you are at  in the cycle.

An interesting thing I have read about in relation to the Agile methods (SCRUM in particular) is the 'Burn Down Rate' which is used to show how much is left to do for 'stories' (requirments/use cases) over a specified time period.  It is a downward sloping curve that can used to say you have so much left until the cycle is complete.  Considering this is more in line with what we get asked (how much testing is left to do, and when will it be done by) by management it may be a more effective way of also showing how much testing is left.  Instead of stories you plug in the test cases for those stories, and you plot the trend of how fast you are 'burning down' the list of things to do.  This way you could say to management we have X percent of test cases left to execute and we look to have them done by Y date.  This is what they want to hear.

I have not done this on a project yet, but am looking forward to trying it out.  After reading this if you have implemented it please respond back with your experiences.  If not, then maybe this will inspire you to change your perception of test execution status reporting & monitoring, and to see if you can 'burn down' some of your testing.

In response to a thread over on QAForums I have to say the following:

Everything & everyone is a cost.  Just for argument sake, how am I as a tester not "making" a product.  I provide a function in the process of making a product, just as a Programmer/Developer does.  Thus, would that not be considered to helping to make that product.  I help to "produce" that item.  A business analyst helps to make the product by specifing requirements, a designer helps to make the product by fleshing out the detailed functionality to support the requirements, a developer helps to make the product by constructing the code to support the detailed functionality specifications, a tester then helps to make the product by validating the software, a documentation writer helps to make the product by creating user guides and help files, and a systems engineer helps to make the product by installing or manufacturing (creating the CD's) the software.

The point of view that a Developer is the only one who "makes" a product is pure Fracking BullShix!  Pure fracking conceit.  That is the main problem with the industry.  Even developers can be considered a cost, ever heard of Maintenance Programmers/Developers.  What are they making?  They are fixing and repairing an already "made" product.  Even then for developers "making" new software there is an associated cost, it's called a salary.

(And I am an ex-developer.  I've done both new development and maintenance work.  I switched to testing because it was more interesting work for me.  When I got in to this area it was a new frontier and that to me is fun.)

So am I just a Cost Center, am I really worth it?  Guess the company will find out when they ship the software without testing it and the client sends it back with a request for a full refund.

Quality is a perception to a large part.  We can talk about "Good Enough" software quality and other aspects for reasons for higher standards.  But it does all boil down to money.  How much you keep and make, basic economics.

As an example, when I was at Peter Norton Computing we (the manager and I) started the QA/Test group.  PNCI had been in business for 7 years prior and doing quite well without a QA/Test group (developers and others did their own testing).  But they had just lost their market dominance to Central Point Software (PC Tools).  PC Tools had more functionality and variety in its suite.  PNCI management decided to expand product functionality of the Norton Utilities and also expand product line (added Norton Utilities for Mac, Norton Backup, and Desktop for Windows, the Norton Anti-Virus was brought in by Symantec).  They recognized in order to maintain the already high level of 'quality' in the software that they needed to have a specialized group doing the testing and let development do pure dev. work.  They made the investment into the Testing/QA area and supported it fully.  Thus we started cranking out the software.  We implemented basic processes and test metrics to evaluate the 'quality' of the software and aid in decisions for releases.  We set certain 'quality' levels and once we attained them we released the product.  It wasn't 'perfect', but a good bit better than the competion.

This was all done to 'improve' the 'quality' of the software and allow the company to regain its market dominance.  Which it did, the products were winning PC Magazine Editor Choice awards left and right.  Typically one of the aspects they won on was due to product reliability & quality.  Their investment paid back in spades.

Our customers loved us for it, and bought a lot of product.  But at times when the perceived quality went down we got dinged hard for it.  And management made the changes to get things straightened out (didn't lay people off, but did other things to let Test and Dev. produce a solid product).

PNCI/Symantec made and kept alot of money because of this back then (89 to 92).  And the 'quality' of the products was high.  The market told us how much they valued functionality AND quality.

This is one case of management seeing the value of Testing/QA and how it could help them make money.  All during this period of time we (Test/QA) educated management on what was involved with Testing/QA, and they were smart enough to recognize the value and invest in it.

Jim

A few years back I was at a conference and attended a talk that opened my eyes to why the majority of my efforts in trying to get management to "Get It" in regards to the need for testing were failing.  It wasn't them, it was me.

I didn't understand the "language" I needed to be speaking to them with.  I was focused on the technical aspects and implications for the project.  Instead I needed to be focused on the economic implications and impacts to the company.  Management deals in money, dollars and cents for you in the U.S., and how much they will make and/or keep from a project/product.

It comes down to "selling" Testing/QA as a "Cost Containment" mechanism (not Cost Savings, that sets a false expectation).  The cost containment comes in checks and balances to make sure the money being spent is done so properly and that if there is any leakage (rework) that it is kept under control.  Project cost leakage (rework) is one of the top reasons why projects go over budget and over time. 

Probably the best tool to use to sell management on Testing/QA is the Cost to Fix curve.  Basically the curve shows that the earlier a defect is found and fixed the lower the cost.  But here in lies part of the sales job problem, most people only go that far.  What needs to be done is to show the relationship between when a defect is introduced and when it is found/fixed and the associated cost with the rework of the system (a cumulative effect).  This definitively shows the relationship in economic terms that will catch managements attention. 

For example, a defect that is introduced in the Requirements phase and not fixed/found until the Testing phase is expensive to repair (longer time in between) and the amount of rework to remedy the defect compounds the overall cost (initial cost + rework cost  = total cost).  Now if a defect is introduced in the Requirements phase and found/fixed in the Design/Coding phase the cost is lower (flatter section of the curve) and its rework cost is now minimized (containing the leakage).  Even if the defect was introduced during the final stages of Coding and found/fixed in the Testing phase the cost is minimized (a steeper part of the curve, but the cumulative effect is minimal) because the rework cost is minimized too.

Now this information can be shown to impact "hidden" money factors that affect the immedite project and the company in the long run.  This leads to impacts on the Profit & Loss (P&L), EBDCITA (Earnings Before Debits Credits Interest Taxes and Amortization), ROI (Return on Investment) and Revenue Stream (short term and long term).  These impacts can be hard dollars (over budget or not making revenue because product is not selling or cost of rework after release) or soft dollars (future business lost due to credibility issues).  This is what the presenter called getting into the GOOB zone (Going Out Of Business).  Showing this will definitely get managements attention.

So how does Testing/QA help to keep this all under control.  Easy, by providing the service of validation of the system (Testing, from the beginning) and monitoring of the project (QA) with review milestones.  This takes the form of the medical principle of "a pinch of prevention is worth more than a pound of cure".  In money terms this makes the statement that a small up front investment can provide a very large return in the end.  How to leverage that small amount of money to counter balance a large one by shifting the balance point.  This is what management understands. 

What Testing/QA needs to do is "sell" this aspect of our contribution to the project; to change the language we talk in to one that management understands and will "buy in" to.  As the old saying goes "Money talks, B.S. walks!"

For the new comers there are some things to learn when starting this line of work. 

First, it will be a continuing process of learning.  You will always be in the learning mode in your job, be it learning the technology or how the business works.  Be continually looking for new information on the subject of Testing, it will help you in your efforts because someone else may have already been there and you can learn from them.  Don't re-invent the wheel.  There are magazines, books and other items to read.  Read them!

Second, give yourself a break.  You will not be Joe Stud Tester on the first day.  This work is both craft and engineering, it will take time to get it. 

Third, get used to being in the spotlight under the microscope.  Being in a Testing position will put you at the center of things during a project.  Learn to deal with it and the things that come with it. 

Fourth, don't take it personally or make it personal.  It is your job to find things and point them out.  You will take a lot of heat for it, and don't dish it back out because it will only make things worse for you. 

Fifth, always be inquisitive and trying new things.  By being inquisite/curious and entraprenurial you will keep up the motivation to do the work.  Because at times it can be monotonous and a headache.

Understand that your work involves performing tests.  This includes knowing how to research a test (review specifications / use cases), write up the process (create a Test Plan and Test Cases, define needed data), setup the test (both the environment and the data needed to execute the test), execute the test (run the test and manage its execution), analyze the test (what worked and didn't work and why), report on the results (write defect reports and execution summary reports), and communicate the results of the test (publish your reports and talk to other people about them).

Learn what the technology is and how it works.  You don't have to know it in depth but understand the basics of what is going on.  Know how to get in and get 'dirty' with the technology.  It will help you be more effective and efficient.

Understand programming and how it is done.  You don't have to be a super programmer, but know how to code and why things are done a certain way in the code.  It will help you in your testing effort and later on with automation will allow you to create your own code properly.  It will allow you to understand how to hook the application under test and drive it with the test tool.

Find a mentor to learn from.  They will help you grow in your position and this line of work.  Again, don't re-invent the wheel.  You gain knowledge and experience by working with someone who has knowledge and experience.

More later....

I figured I'd better explain this one.  This came about as a tangent on the "Resistence is Futile, Individuality is Irrelevent" saying of the Borg in Star Trek: The Next Generation.  I came up with this when I met Scott Adams (creator of Dilbert) at a book signing in Denver.  He was promoting his book "Way of the Weasel" and told us if we wanted a saying included with his signing to write it on a sticky note along with the book.  I did it, and I put "Testing is Irrelevent, Shipping is Futile" on the sticky note.  I handed the book to him and he busted up laughing when he read the note.  Everyone else in the line looked at me strangely.

Scott (no we are not on first name basis) looked back at me after laughing and said "You must be in the industry".  Of which my reply was "Yep! And if you want to use it just some how give me a credit.  Oh, by the way could you draw me a Ratbert, he is the official tester for Dilbert."  Needless to say I just got a Dogbert (like everyone else) and his signature only.  Bummer.

But at least I have a claim to fame that I got Scott Adams to bust up laughing, and the book is still in my collection.

Jim

For most people they know the meaning of the title of this entry.  It comes from the Douglas Adams book "Hitchhikers Guide to the Galaxy", and is the answer for all things in the Universe.  A bit confusing to say the least, or is it.

This is what is the catching point about software testing.  The answer could very well be 42.  Software is written by Homo Sapiens (okay some of us do qualify as Cro-Magnon's) and as such we are fallable and it means that software is going to be flawed.  Hey, if it was perfect do you think I would have a job and be here writing this blather?!

Because of this we (the Tesitng community) exist and our job is to find the flaws in the system.  Some people call this pessimistic, I like to think it being optimistic.  I look forward to things going wrong.  I have always been an inquisitive person and my natural tendencies to question things has given me the innate skills to do this line of work.

My educational background is in the sciences, specifically Zoology (pronounced zo-ology, not zoo-ology), and my core studies were in classes that dealt with identifiying things and running experiments (labs) to prove/disprove a hypothesis.  Sound a little familiar?  I used the Emperical Methodology, or better now known as Scientific Methodology to conduct my lab experiments.  This involved researching the subject and gathering the requirements to perform the test, determining what my objective is (Hypothesis), designing my test and construction of the test environment, conducting the experiment, recording the actual results from the test, and writing up my findings and reporting on them.  This should definitely sound familiar.

So what does this have to do with Software Testing, practically everything.  For both Manual and Automated (Functional/Regression and Performance) testing we need to go through these steps/stages in order to do our work.  But a lot of times we forget to do them diligently due to time pressures.  This is what causes us to get into trouble.  Now I'm not saying be dogmatic about it, but you need to follow the guidelines in order to be effective in your work.  Process and Methodology are a good thing.

These are basic common sense things to do, and we need to do them.  This leads into the saying of  "using the KISS method, Keep It Simple Stupid!".  Which is saying use a little bit of common sense, or you might get a nasty surprise. 

Even when the answer is 42, or am I not making sense?

Jim