2008-Aug-27 - Let the crowd in....

I was reading James Whittaker's posts on  Crowd Sourcing at http://blogs.msdn.com/james_whittaker/archive/2008/08/20/the-future-of-software-testing-part-1.aspx, where he sees the next big leap to be one where everyone who wants to can be a part of the Testing Solution.  While I'm not quite sure I agree with it, and some of the quotes reflect my own opinion, I can see where it can be of benefit as an additional avenue of allowing people time and a chance to test software and submit their issues to be fixed.  This is a good thing, most people, as he says, do find problems not always found in a test cycle either because of contraints, or some quirk of the User Environment that is not mimiced in the lab; getting those issues in early is a good thing.  However, I am a realist, and while its nice to get many of these bugs in I personally don't see where the value is going to be if those bugs are not being resolved but end up in the bug bucket waiting to be looked at.  Though I am getting ahead of myself, there were a couple of points in this I thought was interesting.

The Cloud.  I'm still unclear on what this is, or whether its just an anticipation to what Cloud Computing is going to be, but I'm not expecting people to be sharing configurations and environments across the Net.  In a QA lab you can have all kinds of virtual environments, but they are not personalized, they tend to be representable of what Customers have, either by being proactive and knowing what Customers use, or by being reactive and adding in software or configurations that became known trouble spots.  I'll share a specific browser configuration and plug-in configuration with someone, but I'll be damned if I am going to sit there and share someone's Hello Kitty theme with kitty cat icons made up of the heads of someone's little babies.  I'm sure that works fine for some people, including some of my co-workers (not the Hello Kitty theme, but the cat picture backgrounds) but its not something I'd expect to see in my lab nor would I expect it to have issues with commercial software.

Bug Reporting.  As I said, just because Jim from South Carolina found a particular GUI issue with the tool bar, and it was confirmed with Vijay from Pune as well as Klaus from Dresden, doesn't mean that bug is going to be fixed.  I'm all for finding as many bugs as possible before I release, but honestly, do I need 100+ minor defects or 250 Enhancement Requests sitting in my bug queue because the Crowd found them, and the serious issues they had were already reported and entered?  I don't see what this gains me, other than more defects for triage that may or may not ever get fixed, but from a business standpoint I can say that we found them with the Crowd!  Serious defects should hopefully be caught prior to the release to the Crowd, or maybe specific configurations can be found, and I am all for getting as many of those as possible, but when I look at the numbers of Critical and high priority defects being added in, there are very few found later on; while I'd like those fixed is there any guarantee they will be?  I'm still waiting to see the results on that.

Isn't this really Beta?  I can see this point, and I partially agree with it.  If you are working on an open source project you expect updates that may or may not be very well tested, with a Beta you are taking this knowing that the software may crash (and badly) in your environment because that is what it is.  So what is this?  Beta?  Well, I'd say not really since it seems like the releases are still very early in the cycle, unless the company has a long timeframe between a release build and eventually getting it out.  So where does the testing stop?  Does the release mean everything found is put on hold and we now wait for the next release, especially since Customers now have a copy we can be getting reports from them.  I'm waiting to see more on this, but I am trying to keep an open mind.

Talent Pool.  Who are the people who are joining?  Are these people new to the field and they want more experience?  Are they bored?  What is their experience?  As someone who has trained people off and on I can tell you that a bug report from someone who is just learning compared to someone who has been doing this for a long time is very different.  I'm not sure who is signing up here, the money doesn't seem like much, and maybe I am not their talent pool, but when I get out of work the last thing I want to be doing is testing some other kind of software, heck I don't like to practice programming much outside of work.  I'm not the target audience for the community, but I'm curious as to who is.

Unlike Mr. Whittaker I don't know if this is the next logical step, but I like to take a long view and a higher view of testing as a whole, some things get tried, some change and some just end when its seen they don't lead anywhere.  I don't know where uTest and the Crowdsourcing is going to go, but I am curious to watch and see.

2008-Aug-22 - IOStress part 2!

After the Development team came back from the latest Microsoft PlugFest there was a new version of the IOStress tool, which has a better structure than the last one without all the useless command scripts in the root directory.  It seems to run better, and has picked up on configuration I had set for the previous one, as I was able to just put the directory down and run it with fewer errors than I had seen in the last version.  Running with the Verifier and our Driver still causes a crash, but that is expected for the way the Assertions were being used in 2008, supposedly in Windows 2003 this was more forgiving, but 2008 doesn't like it.  I've crashed the 2008 box 3 times since installing the new version, go me!

I need to get a listing of the tests, and what they do, but for now I have a nice basic structure that runs on our driver and doesn't crash the box, always a good thing.  The one thing I don't like is the reboots the IOStress program does when setting up certain tests and when completing them - since I run this in a remote desktop window and do other things - the test goes away when I don't notice that the remote desktop is gone.  Part of the problem with the reboot is that the test results window disappears quickly after the reboot, I may have to look at getting the mail piece working so I can get the results in email, they may be formatted better.

Undocumented and unofficial tools are so much fun to work with....right.

2008-Aug-15 - Setting up IO Server Stress Tests For Windows

In my current position I test a filter driver, basically a driver placed into the stack on multiple platforms that gathers events on the system and passes them up to a Java agent to send to a central server for review.  Some events are filtered out, and some are not, but default and this is done to keep the messaging down to a level where the driver is not stressed out, but sometimes its good to know how the driver operates under stress.  There are a few scripts we use to load the system with events of all kinds, but Windows provides a suite of tests in something it calls IOStress, and is given out at Plugfest every year.  I have spent some time setting it up for testing in my environment and this is what I have had to do so far to get it to work:

• The IO Stress program needs a net share set up for ntiosrv that points to the iostress folder and supporting files
• Copy the iostress folder from \Software\Microsoft\iostress to a local drive (say D:\) and rename the io.stress60 (plug-iosrv) folder to iostress
• Create the share for ntiosrv

net share ntiosrv=d:\iostress /GRANT:Administrator,FULL /REMARK:"IOStress Share"

• This will set up a share to the D:\iostress directory, change to that location and run - iostress.cmd /ignoredebugger
• The I/O Stress program main window may be minimized, select it
• In the Registration and Verifier screen enter the driver to be tested
  • Select Low Resources Simulation
  • Select I/O Verification Level 1
• In the Registration and Verifier screen, if run with No Debug, there will be a note regarding No Debugger Selected, this is due to the Email and Contact information being empty. Enter values, any will do, into the three fields.
• Select the drives needed to be tested, all available drives on the machine will appear in the Run Information tab, on this page as well the Test Time can be entered in the Specify Number of hours to run field.
• The Stress Tests tab allows selection of specific tests, a slight overview of the tests is included with the zip file that contains the tests, it gives some information on the basic tests.
• If the system reboots you must log in to the machine, the IO Stress kit likes the Administrator account to have a blank password, but that is insecure

Still, even with all this I have encountered a few problems when running the tests:

• There is a complaint about a registry value not discovered, I have yet to find documentation on what that value is, but it seems to still run
• Do not run with the Verifier, this seems to cause crashes, possibly because the IOStress program runs with the driver in the Verifier and having them in twice causes conflicts
• When setting up the environment it claims some network drives are not found, I am not sure if its the ntiosrv share or not, I can never seem to find it in the net view list, but without the share being available the entire test suite will not even start.

I don't mind Windows tests, and undocumented Microsoft tools are such a joy, but it does some checking and has allowed me to verify some issues and find new ones especially on Windows 2008 which is a new platform for my group and we don't have a complete set of tools for that platform yet.  If I get these issues resolved, I'll note the fixes.

2008-Aug-5 - Command Windows Where You Want Them

For most Windows platforms I have used a registry hack to add a Command Window option to the right click command window within Windows Explorer  This is on the Microsoft Site, as well as available though the PowerToys add-on.

I just discovered that Windows 2008 has this embedded.  To get a Command Window in any directory right click the folder name while holding down the shift key, this will create an option to Open a Command Window within the directory.  This should work for Vista and Windows 2008, so if you need DOS windows like I do, this is how to do it within W2K8.

2008-Jul-16 - Perl Packagers

A few months ago I started work on a Perl version of a protocol test app, the original one was written in C and while we have the source code my network C was never very good, but I can handle Perl and protocols so I changed it.  Basically there are two scripts, one that sets up a dummyServer (listener) and another that is a dummyClient (sender).  The sender connects to the listener, sends a few lines of data and disconnects.  Short and simple because all we are doing is checking that the connections are detected, and where the connections come from, because that is what the product does.  Now this worked fine for awhile, but now we are detecting connections over IPv6, and that necessitated something else, and since I couldn't work in C (yet) I did this in Perl and it was a good experience.  I got to learn alot more about networks and protocols, and how DNS will work on our lab network, which I had no idea was limited.

Now that the scripts are done I then discover our product likes to ignore a lot of Perl scripts, good for testing but not good in this case, so rather than try to hack the product to make it recognize these scripts I am looking at changing the scipts into executables.  This has been more fun.  Since initially my work is done on Windows, so I can use SlickEdit, I tried buildind the various Perl Package Utilities on my Windows 2003 machine, what I encountered is noted here.

Perl2EXE, while nice, does not give me what I want, and sadly also does not install properly on my machine.  Giving me all kinds of weird Mac::InternetConfiguration errors, that seemingly a lot of people get, but somehow get past by installing older versions of certain Perl libraries or somehow ignoring them.  I can't seem to ignore them, and honestly, if I need to go through these weird kinds of hoops to get this working on a Windows machine, I'd rather do it on Unix.

PAR is a good alternative, but I want these to be executables not PAR files that I need to run individually, so while it was fine many versions ago with the recent split to have PAR and the Perl Packager (pp) in different distributions I have to install it and the PAR:Dist that gets me the Perl Packager I need.  On Windows I get loader errors with the PAR::Dist, which many have gotten but somehow gotten past awhile ago, there are some sites that list instructions on how to install the Packager, but some are woefully out of date or had steps which did not work for me.

pp - the Perl Packager is the one I am in the midst of using, but as I am now in the middle of adding this to one of my Unix machines (I have AIX, HPUS, Linux and Solaris available) once I get it working then I will know how things are going.

Perl is always fun, and I am always getting into something new.  Never a dull moment.

2008-Jun-13 - What do I want to be?

After working in various positions and companies I can pretty much say I have done all the pathways to progress I ever considered at one time or another.  Starting from single person to Manager of QA, building up the groups overtime, or as individual contributer working on setting up processes, expanding existing testing or even automation I have done all the different aspects of what I have encountered.  Not that this is all of them but in many posts through the SQA Forums, most people ask about progressing towards Automation or Management or stay where they are.  Well here is what I have found.

As a Manager you will get away from the day to day testing, unless you are able to finagle some testing work into your schedule, and I always tried to do so for various reasons but its a good thing to do.  Most importantly to keep your skills up, sure management skills are important but within QA its good to also be technically proficient, because you will deal with Developers at times its good to have the technical backing to either keep down a snowjob or to back up what you are saying when you just know something that you are being told will work, just won't.  It's also good professionally, because you may end up leaving your job at some point, and if like me you enjoy the environment of start-ups, you will need to begin again and test so don't let the skills lack!  I know someone who used to be an IT Manager and worked his way up to there, but then spent so much time managing that when he ended up looking for another job he was so far out of the current environment technically he felt he needed to take a lower level job to catch up.  I say avoid that at all costs!

If you want to do Automation, then make sure you enjoy programming, know how to do it and want to make it your life.  When every day is either spent debugging a test program so that it will work correctly, because the interfaces you want to test have changed in subtle ways, and are not well documented, then you have to enjoy that sort of life style.  Knowing how to generate good frameworks that are well documented, easy for others to use and most importantly give clear and concise results is important.  When you are the toolsmith, your goal should not be just to make any tool, but make one that is easy for others to use and implement in various ways, so you can move on to making the next and greatest tool, don't just think you can make something then let it go.  See that its doing what you thought it was.  In the case you leave one day, don't leave a bunch of tools lying around that not many people know how to use, and can't figure out because the code is not commented well because "it was only a test tool".

My preferred area is a nice middle ground, at least for now, I have done managing, and some automation work (makes me realize that I don't really like getting into coding too deep, I'll never be a hacked I know it but I can get by), but what I like is being the person that gets a project and runs with it.  With my skills as a manager I can lead a project from the QA side, knowing how to diplomatically work with people, because I learned lots from some good HR people in the past, and I know enough about programming to be dangerous and take the time to work out the issues I find.  Some developers love that, especially when you are eager to learn, and I find that is the most important thing for me, I like to keep learning and working on new stuff as well as learn more about what I am working on.  There is so much out there, its almost inexhaustable, but its all good.

2008-Jun-6 - Monthly Topics

It's June and it means its another montly topic, and even though the group likes having the QA Talks every month I find it hard to keep people coming up with something they want to talk about, or even do a presentation.  So most months it falls to me to make something, last month I did Regular Expressions since there is more Perl being used, and a few others on the team are learning how to write scripts for the automation framework.  So either I am better than I think, I doubt it, or no one wants to do the work.

I usually need to be inspired to write a topic, its not easy to just up and present for 20 minutes, and I try to maintain interest to keep people's attention so its tough to make good technical topics that are enjoyable.  Not sure yet what this month will be about, but having read some more lectures by Yifa, a Buddhist nun associated with my wife's temple, I may have something on how to break down tests.  Funny how we find inspiration in unusual places.

I'll probably come back and add but topics I have done or have thought about include:

• Test Heuristics
• Agile & Scrum
• Basic Debugging
• Driver Debugging
• What is Quality?
• Regular Expressions
• Find your Buddha (seeking Test Nirvana)
• Test Frameworks
• Database Test Techniques
• Ruby

That's all for now, I may update the list later on.

2008-May-19 - Nightly Builds - Just Good Sense

Occasionally in some discussions on Agile the topic of Nightly Builds comes up, and while I will agree that its something that should be done during a sprint, and while its mentioned in a couple of newer editions on the methodology I just see it as something that needs to be done.  You don't need to be on a Scrum Team or on an Agile Sprint to see the value in a nightly build, if your code is building every night then it means people are doing their work and verifying that things work properly, nothing more special than that.  Methodologies are great to build a framework around, and often give high level management some way to say they are doing something that is hot and on topic, or allow them to utilize yet another buzzword.  Outside of that nightly builds are good for everyone.

The gains are immense, not only knowing that anyday someone can check out the source tree and get the code to build in their environment but once you have taken the step of having a build happen then  you can go onto the next step of automated testing.  Whether its a suite of Unit Tests or an automated Smoke Test, unless code is building you are not taking that next step, adding those steps allows one to have a good deal of confidence against the code checked in.  Having tests run every night allows you to know that your code is good in the source tree not only because it builds but that its passing a set of tests deemed necessary for the code to always pass.  Confidence assured.

So before worrying about whether or not you are following a specific methodology get some basics in place first, Nightly Builds is one of the most important.

2008-May-5 - Software is NOT Manufacturing

I've seen it alot whenever discussions on methodologies come up, especially the ISO and CMMI ones, but it usually comes down to the whole mechanism of why software is different than making widgets.  Thinking about this off and on, I came to the conclusion for myself, that there are some major differences here and it all comes down to the fact that widgets ARE different than software.  Beyond the physical there are some major differences between generating a product that people use, utilize or somehow adapt to their life and software which can also have the same implications is very different.  I came up with what I consider 3 major points, makes it easy because no one really has a top 3 and a half reasons list, in which they are very different and thats design, implementation and release.  Yes, they are similar but that's about where it all ends to me.  So let's take something amorphous called "software" or "the programme" and compare it to "Plastic Shovel Red No.2".

Mind you this is all my opinion, if you feel different go ahead and comment or stop reading now.

Design.  When you create software there are lots of meetings and specifications and documents that tell you how it is going to be and what it is going to be, unless its a personal or open source project (or even Agile!) where this may change.  Input is gathered, perhaps some market research, after all you want to know what it is you are building before you go ahead and do it.  Someone has to know before starting to even think about it that the particular software is going to be useful, or will be, in some niche that will create an entry point for its particular functionality - if its not going to be useful no one will want it or probably not want to work on it either.  So that need is part of the design and helps shape what the software will be, there is an idea and a goal to reach then, so the work on the design and any associated research is all about what will help the software become that need.  Now the Plastic Shovel is fairly simple, it has some uses (more if you consider the imagination of the typical 4 year old) that are known ahead of time and a shape that needs to be made.  It's considered from multiple angles and multiple measurements (how long should it be?  how wide?  how thick?  when we make the handle should it have an O shape at the top or another shape?) all of this is considered at the beginning.  Once run through a couple of times, perhaps a meeting or two just as might be had for the programme, a decision is made that this is the way we want it.  A prototype is made, for either one, and that is then presented and demo'ed to someone or a group and input is solicited.  Now this is the important part.  Once that input is given software can go through more rounds of feedback and use while Plastic Shovel is decided upon and then you know what....decision made and go make it!

This is what I call the Feedback Divergence.  For software you can continually elcit feedback, because its complex and evolves, almost as if its a living thing made of many living things and what you end up with is often not what you started with.  A plastic shovel, on the other hand, once decided upon has its specs sent to the factory floor with orders to make X numbers of them, software basically has one thing made - the source code.  Sure someone can come back and say that Plastic Shovel was horrible and broke when too much sand was put on it, but that just means we start the process again with Plastic Shovel and make a completely different version of it for the factory to press out later.  That feedback does not go back into the design and create a different one, because its already been made!  Software, because it can be built at any point of the process can elicit input at any time and be made different at that time.

Implementation.  Part of putting the design down on the factory floor for the Plastic Shovel is that a mold is made, and machines are set for creating thousands of cheap Plastic Shovel Red No.2.   Once that is done the Shovel is committed (now I know that's an Agile term but its also true) the Shovel will now be made by the thousands.  It's now being implemented by the workers on the floor whose only job at this point is to make thousands of nice, bright plastic shovels in Red No.2.  Not Red No.3 because little Suzie wants that color to go with her polka dotted suit, or Green No.5 because Jimmy thinks its cool, its all Red No.2.  If you want another color you are buying another product because once that shovel is stamped out on the factory floor its done, been implemented as designed and is not going to be personalized at the factory at all.  Not in the plan.  Software can do that, because its able to be adjusted because its not a physical object.

This is what I call the Hand Divergence.  If I can hold it in my hand there are a limited number of mechanisms by which this object can be adjusted, personalized and or changed.  With a significantly smaller number of those changes actually provided by the manufacturer.  Once its done its done.  Software in some essences is never done, because you are making a new version off an old set of source and code which together comprises the whole, and because its malleable you can infinitely change it.  You can't do that with a plastic shovel, at some point you will come to an end of possibilities to change it, software does not work that way.

Release.  When software is "done" by a project sense its given to the Customer, perhaps on a CD or a download, and there is probably a release party somewhere with beer, food and hula girls.  Well not everyone gets the hula girls, heck some people don't even get the beer.  The point is, you can take your checklist and look at it versus your software and say you have X% functionality done, which will be acceptable to the Customer at this time, with other improvements in queue, or soon to be in queue once they install the programme and use it.  But is it really done?  Not at all.  You've hit a milestone where you can say that the functionality that was originally designed for was satisfied to a degree, the closer to 100% you get the better off your project was managed.  Now that plastic shovel, once its off the factory floor its either on its way to some discount retailer near you to get a price tag, or its going to be bundled with the "Super Special Summer Sand Spectacular" where it takes its place in a bucket with a bunch of other cheaply made plastic items that are all COMPLETE!

I have no divergence with this one, unless you want to call it the Profit Divergence.  Part of what makes the Plastic Shovel Red No.2 special is that once it was sent to the factory floor we knew exactly how many needed to be made and sold to generate a profit for the company.  For software we are either paid for the work up front, where we probably went over budget because of circumstances beyond our control, or it will be sold and people will go out and driver desire for the programme so we can continually make money off it and get people in line for the improvements that come later.  At this point the story for Plastic Shovel Red No.2 is done, it may go back for a redesign but maybe not, software is just beginning and still has a long road ahead.

So there you go, my rationale as to why software is not manufacturing.

2008-Apr-23 - A Developer Went Ka-Choo

(My apologies to Dr. Seuss)

You may not believe it, but here's how it happened.

One fine spring day....a Developer sneezed and went KA-CHOO!!

Because of that sneeze, a task dropped.

Because that task dropped, a Manager got assigned.

Because that Manager got assigned, he had to delegate.

Because he needed to delegate, and Architect got work.

Because that Achitect got work, some brainstorming happened.

Because of the brainstorming, some fingers got pointed.

Because of the fingerpointing, around that table

     Someone got more work than they were able.

Because of that work more talking was done.

Because there was talking, a document got made.

Because a document got made people had to read it.

Because of that reading comments were made.

Because comments were made, they were sent around.

Because they went around people got to read them,

     And when they read them more comments were made.

Because of the comments more people got involved.

Because of more people more comments were made.

     And so comments were sent around one more time.

Because there were opinions no review was made.

Because there was no review more comments got added.

Because of more comments and no review the document got stale.

Because it got stale it was ignored by QA

Because it was ignored more tempers were flared

Because tempers got flared more comments were made

Because more comments were made someone asked why

Because someone asked why they looked at the task

And because they looked at the task, it’s true I’m afraid – they

     Ran into a spectacle

And that started something they’ll never forget, and far as I know its going on yet.

2008-Mar-12 - Who is testing the tests?

When Developer's write code who looks at it?

Usually another Developer, at least for a review, and more importantly someone who is testing it.

When Tester's write a test who looks at it?

Uhm....I sometimes find that hard to answer.

Sure, we occasionally send out test plans and cases for review, but does it happen all the time?  No.  Are you sure everyone looks at it?  No.  Will comments come back?  Not always.  I do try to look at test plans that my group generates, I review it even if I think I will have no comments, though I can at least spot a grammar error or two if I look hard.  But that's not really a technical review is it?  That's kind of the point, where is the technical review?

I've seen this come up on occasion in blogs or posts, that someone writes a test tool and uses it, fixes it and makes sure that it does what its supposed to do then what?  Maybe suggest to others that they use it when checking certain code or functionality, but has it been as strenuously reviewed as code for Production?  I can honestly say for myself, that has not always been the case.  But in a way to catch up and practice what I preach, I am spending more time getting reviews and following up with people, having reviews of tools we use in house done by other people, including Developers.  Not only to make sure that things work right, and I can explain what is going on, but I can maybe find a better method of doing what I want by someone who spends more time coding than I do.

When was the last time you wrote out a spec for a tool you wrote?  If it was complex enough, that is...I can honestly say 1 for all the tools I have made.

Let's face it, we sometimes take shortcuts as in the test environment its not perceived to be as necessary to have strenuous reviews of what we use because what we are using is not going into Production.  What we are testing is!  So why not have the same standards for the tools we use and write?  Be just as critical as if you were going to buy such a tool from someone, and make sure you know what its doing and how.  The more you understand and can talk about the tool you are using, the more secure you can be in that the tool is doing what its doing.

And yes, I say reviews are for QA as well as Development, because in the end, its all code.

2008-Feb-13 - Clearing The Junk

I recently got a book from my sister-in-law, my wife's family comes from Taiwan and they are Buddhist so I occasionally end up going with them to the Buddhist temple and meet some of the monks there.  One I have had the pleasure or meeting, and discussing things with, is the Venerable Yifa, a very smart, funny person and prolific writer.  More can be found out about her here Yifa, I'm also pretty happy to know someone with their own wikipedia entry, pretty cool.

In reading Authenticity: Clearing the Junk, which focuses more on the junk cluttering our lives, I inevitably had to start working on something for test plans and started thinking - do I have junk here?  Is there stuff that is cluttering up my testing and making my work harder and more difficult than it needs to be?  While I can be prolific when I want to be, I can also feel that I don't want to waste time on something because its a point or discussion I feel can be done in a relatively short amount of time, my test plans are not padded in any way.  I don't write more than I have to, probably from advanced laziness where I don't want to have to update any more than I have to in the future if things change, but also because I know things may change in the future so why get set into something that may not be the same in 6 months?

So if my test plans don't have much junk in them what about those test cases?  I took a look and one of the benefits of using Excel is that you only have a limited amount of space to write in, so I am actually boxed in to be junk free there.  Not a bad way to leave myself.  So I decided to look at the Test Harness we use, and yes, lots of junk there.  Not all mine, but I certainly contributed at some point.  So I need to clean that up.

But why stop there?  If I look at my calendar I have a lot of meetings, some of them go far longer than they should, do I contribute junk there?  Probably.  I better start cleaning that up.  Of course sooner or later I am going to get back to looking at my life and seeing how much junk I have there as well, there have been cuts in many things over the past year just to provide a healthy home for my son.  No soda in the house anymore, I do miss that, not as many cookies and snacks, I eat way more fruit than I used to, and baking our own bread has been nice and gives the place a rustic, homey feel.

I guess I've been keeping an eye on this stuff for awhile now, so I guess I am going to a good place in life, and work.

Those Buddhists, pretty smart.

2008-Jan-31 - Do you gather?

There is that old saying "a rolling stone gathers no moss", well yeah, because its like...uhm...moving and stuff.  If you sat around and let your bones and skills ossify you might find there is a bit of moss somewhere, or something else, of course it all depends on your location.  Because its nice to get people together and discuss something about your work, I started a monthly talk here about 9 months ago, and its worked out pretty well.

Why I did it was to give a place where everyone who was in QA, and in Development, could get together and hear a short presentation on a topic then have a roundtable to see if there were ideas of ways to use this in our own environment.  Development used to have a bi-weekly meeting to go over some new programming concept, cool widget or what have you, and while they were great to get in on they did not have much bearing on my day to day work.  Now I can actually come to a subject that is revolving around some facet of testing I do, make a presentation about it, instruct others who might derive some benefit and maybe talk about ways to improve it.  This has been nice, and we've had some good discussions and while I like to keep the presentations down to 20 minutes max, with the rest of the time for discussion, when the VP of our division came to the one on Agile and Scrum that lasted about an hour and a half as questions continually came up.  That one was great because it allowed us to look at ways to improve the process we had on a recently started major project, I am glad to say that we also adopted the information radiator concept and each of the three major sites have a weekly radiation dose of information on the state of the project.

Topics are open to all, and we've had some presentations from Documentation, on test tools like QuickTest, Virutalization on Linux, Debugging methods on our products and we have more planned for this year on Quality and API Testing.  It's great, allows people to brush up on presentation skills on a friendly audience, and gives some great opportunites for information dissemination.

Of course if you are ambitious and can do them offsite at your favorite restaurant or brewpub, more power to you!

2007-May-20 - Testing With The Man - Chapter 7

Section 7 is all about Maneuvering.  Well, not completely about moving it is also about gathering disparate elements and blending them together into one, much like testing where you need to pull together different techniques and skill sets.  This is the focus I am going to take, blending, much like coffee can be blended to improve the flavor (if you like coffee that is) or it can become bitter or overpowering if you have too much of one thing.  Tea can be the same way, blend your tea one way and you have a nice calming beverage, another way and you may as well be drinking roots boiled in water.

Sun Tzu said: in war, the general receives his commands from the sovereign.  Having collected an army and concentrated his forces, he must blend and harmonize the different elements thereof before pitching his camp.

Much like in a project, the Program Manager has sent commands to the Test Lead who now collects his team and concentrates them which requires blending and integrating them together into a cohesive unit.  In a test team everyone has a different skill set, way of testing or may be a subject matter expert of different components or a product, blending the team together to get the best benefit requires skills that take time to learn in order to become proficient.  Having a team that knows multiple test techniques, with some better in one type than another, gives a good balance but how do you make use of them to get the project done properly?  That takes planning, if you have people who are strong manual and exploratory testers letting them use an early release can help find some of the test cases that may not have been thought of or bugs due to design flaws.  Maybe there are automation folks on the team, having them sit with the Developers to help write, and perhaps review, test cases will give them an idea of what is coming and allow them to prepare a test framework for the test cycle.  Perhaps there are some good analysts in the group, let them into the early design and spec review meetings, have them go over every spec and document carefully and get the design and functional issues out of the way.  The best thing to be doing once you get your team is to know what their strengths are, plan to use them according to their strengths, putting the analyst who doesn’t know much Perl to work on the test framework written in Perl is a good way to make sure that you don’t meet a target.

Maneuvering with an army is advantageous; with an undisciplined multitude, most dangerous.

To meet your targets, and to be able to finish a project with a team, works when everyone is focused on the same goal.  An army is a force that has a focus, a plan and an enemy that they are to face.  A multitude is a group of people together for a cause, or even as a response to something, but they are not going to be focused.  Always try to have an army under you, a group that will listen to your commands and be focused on the goals that the project requires, when the people under you are listening and following you can do plenty.  The whole is more powerful than just the sum of its parts, and a team that is focused can do amazing things and meet deadlines that to others may seem impossible.  Do not be at the head of a multitude, a group that is there just for the sake of earning a paycheck is not going to help you get your project accomplished, nor are they going to assist each other.  Make sure that your team is working together, take them to lunch once or twice and talk to them outside of work, you can often tell outside of the company if there might be friction.  If you see it, then take care of it as soon as possible, letting that sort of thing fester will cause trouble and eventually will come back and bite you.

We may take it then that an army without its baggage-train is lost; without provisions it is lost; without bases of supply it is lost.

Now that you have a group of people together to get work done, you need a place to get that work done, tools and equipment, maybe even someone needs that ergonomic chair to be able to sit at the desk to get work done.  Find out what is needed and make sure its there, I have said it before, when you do not have an adequate environment to work within, then you only end up hurting yourself.  Having a group of people who are trained and ready to go, working on one or two machines is not going to get your project done, make sure they all have the equipment they need, if you can’t get it then you need to be creative.  If a machine needs to be signed out for each test, then make sure people know this and set up a common area for people to know and look to see if the Database Server is being used by the Functional Test group Tuesday; stepping on each others toes can not only invalidate tests but can cause bad blood between a team.  While a well organized team will communicate constantly, it’s a good idea to have back up methods of communication, not only for people to make sure things are working properly but if there was an issue traced back to a previous test you know what that test was and can go through reproducing the issue.  Also be sure you can get things repaired in a short amount of time, get images made of your lab machines when they are up and ready, that way if you encounter too many issues the machine can be re-imaged to a baseline and you can move forward and attempt to reproduce the issue.  Taking time to rebuild test machines during a project is something that cannot always be afforded.

We cannot enter into alliances until we are acquainted with the designs of our neighbors.

If during a project there is a need in another group that only you can fulfill, perhaps only the test lab has a specific configuration or a certain server, and you need to help out make sure both sides understand the needs and what the “lease” will be.  If a group wants to work with yours be sure you understand their needs, while I will never advocate being “political” in a company it’s good to not get into situations where you are part of someone being political.  While I am not saying you need to distrust peoples motives, being sure what is happening is good for you and your team, if a group comes into help you or thinks they are, but are only impeding the project, then you need to stand firm and clear up the situation.  Priorities will change in a company, even during “critical” projects, but make sure you understand what the priority change is, why it is happening and how you can get back to your project once it’s resolved as you now need to be creative for lost resources.  Again, my view in working in some places is to always be sure you are covering your teams ass, document everything, be transparent and be sure that the Program Manager understands what is happening with your team; that way there can be no discussion or finger pointing later on as long as you have documented and shown the work you have done.

Whether to concentrate or to divide your troops, must be decided by circumstances.  Let your rapidity be that of the wind, your compactness that of the forest.

Creativity with resources is another learned skill, at least to me; I have never really seen this as something that can be taught.  You have to learn to anticipate changes in your projects, know what resources you can move around to resolve the issues, without losing too much against deadlines and milestones.  This is usually something you can learn working for a manager who is really good at it, see how they do things, or if you work with someone who has encountered multiple difficult projects, talk it over with them.  It might even be worthwhile to post it on an online forum, just make sure you describe the situation adequately and what you think might be a good solution, it eliminates unnecessary questions.  There may come a point during a project where part of the team needs to work on a customer issue instead of on the upcoming milestone, don’t hesitate if meeting the customers need is vitally important to the company (and it usually will be) but let the Program Manager know this is happening and why.  Again it’s a learned skill, make sure you are prepared to make a few wrong decisions, own up and if you get called on it you could try to explain your reasoning and try to move on.  This is something that gets easier as time goes on and you encounter it more.

He will conquer who has learnt the artifice of deviation.

Anticipating, that is a very important ability to have.  Once you learn that a project is spelling trouble, that something will not be there when it is needed or that something is going on with the team then you will be well on your way to many successful projects.  Much of this does come from working on many and multiple projects so don’t be afraid if you get some things wrong, learning to see the mistakes we make so they are not repeated is one way of improving yourself and getting better.  Failure teaches more than success at times.

7 down, more than halfway there!  Thanks for reading.

2007-Apr-30 - Testing With The Man - Chapter 6

Section 6 is all about Weak and Strong Points.  This is about finding the places where you can attack an enemy, or to see where the mistakes are going to be, in project parlance this would be equivalent to checking for failures.  A good thing for a person in QA, or QC, for that matter.  It also ties in to the other sections which focused on offense and defensive postures (Section 4), Energy (or direct/indirect measures) (Section 5) and now when there is an understanding of how to position yourself and probe for weakness – you need to strike.  At least in a military sense.

Sun Tzu said:  Whoever is first in the field and awaits the coming of the enemy, will be fresh for the fight; whoever is second in the field and has to hasten to battle will arrive

Exhausted.

A pretty easy one for any person coming onto a project, if you come in from the beginning then you are able to handle the work and can do it at your own pace, come in with 4 weeks remaining to do a whole project-ful of work and you will rush and exhaust yourself.  In one respect this is a good way to think of when QA should be brought into a project, come in at the beginning and you have a grasp of the overview and the work necessary to be completed.  Documentation, or any necessary items that must be done, can be handled with enough time to meet deadlines, or aid in the planning of the project.  Come in late in the project and its all catch-up.  QA available and involved at the beginning can generate a Test Strategy, Plans, Cases, Scripts or whatever is needed; come in just before a release build is made and it’s a rush to get things in place.  Make sure you have the time and resources available and be able to learn about the project, getting thrown in will assure that you will be exhausted at the end, and regardless of all the Herculean tasks accomplished by others, and you may find that it was not your best work or at all something you want to be associated with after.

Therefore the clever combatant imposes his will on the enemy, but does not allow the enemy's will to be imposed on him.

Another reason why QA should be involved at the beginning, or in some cases the way to be sure that you have input into the project and not let the project have input into you.  Impress QA on the project; do not have the project impressed on QA, which will always be a losing battle.  Just as the armies of centuries ago positioned themselves to the place of greatest advantage, putting QA in a position where it is overwhelmed only leads to failure.  As has been said, be aware of the layout of the field, the position of your opponent and see the strong and weak points in order to predict the work that will come.

Appear at points which the enemy must hasten to defend; march swiftly to places where you are not expected.

Think ahead.  Not much more to add to that.  Might be a good thing to keep on my white board, but never let the current situation focus you only on the moment, be aware of what is around you and coming.  That is what separates the men from the boys, so to speak.  Its also what makes a great manager as opposed to a bad one, knowing what is coming into the department after the current project and keeping your team apprised of it while making sure they can do the current work makes their lives easier.  When your team is happy you are happy, and when you are happy then work is a much nicer place to be.

 Hence that general is skillful in attack whose opponent does not know what to defend; and he is skillful in defense whose opponent does not know what to attack.

Basically the Art of War in one sentence.  It’s also a way to think of planning, as I have mentioned previously be aware of all the needs, this reinforces the way to anticipate changes in the project.  Just as there is the need to “think out of the box” for determining a bug, so planning requires similar thinking.  I am a big fan of planning, but its also the process in which the planning is done, I document a lot of the process but this is one area I am always bad in documenting, because I know how I would plan a task, but its not always the same as someone else so why document my personal method?  Because someone may actually be able to improve their own by knowing what I do, but by the same token I am now skillful in planning when the project I am on doesn’t know the plan; or in some cases if the PM is lacking I look better in comparison.

Knowing the place and the time of the coming battle, we may concentrate from the greatest distances in order to fight.

Knowing when the final test effort will be, no matter how long, gives the time to plan.  Prepare ahead of time, everything you need, and keep an eye on those resources.  Plenty of times I have had to pitch in on an emergency situation because the main project is running and something came up but I cannot take a resource away; I’ve also tested multiple builds with different bugs to verify at the same time but I don’t suggest that.  Knowing when you are due for a release and how much work is left, keep milestones along the way; its a good way to have an idea of what is going on, will help in being able to say at any time – we can ship on that date.  I always try and keep a pulse on where things are and how they are going so if I am asked I can give a status, or at least know who needs to give me an update.  Knowledge is a powerful thing.

All men can see the tactics whereby I conquer, but what none can see is the strategy out of which victory is evolved.

At the end of a project everyone can see the result, but not the path you walked on to get there.  That path may be a tough thing, and while its great historical data, knowing how many bugs were closed when does not always give you a good idea of resource allocation in testing and its results; I distrust pure numbers in a field where Risk Mitigation often involves some sort of feel for where the project is.

Do not repeat the tactics which have gained you one victory,  but let your methods be regulated by the infinite variety of circumstances.

…also…

He who can modify his tactics in relation to his opponent and thereby succeed in winning, may be called a heaven- born captain.

The five elements (water, fire, wood, metal, earth) are not always equally predominant;

Be flexible, the project plan that worked for the last release may not work for the next, be aware you will need to change strategy at some point.  Just as I said history will not always give you good numbers, so you should be able to look back and still be able to see what worked and what did not.  Did doing piece A before piece B give the result we wanted?  Was there a need that came up for automation that took up time?  These sorts of things should come out in a retrospective or a post mortem, in order to improve the process, but more often than not most companies just jump into the next release.  Being a believer in the power of history its worth taking some time to discuss the project, not with heated emotions but a little dispassionately, to get what worked and what did not out in the open.  Advocate for what is needed after the discussion, but let everyone have a say as to what they thought went well or not, you might be surprised at what some people come up with.

Oh yeah..almost halfway done!  6 down.

Thanks for reading!

2007-Apr-11 - Testing With The Man - Chapter 5

Chapter 5 is about Energy.  Following on from the last blog’s commentary let me pick up a theme I sort of touched on, which was brought out in the comments, testing in a passive or active manner.  Think of that as energy.  You have potential (stored) and kinetic (moving) energy; you also have inactive (observation) and active (participatory) testing.  When performing test-y duties they can be done in a manner where you are actively using code or by having it run and checking results, or the environment, without any real interaction with the test scripts.  I am not advocating one over the other, at times I have run scenarios in either way depending on what I was testing.  While in this chapter there is a lot of discussion about men and moving them into battle, I am going for the spirit of the idea and while I may refer to men in the quotes, I will not talk about them as it won’t really relate; unless I wanted to talk all about managing.  Which I could, but maybe another time, I’m going to be jumping around in regards to expenditures of energy but relate it as I can to the quotes.

Sun Tzu said:  The control of a large force is the same principle as the control of a few men:  it is merely a question of dividing up their numbers.

I guess a more current version of this might be not biting off more than you can chew, take small bites, or even manage a large group as many small groups.  On a personal level, if I have a lot of work to do, then doing it in small pieces is much easier to control than trying to complete all of it at once.  When getting a project of work, doing 100% of it, and spreading your time across all of the components that make up that project doesn’t get it done any faster than focusing on one piece at a time.  Baby steps, like we used to say when I was a kid, don’t push yourself until you are ready, just take it slow and focus on what is in front of you.  Same way with a large project or task, take what is in front or needs to be done first and as its done move on, be like water moving around a bunch of stones, the water doesn’t crash into the rocks, well it might if it’s a deluge, but a stream finds a path through them of least resistance and deals with the path that requires the least expenditure of energy.

In some ways a project has an energy balance, or in game terms you have 100 Project Points, pushing yourself across the project you lose points as you move through it.  If you rush then you lose points pretty quickly, and unless you can get time to get those points back you will run to 0 and then it’s Game Over.  If you take time, find the right path, look for the Restore Points, allow yourself time to regenerate then you will get further than just running along the path and trying to reach the end.  Sort of like the tortoise and the hare, the hare may be quicker but may need to rest more, or become too confident at getting there first that he makes a diversion, while the tortoise moves along at a good pace and gets to the end.

The discussion on energy from Sun Tzu uses two terms Cheng and Ch’I, which don’t easily translate well to English, many Chinese terms have a lot of significance behind them which is not really a part of the definition but a part of the ideogram and what lies behind it.  A simple definition for Cheng is facing the enemy, where you march straight to the enemy and keep your face towards them; Ch’I on the other hand is movement, making diversions or changing direction in some form.  Cheng becomes passive as its just movement, a steady flow towards a point, Ch’I is activity where motion becomes part of the movement and the point can then be reached by many pathways.  Pretty much all I am going to do for this one, since it could go on and on, maybe a follow up later on.

In all fighting, the direct method may be used for joining battle, but indirect methods will be needed in order to secure victory.

To get work done in the project requires a path, that was determined within the project plan, to achieve those results occasionally means moving around the point or task and determine if maybe the direction to complete it has changed.  Do you use energy actively or let it collect until the end?  In other words, expend and recharge or use it in a burst?  In some respects using it all at once can be useful, if your culture moves you in a direction where all testing happens at once, or if testing is done over time then alternately saving and expending energy will be the wise choice.  How you move and use your energy in a project is up to you, and will change according to conditions of the project and the company culture.  The important thing is to avoid burnout.

Indirect tactics, efficiently applied, are inexhaustible as Heaven and Earth, unending as the flow of rivers and streams; like the sun and moon, they end but to begin anew; like the four seasons, they pass away to return once more.

There are multiple approaches to the problem, think back to a solution made a year ago, would you make the same results now?  Has experience taught you that maybe a better way now presents itself?  Just as lessons learned are gone in the past, they will come again unless the culture that generated them has changed, just as the problems that arise can be infinite so can the solutions to resolve them.  But just as there is more than one way to plan a project, so there is more than one way to test a project, one may be better than another in a given situation.  A project that has limited resources would be tested much different than one where you have twice the resources, including automation capability, that in itself changes how the project would be handled once it undergoes test.  Still the point I made earlier, there is a certain amount of energy in a project, what changes over time is how that energy can be used and channeled.  A team of testers who are able to do Functionality, Exploratory and Unit Testing are going to be handled and worked differently than a group of Functional, GUI and Automation testers.  Understanding what kind of energy you have, and how it can be used to be most effective will help not only in smoothing a project, but even on a personal level allows better choices in choosing what to do and handle within the time allotted.  Know how much energy you have for a project, then plan it out to get to the end, but remember roadblocks may happen, when they do, be like the water and flow around them eventually they will grind down.

The direct and the indirect lead on to each other in turn.  It is like moving in a circle - you never come to an end.  Who can exhaust the possibilities of their combination?

It’s circular.  Like a point in a circle there are many ways to look at it in two dimensions, 360 degrees of looking actually, but if that point is now in a sphere you have a larger number of ways of seeing that point, use it to your advantage and consider how many ways to test then choose the right one for you.

This one was tough…but its 5 down!

2007-Mar-28 - Testing With The Man - Chapter 4

Chapter 4, I decided upon Chapter rather than sections, is on Tactical Dispositions.  Since this basically means determining where the other army is, and their condition, there is no straightforward way to match this, so I am going to stretch this into my general project planning scheme.

A bit of history.  Before the advent of rapid-repeating firearms, and the British line (the most effective 18^th and 19^th Century force of firepower IMO), armies would march around each other to determine the most advantageous ground.  This did not often end up in actual battles, at times after a day of marching the armies might go home, once gunpowder became part of the war machine this changed.  In the American Revolutionary War when the colonial army fought the British, who would form line, the colonists lost because they could not match the firepower, and shooting speed, of the British army.  The colonists took to hit and run tactics as a way of harassing the British and use their strengths, while the British formed line the colonists would back off and lay ambush somewhere else.  No matter how strong you seem, someone can always find a weakness.

Sun Tzu said:  The good fighters of old first put themselves beyond the possibility of defeat, and then waited for an opportunity of defeating the enemy.

If I am on a project and want to make sure it’s a success, a lot of the failures lie within me (or at least within my own part) if I have an equipment need or a schedule dependency that is critical but I do not raise it as an issue, then its my fault.  My job is to make sure EVERYTHING I need is in place; that my part of the project is on track, following up with other people, making sure they know my schedule and needs helps me get my work done.  In this case the enemy is Project Failure, not something to have on the permanent record.  If I do not want to see my part lose I need to make sure that I have everything I need to succeed, and win, as this says “old fighters would be sure to have their weapons and armies in place then look upon the enemy to see where the weakness may lay”.

I have had situations in the past where I got so bogged down in my parts of the project I did not talk to people and find out where they were, getting blindsided by someone’s late work did not make me meet my deadline.  I was late, and I also had to cut corners, as well as work some long hours, to make up the time, but in the end the blame came down to me.  Learning to watch how other people are coming along, listening in the status meetings for the tell-tale signs someone is not on time, checking in personally and seeing how things are progressing, all things I have learned on the front lines and in my defeats.

Security against defeat implies defensive tactics; ability to defeat the enemy means taking the offensive.

In order to maintain progress you need to know where your supporting documentation is, as well as any dependencies, programmatic or otherwise.  There is the “bunker mentality” of sitting down in a cube and doing work, occasionally popping up “prairie dog”-like in order to check on free food in the kitchen or see who is around.  See my previous example of why this does not work.  As a Manager on a project, the expectation is that you know where everyone is, or at least how to find out quickly what the status is; which also implies keeping people alert and doing their part of the project.  As a person assigned to the project, the better ones will know where other people are, sometimes even those who are not directly impacting the current work.  One of the better people I had working for me in that past knew not only where the people directly affecting her were, but the people THEY depended on, that way she could tell ahead of time what the impact of delays would be.

(Names are changed to protect the innocent….and not so innocent)

Since everything was connected it became easier to know whether Tim would have things done on time if Bob was a little late, by knowing where Bob was Sue could ask Tim where he would be and what Bob’s late work meant to him, Sue could tell where her deadlines would fall.

Sometimes taking the offensive means not just looking at the people and work ahead of you, but the ones behind them, seeing further ahead means being ready for surprises.  Think of how many war movies would be shorter if the man in charge had looked beyond the mass of men before them, or even to the sides, to see the sneaky group coming up that would be the surprise in the battle.

To lift an autumn hair is no sign of great strength; to see the sun and moon is no sign of sharp sight; to hear the noise of thunder is no sign of a quick ear.

Ignoring the part about hair, this fits in with being able to see ahead of you.  Just because you see the sun, doesn’t mean you should ignore the comet hurtling towards you from the other side.  Just because you can hear the thunderclap does not mean that you will hear the people whispering in the next cube.  I just wanted to add this one for effect because I like the wording.  But its also a good metaphor for being able to see the hidden things, and to not just use a talent directly, but indirectly.  Perhaps an early way of saying “think out of the box”?

In respect of military method,  we have,  firstly, Measurement;   secondly,   Estimation   of   quantity;   thirdly, Calculation; fourthly, Balancing of chances; fifthly, Victory.

Measurement owes its existence to Earth; Estimation of quantity to Measurement; Calculation to Estimation of quantity; Balancing of chances to Calculation; and Victory to Balancing of chances.

Who knew you could use math to win wars?  Especially at a time when really it was about numbers of swords and arrows, and I do mean more than just < or > in determining who outnumbered whom; my Algebra teacher would be proud.  Now these five items are interesting, and not just because of the use of them, but by using these five items in the right way there are some good steps of project planning.

• Measurement, originally this seems to be surveying and measurement of the ground, in this case the terrain of the project.  What is the environment?  What will I need to be successful in this environment?  Thinking this through is not only a good mental exercise, but helps in planning the project to its completion
• Estimation of Quantity, Measurement helps very much in estimation. which enables us to form an estimate of the enemy's strength.  In other words by taking measurements of what the project entails, we can then determine what it is we need, how much, when we need it and where we can use it.  In this case Quantity is both physical and it’s the amount of time needed to complete tasks.
• Calculation, and to make calculations based on the data thus obtained.  From knowing what it is that is needed and what time it takes to complete tasks, these can be used together to determine, do we have enough stuff and time to complete by the date?  If not, recalculate, and if so then we can go on.  Sort of a stop in the decision tree; or it may be circular depending upon how you look at it.
• Balancing of Chances, we are thus led to a general weighing-up, or comparison of the enemy's chances with our own.  Now that all the data is calculated, does it look right?  Can this be done, is there a nagging doubt that says step X will probably not work right, no matter how much we want it to?  This is where luck comes in, and its also the point where Guesstimation is either going to succeed or not.
• Victory, if the latter turn the scale, then victory ensues.  So if we guess right, then we are successful, especially if we make the right choices on the way to the end of the project.  Or as I like to think of it, if we get lucky we win the lottery.

Just remember when you live in a deadline driven environment the following is what it looks like when you are behind and the end of the quarter is coming.

The onrush of a conquering force is like the bursting of pent-up waters into a chasm a thousand fathoms deep.

4 down only 9 to go!  If you are enjoying this, or not, let me know.

2007-Feb-26 - Testing With The Man - Chapter 3

We come to Chapter 3 Attack By Stratagem, I like that translation.  Maybe one day my Chinese will get good enough so I can read the original, but that’s a long time coming, I have enough trouble reading printed characters never mind calligraphy.  I may wander my thoughts a bit here, my apologies, maybe I’ll clean it up later.  While QA doesn’t really need to attack, its usually good diplomacy to not go on the offensive in a project, remember War is the last resort to Diplomacy so always try to work things out before going on the offensive.

First in this chapter comes some very good advice “the best thing of all is to take the enemy's country whole and intact; to shatter and destroy it is not so good.”  So much for a scorched earth policy.  But the important thing is to take from the enemy what was theirs and subsist on it.  Not that Development should ever be considered the enemy, but going to Dev-Land and seeing their resources, and what could be useful in testing, is a good strategy.  In some respects consider the area of a bug that has been fixed has been left alone, we are reclaiming or recycling what the Developer left and used to do his work.  Reusing tools is not only good policy but is good for the environment too!  Just as “it is better to recapture an army entire than to destroy it” you do not want to waste resources of any kind, including intellectual resources, make the Developers you work with your friends, get them on your side and in your army and they will help you against the Bug Enemy who comes to frustrate your efforts to complete your task.

I would like to point out, in this section is one of the lines Bruce Lee utters in Return of the Dragon, “my style is fighting without fighting”, originally written here as “Hence to fight and conquer in all your battles is not supreme excellence; supreme excellence consists in breaking the enemy's resistance without fighting.”  Cool stuff.

Part of the thinking here is to deal with the enemy without engaging forces, in planning “the highest form of generalship is to balk the enemy's plans”.  This can be done early by marshalling your forces and preparing for as many contingencies as possible in the project.  In the footnote to this quote Ho Shih puts this very clearly:  "When the enemy has made a plan of attack against us, we must anticipate him by delivering our own attack first."  Don’t wait for the bugs to appear, anticipate them, prepare for them, plan for them and defeat them first.  Know which areas of code have been rewritten, touched, added, deleted and or modified and focus on them; when I have worked on code common to multiple platforms I focus on that area first, before the individual platforms, because I know the changes affect more areas.  If the code does not work properly on both, that is going to be evident very quickly, if there is a need to delve deeper that can be done individually as well but it will be a judgment call on what is more important, something that comes easier with experience.

To recap some in the original “Therefore the skillful leader subdues the enemy's troops without any fighting; he captures their cities without laying siege to them; he overthrows their kingdom without lengthy operations in the field.”  Do not directly engage the faults of a project, anticipate them, and do so in a way that is economical to you; do not have QA resources testing some area that will not work if it’s known, just to see how much is unaffected.  Do not “siege” bugs by using multiple people to try and approach the bug to see where it might be, and if there is a way to get more data on it (there may be cases you need to do it, but this should be a rarity IMO).  Try and find the issues quickly, simply and once documented move on to the next areas under test, using resources in one area and one area only causes delays the project may not have.

Now this gets me to something new, in laying siege to “faults” there are resources, and these resources are finite (if not, I’d like to work for your company) so just as there is a strategy to use your own personal resources on a bug so to are there resources that need to be effective on a project.  Part of being a Manager, Lead or whatever titles the person in charge has means knowing how to allocate resources, hopefully effectively.  The numbers are Sun Tzu’s not mine, so take them with a grain of salt.

• “It is the rule in war, if our forces are ten to the enemy's one, to surround him”.  If your project is small and you have the resources then do everything you can, attack the project from multiple points at once and test everything by overwhelming it; though I admit this is probably a rarity.
• “..if five to one, to attack him…”  This is getting closer in a ratio of resources to project points, not as many people to work with, but still very manageable.
• “…if twice as numerous, to divide our army into two…”  When dividing resources make sure to do it properly, in this respect one part of an army can meet the enemy head on, the other can be used for special tactics, so many focusing one group on Functional or Regression Testing and the other for specialized testing like Unit Tests, Automation, Stress or Load or whatever else needs to be done.
• “…If equally matched, we can offer battle…”  This is probably the reality for many, where there is enough to get the job done, so plan it out and get it done, you won’t have much to gain by creative allocation of resources, stick with the plan you get from your team and what they can do, reprioritize only when necessary and make those judgment calls.
• “…if quite unequal in every way, we can flee from him…”  Well you can’t really flee, unless you want to quit, but you may want to raise those red flags now on how late things will probably be by the time you are done.

Now there are also some ways to make sure your project can fail, I will only adjust these when necessary and most are pretty understandable as they are.

• “By commanding the army to advance or to retreat, being ignorant of the fact that it cannot obey.  This is called hobbling the army.”  Don’t keep moving resources around to meet day to day issues unless you are set up for it, if you are then you have the planning necessary and you are not reacting to situations, you are dealing with them before they arrive.
• “By attempting to govern an army in the same way as he administers a kingdom, being ignorant of the conditions which obtain in an army.  This causes restlessness in the soldier's minds.”  Don't overstress your resources either, setting impossible deadlines or work on your team, listen to them and know when the grumblings are real and not just a sound off of something else.  If you don't know the mood of your team, you are not in sync with them.
• “By employing the officers of his army without discrimination”.  This is a warning to put the right person on the right job, match skills to the task at hand.

One last thing that is mentioned here and I can bring this to two words “Know Thyself”.  Finishing a task comes down to a couple of things;

• “If you know the enemy and know yourself, you need not fear the result of a hundred battles.”  Know your abilities and what it may take to deal with the bugs you find and you will succeed.
• “If you know yourself but not the enemy, for every victory gained you will also suffer a defeat.”  Knowing your abilities but not what it will take to determine what a bug is and you can not only waste time, but yourself, in trying to determine something without reward.  True there are learning opportunities but these should be done when a schedule allows, when you have 2 days left in a project and you need to learn Java to discern a problem in the application this is not the time to do this yourself.
• “If you know neither the enemy nor yourself, you will succumb in every battle.”  If you are unsure of your abilities and what it takes to determine what the bug is this is what I call being over your head, ask for help or come clean with your manager on what you know or don’t know.  Don’t be the person to bring the project down, that will not earn you any friends.

I do like this quote here as well attributed to Chang Yu:  "Knowing the enemy enables you to take the offensive, knowing yourself enables you to stand on the Defensive... Attack is the secret of defense; defense is the planning of an attack."

Finally, 3 down!

2007-Feb-5 - Testing With The Man - Chapter 2

Now that you have enjoyed Chapter 1 – Laying Plans, we come to Chapter 2 – Waging War; which strangely is not really about war itself, but what it takes to do it.  I may make some allusions to certain current political situations, but if so and they offend you my apologies, I am just trying to draw parallels and any judgments made are my own.

As this part starts out there is a distinct drawing of men per chariot, this is done to give a method by which to structure the army, the structure was determined within Chapter 1.  What this gives is a method by which you know the breakdown, and how it fits into the framework that was determined early on.  In a project you have a strategy and a framework that will be what you work with; then you flesh it out with numbers and actual details, basically going from the general to the specific.  In writing out a general SWAG (simple wild assed guess) on how much time a certain component may take to test, to then determining the steps taken to do the work, and how much time each of those steps will take.  There is also a requirement that the soldiers must have enough provisions to carry them a thousand LI; a LI is generally 2.78 per mile today, so you have to not only make sure of your details but that you also have the stuff to get you there.  Sun Zi went into detail on what it takes to get an army somewhere, including expenditures on the home front, he was nothing if not detailed.

That stuff will vary by culture, whether its coffee, tea, Mountain Dew, Lucozade, chai or whatever Code Monkeys like these days.

Sun Zi also realized that a long term battle can cause equipment failures as well, “if victory is long in coming, then men's weapons will grow dull and their ardor will be damped.”  Just as in a project, the longer it goes on the more tiring it can be and oftentimes the more people will just want it to be over.  I’ve been on long projects, some extending immediately after release to a maintenance mode, right when you also need to be preparing for the next project, and this can be difficult.  Planned right it can go well, but again this is where you either plan properly and you can succeed or you plan badly and you fail, having the milestones in the project to know where you are going, and whether or not you are on your way to victory (a good release) or defeat (you’ll be spending the next few weeks working late nights and weekends).  To stay sharp and on focus with the project means making sure you know your own limits, as well as keep on top of anything that will push you off track, don’t rush through the project but don’t drag it on either, it’s a delicate balance that takes experience to realize and understand. 

 “Again, if the campaign is protracted, the resources of the State will not be equal to the strain.”  Here is one I am going to make a dangerous parallel on.  War is an exhaustion of resources, of all kinds, so you want to use them effectively.  In a project if there are to be Unit Tests and Automated Tests make them effective, not just to have them, but make sure you are going to get use out of the Tests not have them in your bag of tricks and use them as resume fodder.  If there is a GUI that needs tests get it done, no need for fancy stuff just because some new tool came out and it would look cool to know, unless doing that cool thing will also save you time on the project, if you get no efficiency from the work it may not be worth doing.  Now being American I can look at the current situation in Iraq, notwithstanding my own views, taking a look at the timeline there were definitely things that went wrong with planning there.  But just as important, as recent election results have shown, not only was short range planning off, and generated a larger mess than it should have been, but the long term planning in keeping the people who should be supporting this was also lacking.  People need to believe in the project and be on board with all of it, get buy-in, even from those who may not agree with you, the best way to look at a project and a plan is from many different sides, and then make a decision using all the information on hand.  My last comment for now on the whole Iraq situation is “There is no instance of a