February 28, 2006 - Automation-Assisted Test Fest

On occasion, I have conducted some "automation-assisted manual testing" where our team of developers and QAers performed scripted manual testing while our automated tools simultaneously generated a load on the system.

It was very useful to have people actually seeing the system as it was slowing down (and sometimes failing).

We would usully hold a 1-hour "test fest" followed by a 1/2 hour discussion about what we saw.

I provided 10-20 written scenarios for people to follow during the session. "Festers" were asked to accomplish specific tasks and note any problems they observed, or unexpected slowdowns, along with the time the problems occurred.

During the followup, we talked about what we observed and I collected their notes. I used their input to correlate the problems found with the load on the system, and to write bug reports as needed.

It proved to be a rather useful, inexpensive, fun exercise which could flush out potential problems and give everyone more confidence in the system-under-test.