Testing in DarkAges!

Innovation in Testing?

In one of the discussion boards, I asked the members what innovation they see or seek in Software testing. Unfortunately I could not kindle a spark and did not get many responses. There can be many reasons behind this, like, lack of interest, lack of knowledge, lack of understanding.  The third was quite visible in some of the responses.

I think it is very much clear that we, as software testers, lack the interest of looking for innovative ideas even though there is no dearth of skill. Well, I think I will discuss that in some other post as it is in itself a hornet's nest. For now, I will focus on the responses I got on my question. For fitment purpose, I have tweaked, re-phrased, highlighted, or edited some of the responses.

1.
The major challenge is the generation and maintenance of collateral, even as data-driven testing has helped to streamline the process. With due respect, the maintenance of test cases has to be "dumbed down" not for lack of intelligence on the part of the testers, but due to the effort involved in trying to keep testing colateral current. Test automation tools like Certify aimed at that kind of simplification, but the business management tools, that would let you add/edit test cases quickly as needs change, leave a lot to be desired. I think that once you tackle that issue you can build on automation tools that would streamline the testing more effectively than what is the case today.

2.
"For me the challenge lies in the way i develop my test scenarios thinking like a user in innovative ways and execute the same use cases in real time environments and find bugs."

Darkage: Sorry, did not really make out what he wanted to say here.

3.
In my opinion, innovation in testing could be anything, which is simple to implement and reduces the test cycle time, reduces the cost involved in testing and at the same time improves the test effectiveness. This could be as simple as developing a macro in excel, which may help in automating certain complex calculations to test or automating report generation and analysis. Or it could even be complex like providing IQ to your automated tests so that it reduces human involvement, thus saving time and manual efforts.

4.
I think that the industry is really good at brainstorming strategy and process, but so may projects bog down somewere in the execution phase.

Without a clear problem statement and goals, throwing more innovation at a problem may just result in more scrap on the heap. Put differently, if no clear problem acutally exists then no amount of innovation can solve that non existent problem.

5.
I developed test automation tools that interfaced with Certify, albeit in a simple Excel-based manner with VBA code. Elements like pair-wise testing optimize the test cases, and the ability to pull data from a report to populate the test cases with current values is what being able to respond quickly is about.

Taking the "V-model" of development as a guide, I built a prototype that could be used to code the QA test scripts in parallel with the software code, with the intent that once it worked for acceptance/user testing it could be used for a test-harness based development paradigm as well. I used this tool to feed Certify, with great success, cutting the cost of testing by 90% or better when you incorporate regression testing.

------------------------

After receiving all these responses, I am still not clear what innovation do you see or seek in software testing. Don't you think testers are not clear about innovation? Ain't they willing? Are they aimless?

..think I'll have to stir up the hornet's nest.

Testing Best Practices..

If you ask me, I would strongly disagree that there are any best practices in testing. I have worked with many companies, big or small; and in my honest opinion, there are no best practices in testing. It is always and always context driven. You follow those practices that suit and fit well in your context. If you are testing a life or mission critical software, you will adopt different strategies than testing a "Hello World" program. Isn't it? 

I see that some people consider it quite unfortunate that we in testing do not have consistent methods, methodologies, terminologies, common ways of working etc. In a way, it is good too. Don't you think we will not be able to think "out-of-the-box" if a speciific way of thinking is imposed on us? Okay, let me put it in a simple way. As a tester, I always needed the space and time to think beyond a developer thinks. And for this, I never wanted to be bound in some typical processes and best practices.

However, one can adopt best practices for her testing project that are common in nature. I recently asked my team to adopt to some best practices and we were successful in delivering our project. I am listing down some of those practices that we followed:

1. Training and knowledge gathering: Since the system was new for us, we prepared training documents of our understanding. These docs included presentations, checklists and word docs. We got these docs endorsed by the business. We also maintained a somwhat similar set for new joiners of the team, so that they get to know the system without bothering others.

2. Project Structure in a checklist: I asked two of my team leads to prepare the project directory staructure in such a way that it doubles up as an audit checklist. The structure was broken down till fourth level and all these levels were hyperlinked to the exact location of files. The first level was the root of the project name.

Next thing that we did was to add a new column showing options like Avaiable and Not Available. This helped us in knowing what documention was available and upto what extent. This also helped us in more effective SQA reviews. (I will be honest here, I don't like SQA reviews. Those guys never put themselves in our shoes and don't want to understand how we delivered the product).

3. We planned for weekly brainstorming sessions. We also made sure that one of the team members play the role of a moderator. This not only help all the team members become aware of all the project activities, but also made them more confident.

4. Checklists for resource movements: Since resource movement was very high in this programme, we made a handover-takeover checklist for the smoother transition and to bridge

teh knowledge gap.

5. Mentoring: Every new or junior member was assigned to a mentor. Mentor was responsible for reviewing the email content of junior members being sent to teh client. We had to do that because client interaction was high and we did not want to take risk of miscommunication to clients.

6. Query resolution Process: A formal process for query resolution was defined as the domain, the technmology and the system wa new for us. We maintained a log of queries that was discussed as & when required. We made sure there are no repeat queries so that time can be saved.

7. One to One discussions:  Stakes were high and stress was high. I planned for 1:1 discussions with the team members so that they can use me as a punching bag and share their issues and concerns with me. I wanted to keep them as motivated as I could.

8. Reviews: We did at least two levels of reviews of all test artifacts so that we make less mistakes. This ensured the full coverage of testing and produced good quality test artifacts. We also made sure that every testers uses a Self-review checklist before publishing her test cases. It actually helped us minimize review and rwork efforts.

9. Matrices: Since stakes were high, we had high visibility to programme stakeholders. I was supposed to publish a snapshot of testing results to programme management everyday. I used MS Excel extensively to prepare project matrices.  And we made it quite a useful document by putting n-number of formulae and hyperlinking it with other sheets and workbooks. Similar spreadsheets were prepared for Release Management too.

The points that I made to the team was to keep in mind the attributes like Quality, Traceability, Reusability, Tracking & monitoring, Performance, Team & client Satisfaction and Knowledge.

We did it!

Flowers

I took this picture in Sea World. We were waiting for the ride Atlantis and this bunch was right next to me. I did not miss the opportunity clicking them.

5 reasons you will fail your Agile Testing project

In the last couple of months I have had many discussions, and many more arguments on the scope of Agile Testing in projects and the success rate of such projects. In most of these discussions, I feel that managers are trying to go for Agile testing by the half knowledge they gain from some not-so-authentic websites or through whatever they have learned from others.

I have attempted to observe and explore the five top-most reasons of failure in some agile testing projects and ways by which you can trounce these roadblocks.

1. Implementing Agile via traditional methodologies:

The biggest and most important reason of failure I have found is that people try to implement Agile testing keeping traditional methods in mind. They follow a waterfall, keep haphazard documentation, perform ad-hoc testing, listen to the project manager and fail in their testing. I have seen people telling me that agile testing is nothing but ad-hoc testing and they did not have to do any documentation. Well, I hope I know the reason they failed.

Further, there are Test Managers who believe in a no-strategy and no-plan for their agile project. Again, they are mistaken that an Agile project does not require any strategy or plan. One has to plan for an Agile Testing project.  For example, Extreme Programming projects keep their planning, monitoring & tracking simple so that next steps can be defined and Project completion can be forecasted. The planning may include Test Driven development or Test First approach, Automated Acceptance Testing, Testing early and iteratively, Testing often and whenever required, Using methods like exploratory testing and finally following a context-driven approach to testing.

2. There are Separate Development & Testing teams:

..And that they are rivals as they are in traditional software development projects.

..and that testers should not spare Developers by finding and reporting each & every defect.

..and that testers should ask for each and every details from developers to test the builds otherwise they should tell them that the entry criteria for testing does not meet.

With these assumptions you cannot be sure of success in your agile project because agility asks for high level of collaboration and integration between developers and the testers. In some Agile projects you may not find any testers, as development teams prepare the automated Unit tests and Client provides Acceptance tests. As much testing is done before and during the construction phase, less testing is required during the End game.

Another important thing that our traditional, individualistic managers forget is, Agile is best succeeded in pairs. Whether it is pair programming or pair testing; agilistic groups work in pairs with the single aim of delivering quality product faster with their simple designs.

3. All we have to do is ad-hoc testing:

I was talking to one of the test leads and he was telling me about his projects. When I asked about the Test Strategy he was using for the project, he told me that it was an Agile project and all they have to do is ad-hoc testing.

How mistaken he was! I did not dare him explain the methodology as he was convinced of what he & his project manager were doing. It was not just he who thought agile testing is just performing ad-hoc testing. I have seen many experienced managers talking in these lines.

So what kind of tests we do in Agile testing projects? In his article at Dr.Dobb’s journal, Scott Ambler has suggested many testing types that can be performed throughout the agile life cycle. Some of these are Confirmatory Testing, Investigative testing, System testing and Acceptance testing. We do not have to forget the Automated unit testing that is an integral part of any agile testing project.

One may refer to “Testing Extreme Programming” authored by Lisa Crispin and Tip House to be acquainted with testing in extreme programming environment.

4. It does not matter who works in my team:

Here I am talking about the (over) confident managers, who claim that they can get the work done by anyone.

Indeed, Agile projects are the games of pairing and integration and require team spirit in all their team members.  That is, not only your team members should be technically very sound, they should be sound socially as well. An Agile Testing project will fail if your team members are not socially communicative and have poor social skills.

Further, communication also plays an important role in Agile projects, as you do not have to play with heavy documentation as you do in traditional development methodologies. Here, all you have to do is to communicate, complete & deliver.  Some Extreme Programming experts discourage having people scattered across, floors, cities or countries. They believe that face-to-face communication is more important to make the project successful.

5. All I need is proper requirement documents:

First of all, you are not delivering your project through some traditional development method, where all you need is bundles of written documentation of customer requirements thrown over the wall for the testing team to write their test plans and scripts.

Agile projects are different. They do not get you ferry load of requirement documents. You have an onsite Customer who has his acceptance tests ready and is clear about his requirements.  He is available to give feedback and clarify acceptance criteria so that frequently delivered pieces of software remain in sync with what he wants. 

As I said earlier, one of the fundamental principles of agile projects is that they need collaboration and integration. And this does not spare the customer too, because she is the one who is driving the project and defining the requirements. If your customer is bad, you will not be able to deliver the good software.

Basically, any project will fail if you do not follow the basic principals of the methodology you are using. All flavors of Agile emphasize on Collaboration, Communication, Integration, Simplicity, Respect, Feedback and Ability to adapt to the changing business requirements. If you miss any of these, your project may fail. The reasons of failure I have described in this article are not the only ones, there may be more reasons that can fail you. However, my experience is that by avoiding even these once, you can save your project.

Invitation to GTAC 2009

Just today I got the invitation from Google for attending GTAC 2009 in Zurich. So, I think I am one of those lucky ones who have been called for the Google Test Automation Conference. Unfortunately, my presentation did not get selected. However, I am glad that at least I have been chosen for the attendance.

I heard one of the best parts of the conference are its roundtable discussions. I also sent few suggestions about my favorite topics of discussion. As you know, this years discussion is aroun 'Testing for the Web', which is quite an interesting topic. All the conference details are published at their site and also on their blog.

I am glad that I have been invited, at the same time I am a little worried too, as few of my critical releases are in the second week of October.

Keeping my fingers crossed!

Testing existed in Dark Ages too!

Heh! Just today someone asked me why I named my blog 'Testing in Dark Ages", while there was no testing existed in the dark ages. I think she is wrong. Testing was always there, even when humans were not eveolved and were either in the form of a fish or evolved as an ape. (well, if you do not agree with the ape theory, I can't help it. I cannot convince anyone on this topic.).

You may ask me how we tested as a fish or as an ape. It's simple, testing is a form of observation. Or, in other words, we can say that testing is observing. As humans, we observe things using our five senses and indirectly by using our brains. As a software tester, we use mathematical calculations, testing theories, matrices and various methods to test a program or to observe whether that program is working fine or not. We also use automation tools and write scripts to make sure our observations are correct. So, while in dark ages, they might have used their senses for observing things. And this observation was testing. This is what that convinced me that testing was always there, even in dark ages...that's why my blog is called "Testing in Dark Ages".

Last, but not the least, I believe man started using tools in stone age, possibly that was the time he became a developer. :)

WEB2.0: Tester is the User; User is the Tester

This is what I and my team colleagues Rohit Jain were discussing about Web 2.0. We had a lot of things in our minds about this new technology and we even sent some of our members to attend workshops on new web technologies. I see the world changing so fast that if you lose a step, you find yourself too behind to get hold of it.

Actually, with the advent of Web 2.0 a new era has started in the world of Internet. It was a technical revolution that enabled users with limited or absolutely no programming knowledge to actively interact with the web world. This revolution liberated the user from being a silent reader who just retrieved information from the Internet to someone who could also add to it, entirely through his browser. This changed the Internet world from “Someone to Everyone” to “Anyone to Everyone”.

Web 2.0 is new, it is evolving, and it has a very high visibility as anyone can use it. The target audience is almost everyone, and with an increasingly large number of users accessing the services day by day, the requirement for speed, reliability, data consistency and performance are all crucial. The coupling between design and code is high which makes the structure highly complex. Hence even a small design change can lead to drastic code changes. Thus to achieve best results, superior performance and better user experience, testing is very critical. Considering the high user base and target applications of Web 2.0, it is important for the tester to step into the shoes of the actual user to ensure the desired usability. Also it is interesting to note that the person who is performing the testing could also be an actual user of these applications. So we can quintessentially say that “Tester is the User; User is the Tester”.

Web 2.0 reaches the world with a promise to provide better social networking, user-generated content on websites, active discussion forums, better search results, enhanced information sharing, multimedia sharing, interactive collaboration and a lot more. This calls for greater ‘user perspective oriented’ testing.  Web 2.0 is useful when information and knowledge needs to be shared with others, when you are working with people geographically dispersed and when improved & increased collaboration with others is required. This could be on a business / organisation level or on a personal level. For example Wiki-sites can be created for information and content sharing among various users. Various networking sites, On-line Photo and Video sharing applications, Blogging etc. are instances where Web 2.0 has touched the common man. From Wikipedia to Youtube, from Facebook to Blogger, from Flickr to Google Ad-sense, from eSnips to eBay, the more Web 2.0 reaches the world, the more emphasis needs to be given on testing.

Although the benefits and applications of Web 2.0 seem pretty glamourous in the coming future, the path it is not a roller-coaster ride. Earlier websites owned and controlled their web experience; from infrastructure to presentation, from business logic to content management, it was their own stuff. But with the advent of Web 2.0, the control has slipped from their hands and it now lies with the user. The 'own stuff of the website' has changed to 'own stuff of the user'. And if you want to give the user control of the contents, presentation and business logic, the companies / websites have to test everything besides their own stuff. Also there is a lot of Third Party content that needs to be taken care of. So testing here would involve cumulative testing of company content, third party content and the user content in order to provide a higher level of user satisfaction. And who are these users... we are! Even the testers, developers, every stakeholder, everyone is a potential user.

Story: Optimum utilization of resources

One of my friends shared this story with me. I think consumerism has gone to a state where we don't think before spending.

Buddha: OK…Tell me how I can help you?

Disciple: Master, My attire is worn out and is beyond the decency to wear the same. Can I get a new one please?

Buddha found the robe indeed was in a bad condition which needed replacement. He asked the store keeper to give the disciple a new robe to wear on. The disciple thanked Buddha and retired to his room. Though he met his disciple’s requirement, Buddha was not all that contended on his decision. He realized he missed out some point. A while after, he realized, what he should have asked the disciple? He went to his disciple’s place and asked him "Is your new attire comfortable? Do you need anything more?"

Disciple: Thank you my Master. The attire is indeed very comfortable. I need nothing more.

Buddha: Having got the new one, what did you do with your old attire?

Disciple: I am using it as my bedspread.

Buddha: Then…hope you have disposed off your bed spread.

Disciple: No…no…Master. I am using my old bedspread as my window curtain.

Buddha: What about your old curtain?

Disciple: Being used to handle hot utensils in the kitchen.

Buddha: Oh…I see…Can you tell me what they did with the old cloth they used in Kitchen?

Disciple: They are being used to wash the floor.

Buddha: Then, the old rug being used to wash the floor…????

Disciple: Master, since they were torn off so much, we could not find any better use, but to use as a twig in the oil lamp, which is right now lit in your study room….

Buddha smiled in contentment and left for his room.

GTAC-2009

The 4th Annual Google Test Automation Conference is being held on October 21st, 22nd, 2009 at Google's Zurich office.

The Google Test Automation Conference began in 2006 under the guidance and inspiration of Allen Hutchinson, then Test Engineering Manager. The first annual GTAC, then called LTAC, after its London location, was hosted at Google's London Office. Due to the tremendous success of LTAC it was decided that GTAC would be formalized as an annual Google conference for test engineers, the location rotating to cover diverse locations, interests and issues.

This year's conference is under the theme "Testing for the Web". We want to discuss test automation strategies, tools, and challenges present when creating applications for the web - realized both in the desktop and mobile environments. Please have a look at the call for proposals for more details.

Google also selects attendees. That is, one has to send an essay to show one's interest in the conference and on the basis of these essay's, Google chooses who they want to invite as an attendee.

Important Dates
August 1 - Deadline for presentation proposals
August 15 - Notification of acceptance
October 21+22 - GTAC conference (Zurich, Switzerland)

For more information, visit gtac site.

Why did I use Risk-Based Testing in my project