Testing in DarkAges!

GTAC 2009 talks now available

On youtube..

Testing & IT Magazines

Its been a long time I am reading & referring to Software Testing and other IT magazines in both online and printed formats. I think it would be a good idea if I share this with others. The reason is just not that others will benefit from it; but also that I may get some more names of magazines or other useful resources.

1. Better software magazine: A publication from SQE group, who are better know for their site stickyminds.com 

2. Testing experience: As they call it, 'A magazine fro professional testers", you can find some very good articles in them.

3. Software Quality Professional magazine: This magazine is published by American Society of Quality. It has various references to articles and papers.

4. Software Test and Performance Magazine: Published by BZ Media, this magazine focuses on Testing & Performance Testing. One can also download articles & white papers from here.

5. Quality Matters: They mention on their website that "Quality Matters is the first magazine in the area of Software Testing and Software Quality based in South East Europe."

6. Professional Tester: An English-language quarterly magazine targeting professional testers of software worldwide. Offers online articles, jobs, information about testing companies and event calendar..

7. T.E.S.T. Magazine: Available in both print & digital formats, it is a good magazine to read about testing.

8. Quality Magazine: There is a lot available in this magazine, provided you try to look.

There are many more magazines available for testers. But as I said, I am sharing only those which I check out.

Apart from these, there is one more that I refer to and that is TechRepublic, as it keeps me connected to the other world of IT.

Is formal testing education required?

Here are the excerpt of a discussion between a Test Manager (TM) and his subordinate (TE):

TM: So, you have been in testing for last many years?

TE: Yes. More than four years.

TM: Good. Do you read books on testing?

TE: Why?

TM: To get testing knowledge?

TE: Well, I think I've got enough testing knowledge by working in this field.

TM: Ummm, do you have a formal education or training in software testing?

TE: No. But I read a chapter on Software Testing during my graduation.

TM: Don't you think one should have formal training or education on testing? Or, that one should read at least some well recognized testing books?

TE: (smiles and leaves).

...and the question on formal software testing education remains unanswered.

Let me tell you that this discussion was not hypothetical. Testing education is still considered as a non-essential item by the academics and the indsutry. Except from CSTER, which had support from Florida Tech, I never heard of any other institute having testing in their course curriculum. Please correct me if I am wrong.

You may refer me to some of the companies which provide software testing trainings and certifications. Do you think those are dedicated to training & education, or they are doing it for making profits? I am not against making money, but the objective of education simply gets lost as soon as you focus on money matters.

Software Testing is a prominent field these days and in my opinion, we must seriously look at the need of proper education in this area too. For any computer science course, alongwith software programming, software testing should be added as subjects. Testing Courses must be designed at various levels. And this can happen only when industry starts asking for such people.

Testers are Idiots

If you read SDTimes, you might have read this story too. (Read full story at http://www.sdtimes.com/link/31789). They published it in March, 2008 and it was based on the assertions from audience of FutureTest conference (www.futuretest.net). The editor mentioned few assertions from audience like:

Testers are idiots. The practice of testing offers no innovation. Testing is boring, manual and repetitive. It’s not a career. Testers aren’t as smart as developers. They’re nitpicky, pencil-pushing quality/process geeks. They’re beside the point and are easily replaced. Testing is not a career; it’s a necessary evil between application users and the brilliance of developers.

So, why am I remebering this story after more than a year of this conference. Because, it reminds me of the punch line, Testers are idiots.

I wrote in one of my previous entries that we miss innovation in testing. Don't you think it is true in some sense? What additional have you done that was different than your usual way of work? Have you ever thought beyond Automation whenever someone mentioned 'innovation in testing'? Why is it that we testers only think of usage of commercial tools and processes when it comes to innovation? Do we lack innovation in testing? Are testers really idiots?

I, somehow, think it is not totally true. There are innovators in testing too. Unfortunately the number doesn't seem that big, though.

Testing is not boring, repetitive and just manual. I as a tester take interest in testing; I try to reduce number of defects by this repetitive process and I do try to automate the repetitive processes. Depending solely on certain commercially available tools may not be right, though.

Testing is very much a career. Would you like to travel in a plane which has not been tested for? Isn't it that because of some of us testers, many lives are saved? If I do not choose this career, will you be so sure of using medical equipments in the hospitals?

Agreed, some of us are nitpicky, pencil-pushing geeks. But you can find same breed in other areas too. Why just testing?

I know we lack innovation. There aren't many innovative ideas floating around in the testing hemisphere. But this is not an imperfection. We are living happily with t. And we are also trying to make it better. Ain’t you agree?

Innovation in Testing?

In one of the discussion boards, I asked the members what innovation they see or seek in Software testing. Unfortunately I could not kindle a spark and did not get many responses. There can be many reasons behind this, like, lack of interest, lack of knowledge, lack of understanding.  The third was quite visible in some of the responses.

I think it is very much clear that we, as software testers, lack the interest of looking for innovative ideas even though there is no dearth of skill. Well, I think I will discuss that in some other post as it is in itself a hornet's nest. For now, I will focus on the responses I got on my question. For fitment purpose, I have tweaked, re-phrased, highlighted, or edited some of the responses.

1.
The major challenge is the generation and maintenance of collateral, even as data-driven testing has helped to streamline the process. With due respect, the maintenance of test cases has to be "dumbed down" not for lack of intelligence on the part of the testers, but due to the effort involved in trying to keep testing colateral current. Test automation tools like Certify aimed at that kind of simplification, but the business management tools, that would let you add/edit test cases quickly as needs change, leave a lot to be desired. I think that once you tackle that issue you can build on automation tools that would streamline the testing more effectively than what is the case today.

2.
"For me the challenge lies in the way i develop my test scenarios thinking like a user in innovative ways and execute the same use cases in real time environments and find bugs."

Darkage: Sorry, did not really make out what he wanted to say here.

3.
In my opinion, innovation in testing could be anything, which is simple to implement and reduces the test cycle time, reduces the cost involved in testing and at the same time improves the test effectiveness. This could be as simple as developing a macro in excel, which may help in automating certain complex calculations to test or automating report generation and analysis. Or it could even be complex like providing IQ to your automated tests so that it reduces human involvement, thus saving time and manual efforts.

4.
I think that the industry is really good at brainstorming strategy and process, but so may projects bog down somewere in the execution phase.

Without a clear problem statement and goals, throwing more innovation at a problem may just result in more scrap on the heap. Put differently, if no clear problem acutally exists then no amount of innovation can solve that non existent problem.

5.
I developed test automation tools that interfaced with Certify, albeit in a simple Excel-based manner with VBA code. Elements like pair-wise testing optimize the test cases, and the ability to pull data from a report to populate the test cases with current values is what being able to respond quickly is about.

Taking the "V-model" of development as a guide, I built a prototype that could be used to code the QA test scripts in parallel with the software code, with the intent that once it worked for acceptance/user testing it could be used for a test-harness based development paradigm as well. I used this tool to feed Certify, with great success, cutting the cost of testing by 90% or better when you incorporate regression testing.

------------------------

After receiving all these responses, I am still not clear what innovation do you see or seek in software testing. Don't you think testers are not clear about innovation? Ain't they willing? Are they aimless?

..think I'll have to stir up the hornet's nest.

Testing Best Practices..

If you ask me, I would strongly disagree that there are any best practices in testing. I have worked with many companies, big or small; and in my honest opinion, there are no best practices in testing. It is always and always context driven. You follow those practices that suit and fit well in your context. If you are testing a life or mission critical software, you will adopt different strategies than testing a "Hello World" program. Isn't it? 

I see that some people consider it quite unfortunate that we in testing do not have consistent methods, methodologies, terminologies, common ways of working etc. In a way, it is good too. Don't you think we will not be able to think "out-of-the-box" if a speciific way of thinking is imposed on us? Okay, let me put it in a simple way. As a tester, I always needed the space and time to think beyond a developer thinks. And for this, I never wanted to be bound in some typical processes and best practices.

However, one can adopt best practices for her testing project that are common in nature. I recently asked my team to adopt to some best practices and we were successful in delivering our project. I am listing down some of those practices that we followed:

1. Training and knowledge gathering: Since the system was new for us, we prepared training documents of our understanding. These docs included presentations, checklists and word docs. We got these docs endorsed by the business. We also maintained a somwhat similar set for new joiners of the team, so that they get to know the system without bothering others.

2. Project Structure in a checklist: I asked two of my team leads to prepare the project directory staructure in such a way that it doubles up as an audit checklist. The structure was broken down till fourth level and all these levels were hyperlinked to the exact location of files. The first level was the root of the project name.

Next thing that we did was to add a new column showing options like Avaiable and Not Available. This helped us in knowing what documention was available and upto what extent. This also helped us in more effective SQA reviews. (I will be honest here, I don't like SQA reviews. Those guys never put themselves in our shoes and don't want to understand how we delivered the product).

3. We planned for weekly brainstorming sessions. We also made sure that one of the team members play the role of a moderator. This not only help all the team members become aware of all the project activities, but also made them more confident.

4. Checklists for resource movements: Since resource movement was very high in this programme, we made a handover-takeover checklist for the smoother transition and to bridge

teh knowledge gap.

5. Mentoring: Every new or junior member was assigned to a mentor. Mentor was responsible for reviewing the email content of junior members being sent to teh client. We had to do that because client interaction was high and we did not want to take risk of miscommunication to clients.

6. Query resolution Process: A formal process for query resolution was defined as the domain, the technmology and the system wa new for us. We maintained a log of queries that was discussed as & when required. We made sure there are no repeat queries so that time can be saved.

7. One to One discussions:  Stakes were high and stress was high. I planned for 1:1 discussions with the team members so that they can use me as a punching bag and share their issues and concerns with me. I wanted to keep them as motivated as I could.

8. Reviews: We did at least two levels of reviews of all test artifacts so that we make less mistakes. This ensured the full coverage of testing and produced good quality test artifacts. We also made sure that every testers uses a Self-review checklist before publishing her test cases. It actually helped us minimize review and rwork efforts.

9. Matrices: Since stakes were high, we had high visibility to programme stakeholders. I was supposed to publish a snapshot of testing results to programme management everyday. I used MS Excel extensively to prepare project matrices.  And we made it quite a useful document by putting n-number of formulae and hyperlinking it with other sheets and workbooks. Similar spreadsheets were prepared for Release Management too.

The points that I made to the team was to keep in mind the attributes like Quality, Traceability, Reusability, Tracking & monitoring, Performance, Team & client Satisfaction and Knowledge.

We did it!

5 reasons you will fail your Agile Testing project

In the last couple of months I have had many discussions, and many more arguments on the scope of Agile Testing in projects and the success rate of such projects. In most of these discussions, I feel that managers are trying to go for Agile testing by the half knowledge they gain from some not-so-authentic websites or through whatever they have learned from others.

I have attempted to observe and explore the five top-most reasons of failure in some agile testing projects and ways by which you can trounce these roadblocks.

1. Implementing Agile via traditional methodologies:

The biggest and most important reason of failure I have found is that people try to implement Agile testing keeping traditional methods in mind. They follow a waterfall, keep haphazard documentation, perform ad-hoc testing, listen to the project manager and fail in their testing. I have seen people telling me that agile testing is nothing but ad-hoc testing and they did not have to do any documentation. Well, I hope I know the reason they failed.

Further, there are Test Managers who believe in a no-strategy and no-plan for their agile project. Again, they are mistaken that an Agile project does not require any strategy or plan. One has to plan for an Agile Testing project.  For example, Extreme Programming projects keep their planning, monitoring & tracking simple so that next steps can be defined and Project completion can be forecasted. The planning may include Test Driven development or Test First approach, Automated Acceptance Testing, Testing early and iteratively, Testing often and whenever required, Using methods like exploratory testing and finally following a context-driven approach to testing.

2. There are Separate Development & Testing teams:

..And that they are rivals as they are in traditional software development projects.

..and that testers should not spare Developers by finding and reporting each & every defect.

..and that testers should ask for each and every details from developers to test the builds otherwise they should tell them that the entry criteria for testing does not meet.

With these assumptions you cannot be sure of success in your agile project because agility asks for high level of collaboration and integration between developers and the testers. In some Agile projects you may not find any testers, as development teams prepare the automated Unit tests and Client provides Acceptance tests. As much testing is done before and during the construction phase, less testing is required during the End game.

Another important thing that our traditional, individualistic managers forget is, Agile is best succeeded in pairs. Whether it is pair programming or pair testing; agilistic groups work in pairs with the single aim of delivering quality product faster with their simple designs.

3. All we have to do is ad-hoc testing:

I was talking to one of the test leads and he was telling me about his projects. When I asked about the Test Strategy he was using for the project, he told me that it was an Agile project and all they have to do is ad-hoc testing.

How mistaken he was! I did not dare him explain the methodology as he was convinced of what he & his project manager were doing. It was not just he who thought agile testing is just performing ad-hoc testing. I have seen many experienced managers talking in these lines.

So what kind of tests we do in Agile testing projects? In his article at Dr.Dobb’s journal, Scott Ambler has suggested many testing types that can be performed throughout the agile life cycle. Some of these are Confirmatory Testing, Investigative testing, System testing and Acceptance testing. We do not have to forget the Automated unit testing that is an integral part of any agile testing project.

One may refer to “Testing Extreme Programming” authored by Lisa Crispin and Tip House to be acquainted with testing in extreme programming environment.

4. It does not matter who works in my team:

Here I am talking about the (over) confident managers, who claim that they can get the work done by anyone.

Indeed, Agile projects are the games of pairing and integration and require team spirit in all their team members.  That is, not only your team members should be technically very sound, they should be sound socially as well. An Agile Testing project will fail if your team members are not socially communicative and have poor social skills.

Further, communication also plays an important role in Agile projects, as you do not have to play with heavy documentation as you do in traditional development methodologies. Here, all you have to do is to communicate, complete & deliver.  Some Extreme Programming experts discourage having people scattered across, floors, cities or countries. They believe that face-to-face communication is more important to make the project successful.

5. All I need is proper requirement documents:

First of all, you are not delivering your project through some traditional development method, where all you need is bundles of written documentation of customer requirements thrown over the wall for the testing team to write their test plans and scripts.

Agile projects are different. They do not get you ferry load of requirement documents. You have an onsite Customer who has his acceptance tests ready and is clear about his requirements.  He is available to give feedback and clarify acceptance criteria so that frequently delivered pieces of software remain in sync with what he wants. 

As I said earlier, one of the fundamental principles of agile projects is that they need collaboration and integration. And this does not spare the customer too, because she is the one who is driving the project and defining the requirements. If your customer is bad, you will not be able to deliver the good software.

Basically, any project will fail if you do not follow the basic principals of the methodology you are using. All flavors of Agile emphasize on Collaboration, Communication, Integration, Simplicity, Respect, Feedback and Ability to adapt to the changing business requirements. If you miss any of these, your project may fail. The reasons of failure I have described in this article are not the only ones, there may be more reasons that can fail you. However, my experience is that by avoiding even these once, you can save your project.

Invitation to GTAC 2009

Just today I got the invitation from Google for attending GTAC 2009 in Zurich. So, I think I am one of those lucky ones who have been called for the Google Test Automation Conference. Unfortunately, my presentation did not get selected. However, I am glad that at least I have been chosen for the attendance.

I heard one of the best parts of the conference are its roundtable discussions. I also sent few suggestions about my favorite topics of discussion. As you know, this years discussion is aroun 'Testing for the Web', which is quite an interesting topic. All the conference details are published at their site and also on their blog.

I am glad that I have been invited, at the same time I am a little worried too, as few of my critical releases are in the second week of October.

Keeping my fingers crossed!

WEB2.0: Tester is the User; User is the Tester

This is what I and my team colleagues Rohit Jain were discussing about Web 2.0. We had a lot of things in our minds about this new technology and we even sent some of our members to attend workshops on new web technologies. I see the world changing so fast that if you lose a step, you find yourself too behind to get hold of it.

Actually, with the advent of Web 2.0 a new era has started in the world of Internet. It was a technical revolution that enabled users with limited or absolutely no programming knowledge to actively interact with the web world. This revolution liberated the user from being a silent reader who just retrieved information from the Internet to someone who could also add to it, entirely through his browser. This changed the Internet world from “Someone to Everyone” to “Anyone to Everyone”.

Web 2.0 is new, it is evolving, and it has a very high visibility as anyone can use it. The target audience is almost everyone, and with an increasingly large number of users accessing the services day by day, the requirement for speed, reliability, data consistency and performance are all crucial. The coupling between design and code is high which makes the structure highly complex. Hence even a small design change can lead to drastic code changes. Thus to achieve best results, superior performance and better user experience, testing is very critical. Considering the high user base and target applications of Web 2.0, it is important for the tester to step into the shoes of the actual user to ensure the desired usability. Also it is interesting to note that the person who is performing the testing could also be an actual user of these applications. So we can quintessentially say that “Tester is the User; User is the Tester”.

Web 2.0 reaches the world with a promise to provide better social networking, user-generated content on websites, active discussion forums, better search results, enhanced information sharing, multimedia sharing, interactive collaboration and a lot more. This calls for greater ‘user perspective oriented’ testing.  Web 2.0 is useful when information and knowledge needs to be shared with others, when you are working with people geographically dispersed and when improved & increased collaboration with others is required. This could be on a business / organisation level or on a personal level. For example Wiki-sites can be created for information and content sharing among various users. Various networking sites, On-line Photo and Video sharing applications, Blogging etc. are instances where Web 2.0 has touched the common man. From Wikipedia to Youtube, from Facebook to Blogger, from Flickr to Google Ad-sense, from eSnips to eBay, the more Web 2.0 reaches the world, the more emphasis needs to be given on testing.

Although the benefits and applications of Web 2.0 seem pretty glamourous in the coming future, the path it is not a roller-coaster ride. Earlier websites owned and controlled their web experience; from infrastructure to presentation, from business logic to content management, it was their own stuff. But with the advent of Web 2.0, the control has slipped from their hands and it now lies with the user. The 'own stuff of the website' has changed to 'own stuff of the user'. And if you want to give the user control of the contents, presentation and business logic, the companies / websites have to test everything besides their own stuff. Also there is a lot of Third Party content that needs to be taken care of. So testing here would involve cumulative testing of company content, third party content and the user content in order to provide a higher level of user satisfaction. And who are these users... we are! Even the testers, developers, every stakeholder, everyone is a potential user.

GTAC-2009

The 4th Annual Google Test Automation Conference is being held on October 21st, 22nd, 2009 at Google's Zurich office.

The Google Test Automation Conference began in 2006 under the guidance and inspiration of Allen Hutchinson, then Test Engineering Manager. The first annual GTAC, then called LTAC, after its London location, was hosted at Google's London Office. Due to the tremendous success of LTAC it was decided that GTAC would be formalized as an annual Google conference for test engineers, the location rotating to cover diverse locations, interests and issues.

This year's conference is under the theme "Testing for the Web". We want to discuss test automation strategies, tools, and challenges present when creating applications for the web - realized both in the desktop and mobile environments. Please have a look at the call for proposals for more details.

Google also selects attendees. That is, one has to send an essay to show one's interest in the conference and on the basis of these essay's, Google chooses who they want to invite as an attendee.

Important Dates
August 1 - Deadline for presentation proposals
August 15 - Notification of acceptance
October 21+22 - GTAC conference (Zurich, Switzerland)

For more information, visit gtac site.