Irony struck firmly today when new UK site OnePlace, http://oneplace.direct.gov.uk designed to give the public an overview of how local government services are performing, is itself, um, not performing...

"The Oneplace website is busy

Unfortunately the Oneplace website is experiencing high volumes of traffic, please try again later.

We apologise for the inconvenience."

Well at least they apologised. That's nice.

I think this again shows that performance testing is key - one has to ask if peak load scenarios were considered and tested?

www.palm.com

Ok so the interwebs and dog are all talking about Palm's new Pre and WebOS. I'm a very long term Palm user (started with a Handspring Visor Deluxe - built like a brick and I swear they invented the word "ruggedised" to describe it) but living in the UK I have only recently had an opportunity to try out WebOS.

Which begs the question: with my Tester's hat on is it any good? Or to be more precise, is it intuitive and usable or just gimmicky? Well I went into an O2 store a couple of weekends back and I kid you not spent well over an hour playing with a display model of the Pre. Summary: it's bloomin' marvelous guvn'r!

First though a disclaimer: if you try WebOS on a Pre in the O2 stores make sure you turn off the WiFi connection - no wonder it seems so fast online ;)

Second an anti-disclaimer: swapping over to the 3G connection was like blazin lightning or something. This thing seriously moves. OK so I was in the centre of Bristol which has excellent mobile signal coverage, but honestly - playing YouTube videos in real-time over 3G was quite a novelty for me (coming from a Palm Centro with slow-as-a-tortoise GPRS only). I really didn't expect this and was very chuffed.

But back to the point of this post - WebOS itself and specifically usability.

The first thing that I observed was that loading, swapping between apps and closing them is 100% perfectly intuitive. Palm have hit the usability nail on the proverbial head here. To load an app: tap it (fair enough), to navigate apps you swipe left and right on a card display of previews (still good), and to close an app you flick it out of site upwards (so so good). It is clear that Palm really have thought about the simple act of opening and closing apps with a great deal of care and attention. And all this happens in real-time with instant feedback to the end user - another key point of usability.

This is also an absolute improvement over Apple's iPhone as well as here with WebOS we have solid, robust multitasking in action - and a way to view what apps are open (the side-by-side card previews) that also, as an aside, knocks Windows Mobile for six.

Ok what else? Well what about speed - swipe here, swipe there and it does and does again as you command. If it were a Cylon it really would yell "by your command" before and after every action as WebOS is incredibly responsive. As long as you keep the number of open apps to a reasonable level (and on the Pre "reasonable" turned out to be: YouTube running a video, 3 web pages, camera app, app store, preferences, the media browser and the media player - that's darn respectable if you ask me) then there should be no or minimal slowdown. This is another key point for usability: we humans hate to wait. Waiting makes our lives a chore. We want it now, thank you very much. And again, WebOS delivers on this front.

Feedback is one of those things you need in any device - pressing something, switching something else and turning yet another thing are all actions that are ingrained in modern technologically oriented humans. We need feedback. Well again WebOS delivers: drag your finger up from below the bottom of the screen in pretty much any app and your favourite apps are displayed in a wavy line across the screen. Drag left or right onto any of the displayed icons and lift finger: app loads. It takes a moment to actively realise that the feedback point is when one LIFTS one's finger from the device, but this works. Go into a store and try it out - it really does make sense.

Although not WebOS, a quick note about the slide-out keyboard. The kb is great, but the slide out mechanism could use a little work. It doesn't slide easily enough for my liking. Plus I found that I ended up spending most of that hour with it fully open... if only the kb was in a candybar form factor...

Well the above is but brief coverage of some of the usability highs for WebOS and the Pre based on my hour of fun in O2. I really can say Palm is back on top. I always loved PalmOS (even in the green screen days) as it just seemed to "get" me and the way I wanted to work. While WebOS is wholly different it is also wholly familiar and really makes sense. Given their recent downs and ups (to Foleo or not to Foleo...) Palm should be congratulated for producing an OS (and device) that really is the true successor to the venerable PalmOS.

So am I going to buy the Pre? Well ye...um... remember that candybar form factor I mentioned just now? Turns out that the Palm Pixi is on its way. So yes I will definitely absolutely be buying a WebOS device to replace my Centro, but whether it is the Pre or Pixi will have to wait and see. When the Pixi is in O2 in the UK I'll be the first in line to try it out next to the Pre (and would seriously consider going for the candybar even if it does have a slower CPU and no Wifi). More no doubt to come.

=C

PS: Every person during that hour who came into O2 spent easily twice the amount of time playing with the Pre and WebOS than they did with the iPhone. Interesting...

Well folks, it has finally happened...

The bank I bank with has decided that only requiring 2 digits to be typed in online when loging into online banking makes their site somehow more "secure". They quote on their site that this will help protect against key loggers - which shows their Test/QA department can't use any test automation ;) If they did they'd instantly know that capturing data from drop-down lists is easy as a-b-c (hey we do it all the time), and replaying the data as easy as c-b-a. What is more, it is EASY to capture such data without the user ever knowing (which makes for a great April 1st joke... apparently).

I phoned them and spoke to a member of their Technical Department and asked him about this today and his response was (paraphrasing as I wasn't able to record the call):

1) "It DOES protect against key loggers as it means that a key logger won't get the data in one go" (to which I replied: "big deal - presumably the Average Joe will log into their bank account a few times a day when your system times out giving the logger all the ammo it needs?");

2) "And anyway even if someone gets the PIN they still need the second piece of information" (to which I replied: "do you mean that publicly available 'place of birth' information?")

Oh and if it is truly the case that a fraudster getting a PIN does not matter (I sure hope they recorded the phone call!) then why ask me for ANY PIN?

This is a brilliantly bad example of a company putting its toes into the security soup without asking the number one fundamental question:

DOES THIS PROPOSAL ACTUALLY MAKE IT ANY SECURER... REALLY... DOES IT...?

My take.. the answer is a resounding "No". I'm sure they would argue otherwise - but just in case a member of their Test/QA is reading this... stick your hand up, question this decision, ask and refuse to budge until taken seriously.

The oddest thing is that after stating the above I ought to say that in principle I am in favour of asking for "x" digits out of "y", but just not 2 out of 4 (as this represents just a 1 in 100 chance of getting the right combintaion) - and certainly not on the pretence that this provides some measure of protection against a key logger. As a starter for 10, how about 3 digits out of 6 as per some other companies? Better still, how about revisiting the whole premiss by which this decision was made.

-C

I've just purchased a bundle of 10 Apple Mac apps from:
www.givegoodfood2yourmac.com

The interesting thing is that if you pick carefully you can actually get 10 apps for less than the price of 3 because purchasing 3 apps gives a 30% discount, and 10 gives a 70% :)

If you want to take the opportunity to load up the Macs in your test lab with a large number of apps for minimum cost this could be the time to do it. Offer appears to expire in about 8 days.
-C

Yes you read that correctly. Today, a new Windows 2003 installation for our Test Lab needed some updates. In summary:

Windows had to update Windows Update to allow an update to be applied to Windows Update.

Well, after Windows had updated blah blah.. Windows Update then checks for any new updates - and top on its list of suggestions is an "upgrade" from Windows Update to Microsoft Update, which to my mind makes it:

Windows had to update Windows Update to allow an update to be applied to Windows Update, which when applied then checked for further updates and determined that Windows Update could be further updated.

Try saying that backwards!

Far too much for me to take in after only half my morning coffee...
-C

If you are considering purchasing an automated testing tool as well as the obvious questions like "does the feature set it provide match the objectives I have set?", one of the questions I feel you should ask is "will the company selling me the tool provide good quality support?"

I've written about this before elsewhere but I'm going to say it again here - Compuware Support has yet again exceeded my expectations.

We came across a problem with our QADirector database yesterday... something about a missing table in the schema. I assigned this problem to one of my Test Engineers, asking them to contact Compuware to resolve the problem. Compuware took his details and a Support Engineer phoned him back promptly.

The Engineer spent an hour or two on the phone diagnosing and rectifying the problem (without seeming rushed or in a hurry to close the call), and also demonstrated to my Engineer some of the advantages of the latest version of QADirectory.

End result:
    Problem - Fixed
    Customer (me) - Happy

-C

Today we ran out of disk space on a server in the Test Lab. It's running Windows 2000 Server so I dutifully (as one does) went into Disk Cleanup and let it do it's thang. After pondering the universe for 30 minutes Windows reported a message along the lines of (paraphrasing as I forgot to write it down): "the following files have not been used recently and may safely be compacted or deleted".

Oh goody thinks me - Windows has for once done something helpful...

...then everything goes a bit Pete Tong. After allowing Windows to compact and delete the files that IT says are NO longer needed I get a System pop-up message from Windows File Protection:

"Files that are required for Windows to run properly have been replaced by unrecognized versions. To maintain system stability, Windows must restore the original versions of these files. Insert your Windows 2000 Server CD now."

This made me laugh, then cry, then laugh some more, and finally shrug with indifference - I guess I should have expected Windows to mess up like this.

Moral of the story (and one I should have learned already ;) - never ever trust anything that Windows tells you.
-C

For my test lab at home (do you have one too?) I took delivery of a "new" server yesterday. It's a refurb Sun Cobalt RAQ 4r running CentOS (community version of RHEL) and an updated BlueQuartz under the brand name of Strongbolt from the nice ppl at osoffice.co.uk.

IMHO a very good deal - for £130 UKP (inc. P&P) it came fully configured, ready to run out of the box. Setting up virtual site hosting via the included BlueQuartz interface was relatively easy - and where I did get stuck, the forum:

www.osoffice.co.uk/forum

came to my aid (ahhhh that's how you get phpMySQLAdmin working..!)

Oh and as a plus it looks well cool as well ;)

If you want to test web sites in a "production" environment without having to actually upload to your production server then the Cobalt / Strongbolt servers are definitely worth considering. I got mine from osoffice.co.uk 's eBay member rollistag, there's one up for sale at the mo:

http://cgi.ebay.co.uk/ws/eBayISAPI.dll?ViewItem&item=330047425777

or you could try osoffice.co.uk direct.

To-date not a single one of the UK Government's IT projects has delivered the desired functionality to a high quality standard on time and to budget. Not one, nadda, nothing... In the UK this is becoming something of a joke with endless "we have learned from our mistakes and will try harder next time" speeches from this politician or that civil servant.

So why the following:

http://www.silicon.com/publicsector/0,3800010403,39163450,00.htm

??

To quote the usually very reliable Silicon.com:

"The government won't test all of the technology underpinning its ID cards plans before the project goes live, it has revealed..."

I'm suffering a real sense of Deja Vu here (was it the same black cat?)

Well, a week after downloading Build 5600 of Windows Vista RC1 I finally got the obligatory "it's ready" email from Microsoft... only it appears just a bit more testing may have been useful because, you see, it just seems far too easy to crash...

Thing is I really wanted to test our company's software against the new logged-into-a-domain-capable Fast User Switching, but I can't. That thing which is more of a fact of life than the fact that one day we will all die, namely Mr Blue Screen of Death reared its ugly head - and yes my computer floundered on its back, wiggled its toes in the air and collapsed in a puddle of odd looking core dump messages.

Is it too much to ask that I should be able to log in with just two users, and swap between them just four times without losing all my open and currently unsaved data in Vista because Mr Blue Screen has decided to intervene?

I know this is "only" RC 1, but I thought such to be a statement of quality - "we believe that this product is spiffingly tippity-top and we're going for a build that /may/ be suitable for release" or some such.

I look forward to RC 2 - may it last more than 30 minutes before it Blue Screens (will that be a record?) Only thing is according to the email RC 1 is "The final pre-release of Windows Vista..."

*flurble*

I am impressed with the Hong Kong postal service, and the no-doubt numerous companies in between "there" and "here". Three times in a row now an item bought from HK on eBay has arrived at my door in the UK faster than posting a "first" class letter from my house to:

 - my parent's (who live approx. 20 miles away);
 - my "local" tax office (based in the midlands, approx. 100 miles away).
 - my brother in law and his wife (who live just-over in Ireland);

You've got to admire the efficiency of a process that can get widgets to me from half way around the globe faster than the UK's Royal Mail can deliver a letter to my relatives!

I'd love to see the documentation for that process - must be some nuggets I can pull out to improve the testing process we use at work.

Anyone have a copy? ;)

The test lab at work continues to grow nicely, a sure sign that testing as an activity is being taken ever more seriously. This is good as having the right tools for the job means we can be more proactive and increase our coverage of any software build we are given to test. It also means we are amassing an ever increasing amount of packaging, mostly cardboard from the boxes it all comes in. I'm beginning to think that there is a relationship between the number of cardboard boxes, the number of mugs of coffee drunk I drink during the day (far too many) and our overall testing efficiency;)

I just wanted to share my pain of having to clear up yet another 2 dozen computer boxes last week (complete with enormous quantities of packing tape and SHARP industrial staples) - why can't there be a collect and return policy? Surely Mr Dell or Mr Jobs could collect their old computer boxes when they deliver new computers, and reuse them to package the items we buy on our next spending spree?

The one that really got me was a new hard disk that came in it's little Seagate branded box (fair enough) which was inside a cardboard box large enough for a mini tower (even though we only ordered the hard disk), which then was placed in a box almost big enough for a large server by the courier company. Well at least the drive arrived undamaged!

I'm suffering from geekedness. I just realised that I've set out on a mission to learn as much as possible about as many different hardware/OS platforms as possible in as short a time (as possible).

At home I have computers running Amiga OS 3, Fedora Core 4, Sun Solaris 10, Win 2k Pro, Win XP Home and Palm OS 5.

I just visited eBay and saw an SGI Octane 2 that looks yummy - and yes I'm interested because it has Yet Another (TM) OS (Irix 6.5) installed, on a completely different hardware architecture.

But I'll be good;) I'll wait till I've re-acquainted myself with Solaris first.

So what's the point I hear you cry (ie: what' this got to do with QA/Testing)?

Well, it's all about testing with fluency. You see, when I set about designing, writing and executing Test Plans, Schedules and Cases I like the whole thing to flow from A-Z, and this includes configuring the computers to perform such tests on. I find it a real disruption to the art and science of testing if I have to spend hour upon hour configuring something just to perform a single test because of my lack of understanding of a particular platform. Instead, I prefer to put in the leg work first to comprehend the system upon which I'll be testing, and then make use of the knowledge in a practical fashion: "Testing the Ay through Zee as easy as ABC".

My point? Well I've been caught off guard today... I'm here at work, at lunch, thinking "why o why can't I get it to work?" - and "all" I'm trying to do is something simple, a straightforward configuration... something that should just work (but doesn't). Now if only I had time to learn more about it all...

And that's my second point - Test Engineers are far too often expected to just "know" how to do something... "just go over to the server farm and set up an flibbetywhatsit with a whirlywibble and run through the test cases." Far too often, Test Plans and Schedules do not take into account the need for:

1. Training Test Engineers;
2. Allowing time for the Test Engineers to practice what they have learnt;
3. Giving adequate time to gain further knowledge through further experimentation.
   

I'll stop grumbling now and step down from soap box, and go back to getting it all to work...

I'd like to introduce you to a term I just made up: "CV Testing".

To me this is a term used when someone applies for a job and hasn't read their CV. Comments such as:
        "Does my CV say that?", or
        "I've never heard of it" (but it's on your CV...)
are probably not the best way to get a job.

Unfortunately some of the interviews for potential testers that I've been involved in over the years have gone this way, which is a shame, as an otherwise good candidate is let down by what appears to be pure fabrication, given that anything written on their CV is "fair game" in an interview. This includes hobbies and interests... "so Mr X tell me about your experiences in the world of supercharged snail racing?"

Have you noticed that there are plenty of schools teaching "Computer Science" but none that I know of teach "Testing Science"? (Let me know if I am wrong.)

After several years of professional testing experience in a variety of companies I remember back to an interview at a previous company in which I was asked "Why do you want to be a Tester? .. You clearly are intelligent enough to be a Developer" - grrrrr! Funnily enough I took the job, and spent the next 18 months re-educating the management of the company:

1. Testing is no different from any other science because Test Engineers use established scientific methodologies (planning, hypothesising, testing through experimentation, documenting results, peer reviews, etc);
2. Test Engineers have equal ability to the Software Engineers that they work with - you know it, I know it... but there are still many that don't;)
3. Test Professionals are (or should be!) trained to the same high and exacting standards as other Professionals.
   

I strongly see testing as a science. I know that many also see it as an art form, but to be honest the same could be said of development. Why am I a Test Engineer? Because IMHO I have the opportunity to practice the traditional scientific process that I was educated in as a part of my university degree (BSc (hons) Environmental Science in case you wondered).

I am a Tester and I am a Scientist - cool :)

Yes, keeping true to the traditions of Testing my first entry is a simple test of this new (and seemingly great) blog tool provided by SQA Forums / SQA Blogs .com