2008-Jul-28 - Usability test tool 'Silverback'

Sites and services put more and more attention on usability.
When a real user is trying the site, she talks about what she feels about it. This information gives great inspiration on how to improve the site or service.
That is what Silverback is for. Although it's for Mac OS only and sells about $50,
I think this kind of software is necessary.
The software is able to record tester's operation, facial expression, and talk, and makes a Quicktime video.
I think the tool is simple but amazing for usability test.

However I have not tried it yet, since I don't have Mac OS.

2008-Jun-10 - Comparison of FITNESSE and NOSE

FITNESSE and NOSE are both test automation framework. FITNESSE is usually described as a functional and integration test tool. NOSE is usually said to be a unit test tool.

I evaluated both of the tools before we select one for our project testing. 

FITNESSE has a wiki UI, and uses tables to manage test cases and results.

NOSE does not have web UI, but all cases execution summary is displayed in command console.

Here are some more comparisons:

                                  |     FITNESSE    |    NOSE
1. Checkpoint/assert   | in the table         |   in the codes
2. File/Function          |                           |
   naming convention   |   camel               |    test_XXX
3. Case executed       |                           |
   sequence                 | as in the table     |  prioritized by case name
4. Result storage        |  in the table         |  in command console/has API

Conclusion:
FITNESSE is fit for data driven test cases, which need a table of data for similar case.
NOSE is fit for module test, with no UI, and flexible to use its APIs.

2007-Jun-5 - Notes of QuickTest Pro 8

1. During playback, after modify the content in an editbox, the Save button is still grayed/disabled. A workaround for it is to click on other components like Check Box on the same page. e.g. Browser( ).Page( ).Frame( ).WebCheckBox( ).Click.
2. Before test web UI, open Internet Options -> Advanced, enable "Display a notification about every script error".
3. Bitmap checkpoint is too strict, hence too much false positive. Note that the bitmap screenshot may have an extra purple frame around it that needs to be removed. 
4. Optional step allows QTP to skip some failed steps, instead of terminating the test.
5. If the Debug menu is grayed/disabled, go to Windows Add/Remove Windows Components, and install Script Debugger.
6. Test Batch Runner is a useful tool in QTP program package to run test cases in batch.
7. To run external programs, use "SystemUtil.Run" function.
8. Able to run DOS commands.
9. Other programs can call the automation scripts created by QTP:

WScript /e: VBSCRIPT {path to the script}

          For more information, refer to QT Automation Object Model Reference -> Configure Settings for Running a Test.

    10.  To parameterize test data in script:

for n=1 to 3

......=database("{dataname}", "{datatable}")

datatable.SetNextRow

next

 

     11. Difference between GetTOProperty, GetTOProperties, and GetROProperty:

GetTOProperty:  property from test object description

GetTOProperties:  collection of properties and values used to identify object

GetROProperty:  current value of test object property

 

2007-May-12 - Quick start guide - Sniffer

Sniffer Pro:

Example: Capture FTP transactions:

1) Define Filter -> Address:

    Set Address Type: IP, Mode: Include.

    Set Station 1. Dir. Station 2.

    -> Advanced:

    Check Availabe protocols -> IP -> TCP -> FTP (check other protocols as you need)

    Save.

2) Click Start button. And start the ftp transaction.

3) When Stop and Display button becomes activated, it means it has sniffed some packages. Click the button to stop and display the captured data.

4) Observe the Decode tab to find out the details.

5) Click Save button to save the capture.

6) Define Filter -> Data Pattern:

    Add Pattern: Choose the info you want to observe then click Set Data. eg. TCP: Source Port = 21.

2007-May-6 - Web UI test tools

1. Selenium

Open source.

Supports multiple platforms/browsers.

Does not support cross domain.

2. Quick Test Pro

Easy to insert steps in the middle of the original recorded steps.

Quickly recover from errors.

Reusable action.

Unintentional mouse movements during test case running would not fail the case.

Takes much time to add checkpoints. Bitmap checkpoint is not quite applicable.

3. Rational Robot

Supports multiple IDEs.

Supports multiple programming languages.

Not quite stable. Slow to recover from failure.

4. Mercury Winrunner

Also an automatic tool to record, examine, and simulate UI actions.