So, 2011 was a busy year working with SharePoint.  We did a major site upgrade where the old public site running on IIS was moved over to SharePoint, that was a major effort and took a lot of coordination with a lot of people.  It came off really well, and has been successful since not many people noticed the underlying change, only the new look of the site.  To me, that is a major success.

In addition I went through the task of learning SharePoint, as well as it's Search functionality, we had it going well for awhile then things just stopped.  We still have no idea why, but it's been slightly educational in learning how things work.  A nice environment where people just jump into things, help out where needed, and take the time to learn.  That's what makes this job great.

Testing was a big focus, not only because we added a new person to help test last year, though she moved on it was good to sort of settle things and document much more.  Teaching is always interesting since everyone comes in with a different focus and idea, that helps me readjust my thinking when I start to train someone.  From that I have more wiki pages, more places to document what we have been doing and more documentation on just everything.  I also got a start on automated testing, I now have a Framework with SpecFlow where I am now using BDD to create automated cases for Functional and Acceptance testing.  First time the Acceptance Test was used after a deploy we found an error, that is success to me.

I needed to add goals for the year, which I am bad at, for myself I have overall goals but they get subsumed by whatever is going on at work at the time.  Priorities come up and I need to learn something, say about writing a PowerShell script here or there, always something new.  Though I think I am better now at focusing more on what I need to do and I will be going through that as a task for 2012.  Should be an interesting year and I hope to do better about recording things here as well.

Since I am still in the midst of lots of SharePoint work, one of the recent items I started on was a quick list of things to check when troubleshooting.  As SharePoint is a framework and a platform it's often hard to find out where issues are coming from, so I use this as a guide to track down errors:

• Check ULS logs with the ULS Viewer
• Verify web.configs for the affected Site
• Check the Event Log on the Central Administration machine or the Web Front End for errors
• Turn on/off Custom Error Reporting for Local/Developer/Test environments only
• Check IIS logs on the Web Front End for any errors not caught by ULS (this is a rare case)
• Clear page cache
• Assure the SharePoint User Permissions are correct

I also have a few locations I check to see if anyone else has my same issues

• MSDN Forums
• SharePoint Overflow (soon to be StackExchange Overflow)
• The SharePoint Homepage, to check for updates

I've found SharePoint to be difficult to check sometimes because its not just a web server, often pages are encapsulated within what are called WSP files, these are basically templates that pull data from a database and build the pages.  It makes tracking issues down complex as often the problem is code related, not a missing bracket or end tage on a page.

The environment I work in now is hevily Window-ified.  This is due to the extensive use of PowerShell for the Build/Release processes and the fact the major project is a SharePoint installation.  Now that we have moved to Subversion and have packages coming out, sometimes with only small fixes, I decided I needed a way to be able to get some information on a package.  To be able to include this in the Build scripts I needed something in PowerShell that would query Subversion and give me a list of comments from all commits since the last release version.  This worked out well, took a bit to figure out, but it came out nice in the end.

function GetLog([string]$target, [int]$base_version, [int]$target_version, $relNotesDir="C:\Temp")
{
 if(($relNotesDir -eq $null) -or ($relNotesDir -eq "")) {
  "The " + $relNotesDir + " is empty. `n"
  $relNotesDir="C:\Temp"
 }
 if (!(Test-Path -Path $target)) {
  $target + " does not exist.`n"
  "Try the $target path again, make sure it is typed correctly and exists.`n"
  exit
 }
 if (!(Test-Path -Path $relNotesDir)) {
  "The Release Notes directory " + $relNotesDir + " does not exist.`n"
  "Enter an existing directory and try again.`n"
  exit
 }
 
 "Looking for changes from " + $base_version + " to " + $target_version + " in " + $target
 "Writing Release Notes out to " + $relNotesDir
 $releaseNotes = Join-Path -Path $relNotesDir  -ChildPath( "releaseNotes_" + $target_version + ".txt")
 $Command = "svn.exe"
 $Range = "" + $base_version + ":" + $target_version
 $Params = "log",$target,"-r",$Range,"--xml"
 $svnlog = [xml](& $Command $Params)
 $svnlog.log.logentry | foreach {
  if ( $_.revision -gt $base_version ){
   $a = ( $_.revision + " : " + $_.author + " `n " + $_.msg )
   $a >> $releaseNotes
  }
 }
}

Hope this helps someone out!

Previously I noted how I had written my original tests with Fitnesse to use the TestRunner and access Fitnesse in a way that worked, but as I recently discovered is no longer supported.  Oh well, so I update my tests.  Which was necessary anyway, since on the Build Machines where these tests would be run Fitnesse is not going to be running, and if it was I shouldn't need to usurp it for my tests.  So I moved over to using the RESTful method, which I had a hard time with earlier, but now is so insanely simple I wonder why I had a hard time with it before.

Without futher ado, here is the RESTful method to run Fitnesse tests.

First I set up my variables, I've also added in an archive file now, so I can keep track of my results, the RESTful results are nice so I actually output each test into its own results file, and I make an aggregate to be an archive.  Eventually I will get rid of the individual results and keep the archive around - this also let's me go onto any machine that ran a test and see how things have been going.  The Arrays I have are suites that I set up in Fitnesse to contain my tests, I have one for QA that I would run, and a Smoke suite that is a selected set to verify the build worked.

 

# Get the current directory

Push-Location .

# Variables for the script

# This location should not change, since its the standard checkout location

[string]$fitnesseLocale = "C:\MAIN\trunk\AutomatedTests\Fitnesse\Fitnesse"

[string]$fitnesseJar = "fitnesse.jar"

# Need to get the local machine here, it will be how Fitnesse runs

[string]$serverName = $env:COMPUTERNAME

[string]$portName = "8080"

# These are needed for the CheckFitnessProcess function,

# placing them here

$fitRunning = $false

# Set Java executable to use within PowerShell

$Java = "java.exe"

# Archive file

$archiveFile = "fitnesseArchive.txt"

# Use a Test Array to be able to bundle all the tests

# Tests that are commented out are having issues being run

# Smoke Testing is Development Based

# although its now geared towards checking Login

$smokeArray = ( "SpringsLoginLiteTests" )

# Tests for the QA environment

# although its now geared towards checking page links

$qaArray = ( "SpringsLinkChecks" )

# Tests to check how this script runs, this is a hidden command

# This just validates the Registration Verify Fields

$checkArray = ( "SpringsTest.UserLoginFfTests" )

# Location of the Test Results, this is hardcoded within the SVN structure

# so it should always be available, I am sure there are better ways of doing this

$testResultsDir = "C:\IHI_MAIN\trunk\AutomatedTests\Fitnesse\Fitnesse\testResults"

if (!(Test-Path -path $testResultsDir))

{

  New-Item $testResultsDir -type directory > $null

}

Then I still run my check on whether Fitnesse is running, it doesn't really need to be running anyway; so I decided that since REST only runs as needed then I was going to shut down Fitnesse if it was going.

# CheckFitnesseProcess determines if Fitnesse is running, if Fitnesse is not running

# then the script continues. The current test configuration does not require Fitnesse

# to run and does a better job when it's not.

# Adapted from: http://odetocode.com/Blogs/scott/archive/2006/07/17/5330.aspx

function CheckFitnesseProcess {

  param ([string]$AppPoolName="")

  $ProcessId = $null

  if ($AppPoolName.ToUpper() -ne $NOT_PASSED) {

    [string]$filter = $null

    if ((Get-Process java -ErrorAction SilentlyContinue) -ne $null) {

      Get-Process java | ForEach {

      # use WMI API to get process details

      $filter = "Handle='" + $_.Id + "'"

      $wmip = Get-WmiObject Win32_Process -filter $filter

      # CommandLine property has AppPool name; grab name using contains

      if($wmip.CommandLine.contains("fitnesse")) {

        # Uncomment the next line for debugging

        # Write-Host "Found this java process: $wmip.CommandLine `n"

        $fitRunning = $true

        # Saving the Process ID to stop the process later,

        # using a Global as I could not get this work if I declared

        # the variable at the beginning.

        $Global:fitProcess = $_.Id

      }

    }

  }

  if ($fitRunning -eq $false) {

    # All is good

  }

  if ($fitRunning -eq $true) {

    Write-Host "Fitnesse is running, it's best if its not - shutting it down...`n" -ForegroundColor Red

    # Stop the Fitnesse process

    Stop-Process -Id $fitProcess

  }

 }

}

I'm going to skip the Help function, if you need that it should be in my previous post, the important one is the Main function.

function main()

{

  # Need to be in the Fitnesse directory to access TestRunner, and

  # the Fitnesse batch file in case Fitnesse needs to be started

  cd $fitnesseLocale

  #Check and see if Fitnesse is running

  CheckFitnesseProcess

  foreach ($test in $testArray)

  {

    # Start the test

    $myDate = Get-Date

    Write-Host "`nRunning $test`n"

    # Build RESTful URL

    $testLink = $test +"?" + $testType +"&format=text"

    # Get the output from TestRunner so we can scrape it for test passes and fails

    $testResults = & $Java "-jar" "$fitnesseJar" "-c" "$testLink"

    Write-Host "Completed $test`n"

    # Write-Host "Test Results..."

    # Scan through the output, then display the results

    # Dump $testResults out to a File

    $testFile = $test + $myDate.Day + $myDate.Month + $myDate.Year + $myDate.Hour + $myDate.Minute + ".txt"

    $testResults | Out-File "$testResultsDir\$testFile"

    # Print out date to an archive file

    $testDate = "Last run time: " + $myDate.Day + "/" + $myDate.Month + "/" + $myDate.Year + " at " + $myDate.Hour + ":" + $myDate.Minute

    $testDate >> "$testResultsDir\$archiveFile"

    foreach ($line in $testResults)

      {

      if ($line -match "\A\.\s"){

        # This is a Pass

        Write-Host $line -ForegroundColor Green

        # Output to archive file

        $line >> "$testResultsDir\$archiveFile"

      }

      if ($line -match "\AX\s") {

        # This is an Exception, a warning but not a Failure

        Write-Host $line -ForegroundColor Yellow

        # Output to archive file

        $line >>

I'm running Fitnesse with SharePoint and in trying to automate my tests I ended up wrapping everything up in PowerShell, this is my first full script that handles all the tests I wanted to do.

First set up all the variables needed.

[string]$fitnesseJar = "fitnesse.jar"
[string]$fitnesseClass = "fitnesse.runner.TestRunner"
# Need to get the local machine here, it will be where Fitnesse runs
[string]$serverName = $env:COMPUTERNAME
[string]$portName = "8080"
# These are needed for the CheckFitnessProcess function, placing them here
$fitRunning = $false
# Since we need to manipulate this variable in a few different functions, its Global
[int]$Global:startTries = 0
# Set Java executbale to use within PowerShell
$Java = "java.exe"
# Use a Test Array to be able to bundle all the tests
$smokeArray = ( "FitNesseTestPageName1" )
$qaArray = ( "FitNesseTestPageName2" )

I use a function to check if Fitnesse is running, I wanted to only run it when I needed to and since this script can be automated I cannot be sure that Fitnesse will always be running

# CheckFitnesseProcess determines if Fitnesse is running; in which case
# it jumps down to running the tests, as long as Fitness is running the Check does
# not need to do anything else.
# If Fitnesse is not running then the script attempts to start it, does a
# check to make sure that Fitnesse started then runs the test.  If this script
# starts Fitnesse then it will attempt to clean up after itself and stop
# Fitnesse by doing a check on $startTries, if this is > 0 then this script
# started Fitnesse and can shut it down, otherwise it leaves the existing
# process running
# Adapted from: http://odetocode.com/Blogs/scott/archive/2006/07/17/5330.aspx
function CheckFitnesseProcess {
  param ([string]$AppPoolName="")
  $ProcessId = $null
  if ($AppPoolName.ToUpper() -ne $NOT_PASSED) {
    [string]$filter = $null
    if ((Get-Process java -ErrorAction SilentlyContinue) -ne $null) {
  Get-Process java | ForEach {
   # use WMI API to get process details
   $filter = "Handle='" + $_.Id + "'"
   $wmip = Get-WmiObject Win32_Process -filter $filter
   # CommandLine property has AppPool name; grab name using contains
   if($wmip.CommandLine.contains("fitnesse")) {
    # Uncomment the next line for debugging
    # Write-Host "Found this java process: $wmip.CommandLine `n"
    $fitRunning = $true
    # Saving the Process ID to stop the process later,
    # using a Global as I could not get this work if I declared
    # the variable at the beginning.
    $Global:fitProcess = $_.Id
   }
   # The next statement was put in to check for the processes, there was a problem
   # when updating Java that the process find command did not work, if this happens
   # again just uncomment the Write-Host line to find out what is going on, otherwise
   # this just gets bypassed but caught later on.
   if ($fitRunning -eq $false)
   {
    # Uncomment the next line for debugging
    # Write-Host "Did not find a Java/Fitnesse Process.  Found $wmip.CommandLine `n"
   }
  }
 }
 if (($fitRunning -eq $false) -and ($Global:startTries -lt 3)) {
  # Don't need to note that we have had 0 attempts
  if ($Global:startTries -gt 0) {
   # Need to give Fitnesse some attempts at starting, 3 is a good max
   Write-Host "Fitnesse is not running, have tried to start $Global:startTries time(s),`n"
  }
  $Global:startTries++
  startFitnesse
 }
    if (($fitRunning -eq $false) -and ($Global:startTries -eq 3)) {
  # At some point we need to give up starting it
        Write-Host "Fitnesse is not running, tried to start $Global:startTries time(s).`n"
  Write-Host "Strike Out!" -ForegroundColor Red
        Write-Host "Looks like Fitnesse cannot be started or found,`n"
  Write-Host "check the server and try again.`n"
  # Go back to where we started
  Pop-Location
        exit;
    }
 if ($fitRunning -eq $true) {
  # Success!!
  Write-Host "Fitnesse is good to go.`n" -ForegroundColor Green
 }
  }
}

Then I needed something to start Fitnesse, this uses the Fitnesse Batch script that comes with many of the installs

function startFitnesse()
{
 Write-Host "Attempting to start up Fitnesse...`n"
 # Starts the Fitnesse process using the batch configuration file
 # Not sure we need the runAs Admin but that can be adjusted
 Start-Process -FilePath fitnesse.bat
 # Giving it time to wake up
 Start-Sleep -Seconds 2
 # Need to check that the process actually started, do the check again
 CheckFitnesseProcess
}

Every script needs a help option

function myHelp()
{
 Write-Host "The Fitnesse Tester understands the following command line arguments:`n"
 Write-Host "    qa    - will run the QA Test Suites, a long set of tests`n"
 Write-Host "    smoke - which will run the Build Smoke Tests, a short set`n"
 Write-Host "`n Running this script with no arguments will display this message`n"
}

Then there is the main and rest of the script that runs everything

function main()
{
 # Need to be in the Fitnesse directory to access TestRunner, and
 # the Fitnesse batch file in case Fitnesse needs to be started
 cd $fitnesseLocale
    #Check and see if Fitnesse is running
    CheckFitnesseProcess
 foreach ($test in $testArray)
 {
  # Start the test
  Write-Host "`nRunning $test`n"
  # Get the output from TestRunner so we can scrape it for test passes and fails
  $testResults = & $Java "-cp" "$fitnesseJar" "$fitnesseClass" "-v" "$serverName" "$portName" "$test"
  Write-Host "Completed $test`n"
  Write-Host "Test Results..."
  # Scan through the output, then display the results
  # Dump $testResults out to a File
  $testResults | Out-File "$test.txt"
  foreach ($line in $testResults)
  {
   if ($line -match "Assertions:") {
    # Write-Host "$line"
    # Do a regular expression to get the Assertion info
    # Example: Assertions: 8 right, 0 wrong, 0 ignored, 0 exceptions
    $line -match ":\s(?\d+)\sr.*(?\d+)\sw.*(?\d+)\si.*(?\d+)\se.*"
    # Send a summary of the results to the console
    Write-Host "Passed:     "$matches.right -ForegroundColor Green
    Write-Host "Failed:     "$matches.wrong -ForegroundColor Red
    Write-Host "Ignored:    "$matches.ignores
    Write-Host "Exceptions: "$matches.exception -ForegroundColor Yellow
   }
  }
 }
 # Go back to where we started
 Pop-Location
 # Stop the Fitnesse process so we don't leave things running
 # But only if this script started it, which means $startTries is > 0
 if ($Global:startTries -ge 1){
  Write-Host "`nSince this script started Fitnesse, it'll be shut down now.`n"
  Stop-Process -Id $fitProcess
 }
}

# Check command line values to see what test is running
if ($args -eq "smoke")
{
 $testArray = $smokeArray
 Write-Host "Running Smoke Test.`n"
}
elseif ($args -eq "qa")
{
 $testArray = $qaArray
 Write-Host "Running QA Test Suite.`n"
}
elseif ($args -eq "check")
{
 $testArray = $checkArray
 Write-Host "Running Check Suite.`n"
}
else
{
 myHelp
 exit;
}

# Run main
main

I've moved onto REST but in case anyone else is trying to work with the TestRunner and PowerShell you have something you can start with, and not have to work it all from the beginning like I did.

I've been learning a lot about SharePoint over the past 9 months, the project I am working on is to bring a non-profit health community onto SharePoint and increase the offerings the non-profit has and to expand the social networking.  My work is revolving around testing the conversion of the site, making sure that what we end up with is going to be equivalent with what is currently in place.  It's interesting work and I've had some interesting tasks over the past few months.  Some of the more difficult work has been dealing with SharePoint and its eccentricities in order to assure that what we end up with is what was asked for.

Web Parts are like little applications that SharePoint uses to do all kinds of work, this could be anything from displaying a document library to a search result.  While they are like little applications Web Parts are quite different and aren't very testable on their own, I haven't seen much on them regarding Unit Testing yet and with their reliance on SharePoint data its probably not that easy.  Mostly the plan is to test the Web Parts in an exploratory fashion, when they are ready and added to pages they will be used and abused to put them through their paces.  Not the way I'd like to do it, but I am also pulling some information from the Microsoft page on how to test Web Parts.

Automation with SharePoint is kind of tough, not that its difficult to work with, but the page layouts and Web Parts don't always make things run smooth.  Web Parts can update like Ajax driven pages will, some of the ID's given by SharePoint for page elements can also be cryptic or dynamic.  It's not impossible, I have done tests with Selenium, Fitnium and Fitnesse after some time to work out the issues in calling page elements.  It just seems to take more time than it should.

The Social Networking aspects make things a little hard to automate everything, it would be nice to free myself up for other work but that's more a dream than a reality.  Testing Discussion Groups, Comments and Ratings means eyes need to be on the page as things are done in order to verify that SharePoint responds properly.

When trying to find errors there are two important places to look:

1. ULS Logs - Get the Microsoft ULS Log Viewer, if there is something happening real time, this is the place to see it happen.  Access to the SharePoint server under test is required to get this to work though.
2. Event Log - the Event Log is where alot of the captured errors go, if the Developers have not set this up in the code they should otherwise you lose and important resource.

I'll be putting in more links to some of the material I have captured over the past few months.

I got an IM from a developer I used to work with and he had that good old familiar question "do you have a copy of that graph that shows the cost of fixing a bug?"

Well I didn't at the time, mostly I dealt with this with the book I keep on Quality Assurance, it has the graph and a little blub about it.  Since he needed one online, and pretty much everything is on the internet I did a check and lo and behold, I found a developer centric version.

Rate of Diminishing Returns Of Fixing Software Bugs

He needed an online version because he had a VP who wouldn't drink the water on catching defects early, I guess there are still plenty of them out there.  Either way, I found the link and he went away happy, another QA-centric developer satisfied.

It feels good to help on a Monday.

I came into Regular Expressions using Perl, and I must say I liked it and enjoyed it immensely once I was able to actually figure out how to do the syntax, never mind doing captures in order to use specific values from strings later.  Although, like most things, once you do it then its simple and you think, "that was not so bad" when you look at something that looks like you kid typed when you left the computer on.

Perl was one thing, PowerShell is another.  Now that I have this figured out its simpler, but finding the information on it was not so simple, although I will say one thing that made it easier was having a Designer tool that went over some of it.  The Rad Software Designer is a very good tool for being able to help figure out the syntax as well as give you a place to assure that the results will be what you want, and since PowerShell is based off the .Net Regular Expression implementation the result will work fine there.

My goal was to be able to match a line like this:

    Assertions: 8 right, 0 wrong, 0 ignored, 0 exceptions

So I could get the numerical values for display, turns out I only needed this:

    :\s(?\d+)\sr.*(?\d+)\sw.*(?\d+)\si.*(?\d+)\se.*

...and then I could use some named capture values to then display the information I need.  Now if I could only figure out how to turn off the True statement being displayed, that would be great.

I am doing some of the Microsoft Hand On Labs (HOL's) for SharePoint 2010, normally I would post a link to these but they are so amazingly buggy with typos and code that doesn't compile (or write without errors appearing) that I would rather let anyone really wanting them to seek them out.  I downloaded them from Microsoft somewhere, they are from early 2009 and it looks like someone wrote them up quick and then somehow got them on the Microsoft site; I can't get past two pages without a typo and I hate that.  The code has errors, I know enough to fix some of the warnings that show up in Visual Studio 2010 when I write out the C# examples, so I would not push this on anyone.

Still, because of this I have had to go out and research errors now and again, what I have found are a couple of interesting posts on sites:

• Sharepoint 2010 beta release known issues
• (I encountered this in the lab I was running [HOL 8]) - Interesting little bug in VS 2010 beta-2

So far I am having middling luck in scripting many of the Web Parts I have been able to compile with Fitnesse, I have also found that I right now need to deploy and add Web Parts on the same environment.  I cannot deploy through Visual Studio 2010 on one machine then add that Web Part on another machine, I kept getting errors regarding that the object did not exist even though it appeared in the Web Parts list.  So, I am running things locally until I know they are on the site and working, then I go to another machine and script with Fitnesse.

Fitnesse also doesn't seem to like the naming conventions that SharePoint gives for certain objects, I am trying to find a solution for that as select lists may have their own ID's and Name's with the SharePoint strings in the future - so I am working that out now.

Enjoy!

Note: I've edited this page to now note the Labs, they are for 2010 and I figure why not share the fun.

Now that my current web redesign project is going to be based on SharePoint 2010, yes its in Beta and we know this but we are still going forward with this anyway...not my decision but it should be interesting, I have migrated some of the old Fitnesse scripts I had to the new platform.  In addition, I have upgraded from using the Fitnium add-on to FitLibrary and the SpiderFixture since it gives better Internet Explorer integration than Fitnium did, while FireFox will still be tested it won't be the focus that it was in the past.

So far I have encountered some issues with the way Fitnesse sees the pages, since SharePoint's web parts can act like Ajax some adjustments need to be made, also SharePoint masks some of the standard ID's of the base pages with its own ID scheme.  Some of those ID's are not only odd, but sometimes they are very long, in addition SharePoint's ability to be made of many dynamic pieces makes validating some pages very tedious.

In the past I did make Fitnesse a service on my machine, that way I always had it up and running, however, in recent releases I have had trouble with the connections from the browser to the site under test.  Today I noticed something odd, when I was running a new install on the command line everything worked, running the same script on the Fitnesse server that ran as a service there would be failures.  On the command line the browser would open a new instance and run its tests, on the service version nothing would show up.  So I did a check and sure enough I did a nice default, when I set up the service I set it to run as Local System, but when I run command line I run as myself.  Changing the Log On to myself, and granting myself access rights, I can now get the same scripts to pass on the new install.

I'm now running FitLibrary 20091021 release with Fitnesse 20100103, so far basic stuff looks fine and good, I just hope I can keep this running.

Since moving over to the WebDriver code for Fitnesse as a fixture, including the SpiderFixture extension, I have had better luck in getting some additional tasks to work.  One item I had been trying to do in Selenium, and never could get it to work, was in making entries into the RadEditor we use on the SharePoint server the product runs on.  With WebDriver there is an option to Execute JavaScript, I don't think it does well with multiple lines but I hadn't tried very hard to get things working in that way; what I ended up with was a single line of JavaScript that finds the iframe, adds the text then moves back to the base frame.  The frame moving took me a few minutes to realize, once you move to a particular frame on the page going back to the base frame is a good follow-up.

For those who can use it, the JavaScript to get the RadEditor to work is:

execute javaScript

$find(${RadEditoriFrame}).set_html(${test_text});

My RadEditoriFrame was set up as: ctl00_PlaceHolderMain_zoneMain_ctl00_txtPost

Best way to verify that this is the one to use, especially on SharePoint, is to use an Object Viewer on the page and verify the editor box is highlighted.

Hope that helps you out, it took me awhile to get this working.

I was in need of a Load Test Tool, now that I have Fitnesse up and running in my environment, I wanted to be able to check and see how our environment was going to behave under a significant amount of Users.  Significant at this point is more than 5, so in looking around I came back to Pylot, not that I needed another language installed on my machine but I liked the simplicity and the ease by which within a day I was up and running with tests that were already confirming to me that the pages I knew were slow, are indeed slow.

Check the Getting Started Guide  on how to get it up and running.  But there is one thing missing, if you want to run the Script Agent you will need the Python Win 32 Extensions in addition to the normal set up.

When running the Script Agent and running back the XML script I did get a couple of errors about:

    ERROR: can not parse testcase file: not well-formed (invalid token): line #, column #

It took a bit to work through this but basically what happened, in my case, was that the recording took in a NULL value somewhere in the CDATA line on the login and logout of my site, removing those allowed my script to run without a problem.

Once you get used to the XML format its easy to parameterize the testcases, and adding in new test steps for text validation and new pages, while the URL's are not the best mechanism in my site to use they do allow me to do a check against bookmarks on the site rather than clicking on the page elements.

I'm liking this tool a lot.

Some things I have learned when using Fitnium and Fitnesse.

SuiteSetUp and SuiteTearDown are not for actions, those need to go into regular tables, the SetUp and TearDown are not for Action Fixtures.

fit.exception.FitParseException: Can't find tag: table - typically means malformed HTML in a page but MOSTLY means you don't have a SetUp or TearDown in your Suite, and within those pages you need something, ANYTHING, even if its simple text.

Another possibility is if the wiki page containing the test has both the Suite and Test properties showing in the left property bar, both properties can also generate the error as its trying to run a Test but has nothing to do.

The jar file usually needs to be on any page its called on, don't trust the inheritance with the Parent pages or the Root page, you can try it but if you start getting function errors on the page - well now you know what needs to be added.

When adding URL's make sure they are not in camel case, especially URL's that have been parameterized otherwise the URL may not be sent to Selenium the way you think...and oh yes, PARAMETIZE everything.

In order to adjust for the multiple environments that the Fitnesse tests will be running in, within my environment later on, I have spent the time to parameterize the framework now and just added in some Symbolic Links.  The Symbolic Links is a wonderful feature for Fitnesse, as it allows a wiki page to be called from another, so I can set up one page with a test, or a set of tests, and have it called from another.  The true power in this comes from being able to set up multiple high level pages with specific environment information, so one set of tests can be run on Server A if its called from the Server A Test Page which has a Symbolic Link to Server Tests, I can also call the same tests from the Server B Test Page, which has different test settings, and reuse Server Tests within it.  Simple, yet powerful.

It did take a little bit of playing around to get the set up working, so here is what I found out.

• Create the page that will contain the Symbolic Link
• Go to Properties and set the Suite property on the page and save it, as this page is calling another Suite is needed so the Page will go down the Hierarchy
• Find the path that will be in the Symbolic Link
• Go to Properties and set the Name for the Link then the path to the Page, click on Create/Replace
• Save the settings and edit the page with whatever is needed, the Symbolic Link if used once is automatically called

I don't know what will happen with the need for multiple calls, I have seen examples where instead of looping a page was set up as follows:

    [Username1]

    [Password1]

    [Symbolic Link]

    [Username2]

    [Password2]

    [Symbolic Link]

This should allow testing each Username and Password on that Linked Test Page, I have not tried this yet, have not had a need to, but it should be workable as I believe Fitnesse will take the settings it comes to last.

There is probably more to get out of Fitnesse that I haven't discovered yet, I can't wait to see what I find next.

Over the years I have collected quite a few books, not always on testing as I used to be able to get enough online and through discussions with people to keep up.  Occasionally I still bought the book that to me, was useful, or was a necessary part of the books I wanted to always take with me from Job to Job.  So here is the list of what I have (your mileage may vary):

Testing Books (some of these come into the office with me, especially Lessons Learned In Software Testing, which I have pulled out from time to time to get a point across to Dev Managers), this is not in any special order:

• Quality Software Management - Volume 1: Systems Thinking, Weinberg
• The Complete Guide To Software Testing, Hetzel
• Effective Software Testing, Dustin
• How To Break Software, Whittaker
• Lessons Learned In Software Testing, Kaner & Bach & Petticord
• Managing The Testing Process, Black
• Customer Orientated Software Quality Assurance, Ginac
• Software Testing In The Real World, Kit
• Change-Based Test Management, Sistowicz & Arell
• How We Test Software At Microsoft, Page, Johnston, Rollison
• Writing Solid Code, Maguire
• Debugging The Development Process, Maguire
• How To Break Software Security, Whittaker, Thompson
• Implementing Automated Software Testing, Dustin, Garrett, Gauf
  

Technology Books (I use Perl ALOT as you can see), since I am mostly self taught these are all the things I have needed for my job in past...most of this is O'Reilly, a geeks best friend.

• Learning Perl On Win32 Systems, Programming Perl, Advanced Perl Programming, Perl Best Practices, Perl CD Bookshelf, Mastering Regular Expressions, Perl Cookbook
• Debugging Perl
• Learning Bash Shell
• Sed & Awk
• JavaScript
• CGI Programming
• SQL In A Nutshell
  
• Unix Shell Programming
• Unix In A Nutshell
• Python Visual Quickstart Guide
• Python Programming For Beginners
• Python Essential Reference
• Unit Test Frameworks
• Teach Yourself Java
• Java Development With Ant
• Open Source Programming with CVS
• Unit Test Framework, O'Reilly
• Pragmatic Unit Testing, Hunt, Thomas
  
• Practical Performance Analyst, Gunther
• Aptana RadRails, Ramirez
• Test Driven .NET Development with Fitnesse, Adzic
• Fit For Developing Software, Mugridge, Cunningham
• Head First C#
• Windows PowerShell In Action
  

Fitnesse is the newest automation tool I have introduced here, we also use PowerShell to drive a lot of the builds, because there is some integration between the two (think Smoke Test) I worked on getting the TestRunner working with PowerShell.  I couldn't get the .Net version to run properly, it kept dying when it would render the page, but the Java runner worked great once I was able to work through the java command line with the BuildMaster.  If not for his help I wouldn't have been able to finish.

This script launches the Java TestRunner, points it to the Fitnesse server, and runs the three tests (one Test and two Suites) I currently have set up, then scans through the XML results to display some results in the command line window.

[string]$fitniumLocale = "C: itnium"

[string]$fitnesseJar = "fitnesse.jar"

[string]$fitnesseClass = "fitnesse.runner.TestRunner"

[string]$serverName = "my_server"

[string]$portName = "8888"

$testArray = ( "Test1" , "Test2", "Test3" )

function main()

    Write-Host "Running Fitnesse Tester.`n"

    cd $fitniumLocale

    # Run through each test in the bundle

    $Java = "java.exe"

    foreach ($test in $testArray)

        # Make a date string to get the right file date/format

        $dateStr = [string] (get-date)

        $Month,$Day,$Year,$Hour,$Minute,$Second = $dateStr.split(" :/")

        $testResults = "$test" + "$Month" + "$Day" + "$Year" + "$Hour" + "$Minute" + ".xml"

        [string]$opts = "-xml $testResults"

        # Start the test

        Write-Host "`nRunning $test`n"

        # Get the output from Java so we can scrape it for the test passes and fails

        & $Java "-cp" "$fitnesseJar" "$fitnesseClass" "-xml" "$testResults" "$serverName" "$portName" "$test"

        Write-Host "Completed $test`n"

        Write-Host "Test Results"

        # Scan through the output file, we want that for archiving

        [System.Xml.XmlDocument] $xd = new-object System.Xml.XmlDocument

        $file = resolve-path($testResults)

        $xd.load($file)

        $nodelist = $xd.selectnodes("/testResults/finalCounts") # XPath is case sensitive

        foreach ($testCaseNode in $nodelist)

        {

            $right = $testCaseNode.selectSingleNode("right").get_InnerXml()

            $wrong = $testCaseNode.selectSingleNode("wrong").get_InnerXml()

            $ignores = $testCaseNode.selectSingleNode("ignores").get_innerXml()

            $exception = $testCaseNode.selectSingleNode("exceptions").get_InnerXml()

        # Send a summary of the results to the console

        Write-Host "Passed: $right" -ForegroundColor Green

        Write-Host "Failed: $wrong" -ForegroundColor Red

        Write-Host "Ignored: $ignores"

        Write-Host "Exceptions: $exception" -ForegroundColor Yellow

Yes, I doubt.  I doubt a lot, especially on projects that tell me that things are working good and we're on target, I sit back and think - am I missing something?  I think its partly endemic to people who work in Test, because let's face it we are in the position of being the person next in line from when something is done and there needs to be a step to make sure that the something is done right.  If you've been in this position this may be familiar to you:

Developer: Hey, the code is ready I'm going to hand it over to you later, I'm done with it and it looks good!

Tester: Thanks for letting me know, I'll get back to you once I check it out.

Then you do, and your getting back to the Developer involves defect reports.  Was it good?  Probably, it met what the Developer was trying to do with it, there were Unit Tests which ran and passed, or what was the minimun to pass did, so it was handed to Test.  Now, Test has its own tests to run, its plan and its criteria to be able to say that things are stable enough to put it into Production and let Customers use it.  Testers usually have Tests that Developers never run, or want to get into, so they are looking at things from a higher level, sort of like the Ranger in the Watchtower, we have our scope up and we are looking all around.  Meanwhile, the Developer was just doing something over in one place and getting that to look nice, while nearby another Developer was doing the same thing but somehow they will collide.  Testers are there to raise the warning, hey there is some cross-purposes here or to see that there may be, if its been communicated that Developers are working together on something and maybe no one involved realizes there is some dependency.

So where does the doubting come in?  Remember those Customers?  Well, if a Customer sees something wrong and calls up with a problem, or is angry that there is a problem, many companies will look to find out why that problem got out there.  Someone has to take responsibility for it, and because we have that person in Test between the Developer and Production its that person's fault that anything got past, not that the Developer put it there - by mistake or accident - but that the Customer saw it.  To avoid that I became a doubter.

Now, being a doubter can be a strength, it makes you question everything and you look for issues that may need resolution, but to be KNOWN as a doubter is pretty bad.  Then you get a reputation, and sometimes people won't want to work with you because they know you'll come back with issues, complaints or will be negative right away.  What is needed is something I like to call Constructive Doubt.

Constructive Doubt is what I do when I get code and need to look at it, I won't immediately snipe back and say "heh, more bugs for me to find?" when a Developer comes and says something is ready.  Diplomacy is part of Constructive Doubt, because you want to build with it, anyone can tear down something, but it takes real skill to build something.  Use that doubt to figure out some new way to test, use a new tool, a framework or something that will allow you to doubt in an automated fashion.  Sounds like I am talking about Testing, but in reality part of a Test is a doubt, you doubt something is going to follow the requirement, sure you could call it a Verification or a Validation but in the middle of it all is a seed of doubt; you want to KNOW that something is doing the right thing in the right way.  Sometimes when I have tested something enough I know it inside and out I sit back and think, how else could this fail?  How else could I figure out that something failed?  Where else could I doubt the process is working?  Then you look, you learn and sometimes you find something new.

I'm in QA, I doubt and I build up on that.

Yesterday I went through the process of upgrading our soon to be used Subversion server, we are moving from VSS to SVN and the initial install of Subversion was done by our BuildMaster.  I got asked to do the upgrade and bring things up to current versions and document the steps, so for posterity here they are.

Subversion installation used 1.6:

http://subversion.tigris.org/svn_1.6_releasenotes.html

From Compatibility Concerns

• Older clients and servers interoperate transparently with 1.6 servers and clients.  However, some of the new 1.6 features may not be available unless both client and server are the latest version.  There are also cases where a new feature will work but will run less efficiently if the client is new and the server is old.
• There is no need to dump and reload your repositories.  Subversion 1.6 can read repositories created by earlier versions.  To upgrade an existing installation, just install the newest libraries and binaries on top of the older ones.
• Subversion 1.6 maintains API/ABI compatibility with earlier releases, by only adding new functions, never removing old ones. A program written to the 1.0, 1.1, 1.2, 1.3, 1.4 or 1.5 API can both compile and run using 1.6 libraries. However, a program written for 1.6 cannot necessarily compile or run against older libraries.

Subversion upgrade process

o   Turn off the Apache Service

o   Prepare Subversion for upgrade (back-ups)

1.       Zip up directories to keep a rollback in place

1.       Back up the configuration files under Apache (can be done within the Zip file)

1.       svnaccessfile

2.       conf/httpd.conf

3.       other stuff!

2.       Move ZIP files to a new location as storage

o   Upgrade Subversion

1.       Unzip the upgrade

2.       Move the binaries into place

1.       Copy /bin directory to install location

2.       Copy D:Program FilesSubversioninmod_dav_svn.so and mod_authz_svn.so to the Apache modules directory

3.       Copy the /iconv directory to install location (this helps to update the version numbers for Subversion within the Apache footer - at least I think it did, my cache was acting up in FireFox)

o   Upgrade Tortoise

1.       Install Updated software package

o   Reboot

o   Create a new Repository:

1.       Svnadmin create d:SourceControlSVNMAIN

o   Test/Verify Upgrade

1.       Connect using the Tortoise Repo-Browser from a remote machine

1.       Connection should allow viewing of repository

Backup Process

Using FSFS as the database backend.

Criteria for the backup:

The process should run [nightly?? (this can be adjusted depending on how long it takes)] using Windows Scheduled Tasks.

Use the following command from a Windows Scheduled Task:

svnadmin hotcopy (repository path) (backup path)

The backup process will copy server configuration and hook scripts.

Keep the backups on an area that is backed up (get location from IT)

Is it possible to "share files" in SVN?

If you use the svn:externals –r option you can, but it seems SVN can only do directories not files

                http://blog.kowalczyk.info/article/Short-tutorial-on-svn-propset-for-svnexternals-p.html

Security/branches:

·         Authentication via AD

·         Groups and user access defined in svnaccessfile

·         We want to have two groups with users defined to each

·         By default all Company group users will have all access; Consultant group users have access to only a branch (and possibly build stuff)

·         Out of the gate just the trunk, all users edit there.  More branches down the road.

Sites with additional information:

http://www.sublimesvn.com/blog/2009/02/configuring-subversion-with-active-directory-authentication/

http://concise-software.blogspot.com/2009/02/instant-windows-svn-server-with-ssl-and.html

Because I am working with Fitnesse and Selenium, I wanted a way to be able to remotely access tests on a central server, right now I am setting up the services on my own machine.  These are the instructions I used to get things in place.

Set-up Fitnesse as a Service:

• Install the Windows Resource Kit if its not already in place
• Run the following command:

          instsrv Fitnesse "C:\Program Files\Windows Resource Kits\Tools\srvany.exe"

• If the resource kit is not in that location update the path, but you MUST use the full path to srvany.exe or it will not work
• Modify the Registry key in HKLM\SYSTEM\CurrentControlSet\Services\Fitnesse
• Change the Image Path to point to Srvany.exe's full path (if not already done so)
• Add a new key - Parameters    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Fitnesse\Parameters
• and three string values,
        AppDirectory - AppDirectory is the root directory of our service where the Fitnesse bat file resides
        Application - is the java.exe executable needed to run our service
        AppParameters - is the parameters used by the java.exe to start our Fitnesse service.
• In the Services panel verify that the Account being used to start the service has the right permissions
• Start the service
• To access the Fitnesse Web UI use the URL - http://local_ip_address:port/

Items to note:

• If the Service starts, but does not open the port make sure Windows Firewall is set to allow access to the port.
• If the Port is open but the open port is not appearing in a netstat -a then use the command line arguments to verify that the command line options work.

Example settings for Fitnesse Service:

AppDirectory - c:\fitnium
Application - C:\Program Files\Java\jre6\bin\java.exe
AppParameters - -cp fitnesse.jar;selenium-java-client-driver fitnesseMain.FitNesseMain -p 8888

Selenium Server:

• In order to assure that Fitnesse can run its tests Selenium needs to be set up the same way
• Modify the Registry key in HKLM\SYSTEM\CurrentControlSet\Services\Selenium
• Change the Image Path to point to Srvany.exe's full path (if not already done so)
• Add a new key - Parameters (HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Service\Selenium\Parameters)
• and three string values,
        AppDirectory - AppDirectory is the root directory of our service where the Selenium bat file resides
        Application - is the java.exe executable needed to run our service
        AppParameters - is the parameters used by the java.exe to start our Selenium service.
• In the Services panel verify that the Account being used to start the service has the right permissions
• Start the service 

Example settings for Selenium Service:
AppDirectory - c:\fitnium
Application - C:\Program Files\Java\jre6\bin\java.exe
AppParameters - -jar .\selenium-server-1.0\selenium-server.jar  

I had decided to go with Fitnesse and Selenium as part of our Acceptance Test Framework, it was a challenge to say the least, and with the options that were out there it was bit of work and testing to find the best combination to get things to work in our environment.  Basically, I am testing an update to a HealthCare Process Improvement site, nothing like being a QA person on a project like that I must say!  So my tests boil down to being able to login and do certain functions, as well as just roam the site and check for pages that are not working, in the midst of this I wanted some structure so that I could do things like have login and logout before the tests so I just set up a session and do everything.  Sadly, it did not work out that way.

WebTest Fixture

I went through Gojko Adzic's Web Test Fixture for Selenium and Fitnesse, this I struggled with for a little while but I liked the ability to use .Net or Java and the language to call Selenium was pretty simple.  Though you have to remember to use those !'s on certain commands, something I found through trial and error with Start Browser, and it worked out pretty well.  Getting Suites to work is something that I started with here, and had a lot of problems with, I kept getting fixture errors in trying to set up my suite that really became a stumbling block, seems as if I am destined never to get the fixture library to carry through the suite and putting it on every page became a problem after awhile.  Biggest problem though was the version that is complete on the site is an older version of Selenium that does not work with FireFox 3, so if you have that on your machine you can either do your own buildup, or try to update the Selenium libraries in the installation.  You could probably compile your own code and get things working that way as well, but its not the method I wanted to go with so I eventually gave up on it.

Fitnium

Something I came across and this worked out pretty well, moreso once I resolved my whole Login and Logout issue, and the fixture on every page problem that seems to plague me is something that carried here as well.  Still, I was able to work out my issues before getting responses to my posts on the Fitnesse Yahoo Group, not sure if that is good or not but I will say its a positive that I could do it.  Again a big win for me was being able to work out the fixture and suite problem here, although some of the Fitnium commands could be a little easier to remember but that I can work through.

Fitnesse

I do like the Acceptance Tests you can make, and the Suites, they will definitely help with some of the work we need to do for our product, I think my biggest issue is like what most people say - you need to write the tests.  Structuring them in a logical way, or what seemed logical tome, did not work as well and gave me trouble until I used SetUp and TearDown for my tests and put the Login and Logout steps in those pages rather than the SuiteSetUp and SuiteTearDown.  I don't know why that happens, but it'd be interested to know.  Writing the tests can get time consuming, and I still want an easy way to get Selenium scripts to output into Wiki pages, Cory Foy's script did not work for me and I have no idea why (I don't do JavaScript much).  The documentation can be confusing and incomplete, I mean where is that !r command defined?  I even bought Gojko Adzic's book to help me get going, and I was able to get some nice stuff out of it, but being a doer rather than a reader I didn't get as much out of it as I thought I would, but I'm keeping it handy, who knows, I may get more into this.

Overall, it worked out well, but I would say if you are going to set these up:

• Plan out your tests, and how you want them to work
• Consider a few alternatives, and your Users and see what works
• Read up on the tutorials and see if you can go through the code there, if not getting knee deep in this will only be frustrating in the future
• Be sure these are the tools for you, Fitnesse can be daunting to a techie, and it will be moreso to a business user

Other than that, I'm glad I went through this and now have Selenium and Fitnesse working in my environment.