I'm running Fitnesse with SharePoint and in trying to automate my tests I ended up wrapping everything up in PowerShell, this is my first full script that handles all the tests I wanted to do.

First set up all the variables needed.

[string]$fitnesseJar = "fitnesse.jar"
[string]$fitnesseClass = "fitnesse.runner.TestRunner"
# Need to get the local machine here, it will be where Fitnesse runs
[string]$serverName = $env:COMPUTERNAME
[string]$portName = "8080"
# These are needed for the CheckFitnessProcess function, placing them here
$fitRunning = $false
# Since we need to manipulate this variable in a few different functions, its Global
[int]$Global:startTries = 0
# Set Java executbale to use within PowerShell
$Java = "java.exe"
# Use a Test Array to be able to bundle all the tests
$smokeArray = ( "FitNesseTestPageName1" )
$qaArray = ( "FitNesseTestPageName2" )

I use a function to check if Fitnesse is running, I wanted to only run it when I needed to and since this script can be automated I cannot be sure that Fitnesse will always be running

# CheckFitnesseProcess determines if Fitnesse is running; in which case
# it jumps down to running the tests, as long as Fitness is running the Check does
# not need to do anything else.
# If Fitnesse is not running then the script attempts to start it, does a
# check to make sure that Fitnesse started then runs the test.  If this script
# starts Fitnesse then it will attempt to clean up after itself and stop
# Fitnesse by doing a check on $startTries, if this is > 0 then this script
# started Fitnesse and can shut it down, otherwise it leaves the existing
# process running
# Adapted from: http://odetocode.com/Blogs/scott/archive/2006/07/17/5330.aspx
function CheckFitnesseProcess {
  param ([string]$AppPoolName="")
  $ProcessId = $null
  if ($AppPoolName.ToUpper() -ne $NOT_PASSED) {
    [string]$filter = $null
    if ((Get-Process java -ErrorAction SilentlyContinue) -ne $null) {
  Get-Process java | ForEach {
   # use WMI API to get process details
   $filter = "Handle='" + $_.Id + "'"
   $wmip = Get-WmiObject Win32_Process -filter $filter
   # CommandLine property has AppPool name; grab name using contains
   if($wmip.CommandLine.contains("fitnesse")) {
    # Uncomment the next line for debugging
    # Write-Host "Found this java process: $wmip.CommandLine `n"
    $fitRunning = $true
    # Saving the Process ID to stop the process later,
    # using a Global as I could not get this work if I declared
    # the variable at the beginning.
    $Global:fitProcess = $_.Id
   }
   # The next statement was put in to check for the processes, there was a problem
   # when updating Java that the process find command did not work, if this happens
   # again just uncomment the Write-Host line to find out what is going on, otherwise
   # this just gets bypassed but caught later on.
   if ($fitRunning -eq $false)
   {
    # Uncomment the next line for debugging
    # Write-Host "Did not find a Java/Fitnesse Process.  Found $wmip.CommandLine `n"
   }
  }
 }
 if (($fitRunning -eq $false) -and ($Global:startTries -lt 3)) {
  # Don't need to note that we have had 0 attempts
  if ($Global:startTries -gt 0) {
   # Need to give Fitnesse some attempts at starting, 3 is a good max
   Write-Host "Fitnesse is not running, have tried to start $Global:startTries time(s),`n"
  }
  $Global:startTries++
  startFitnesse
 }
    if (($fitRunning -eq $false) -and ($Global:startTries -eq 3)) {
  # At some point we need to give up starting it
        Write-Host "Fitnesse is not running, tried to start $Global:startTries time(s).`n"
  Write-Host "Strike Out!" -ForegroundColor Red
        Write-Host "Looks like Fitnesse cannot be started or found,`n"
  Write-Host "check the server and try again.`n"
  # Go back to where we started
  Pop-Location
        exit;
    }
 if ($fitRunning -eq $true) {
  # Success!!
  Write-Host "Fitnesse is good to go.`n" -ForegroundColor Green
 }
  }
}

Then I needed something to start Fitnesse, this uses the Fitnesse Batch script that comes with many of the installs

function startFitnesse()
{
 Write-Host "Attempting to start up Fitnesse...`n"
 # Starts the Fitnesse process using the batch configuration file
 # Not sure we need the runAs Admin but that can be adjusted
 Start-Process -FilePath fitnesse.bat
 # Giving it time to wake up
 Start-Sleep -Seconds 2
 # Need to check that the process actually started, do the check again
 CheckFitnesseProcess
}

Every script needs a help option

function myHelp()
{
 Write-Host "The Fitnesse Tester understands the following command line arguments:`n"
 Write-Host "    qa    - will run the QA Test Suites, a long set of tests`n"
 Write-Host "    smoke - which will run the Build Smoke Tests, a short set`n"
 Write-Host "`n Running this script with no arguments will display this message`n"
}

Then there is the main and rest of the script that runs everything

function main()
{
 # Need to be in the Fitnesse directory to access TestRunner, and
 # the Fitnesse batch file in case Fitnesse needs to be started
 cd $fitnesseLocale
    #Check and see if Fitnesse is running
    CheckFitnesseProcess
 foreach ($test in $testArray)
 {
  # Start the test
  Write-Host "`nRunning $test`n"
  # Get the output from TestRunner so we can scrape it for test passes and fails
  $testResults = & $Java "-cp" "$fitnesseJar" "$fitnesseClass" "-v" "$serverName" "$portName" "$test"
  Write-Host "Completed $test`n"
  Write-Host "Test Results..."
  # Scan through the output, then display the results
  # Dump $testResults out to a File
  $testResults | Out-File "$test.txt"
  foreach ($line in $testResults)
  {
   if ($line -match "Assertions:") {
    # Write-Host "$line"
    # Do a regular expression to get the Assertion info
    # Example: Assertions: 8 right, 0 wrong, 0 ignored, 0 exceptions
    $line -match ":\s(?\d+)\sr.*(?\d+)\sw.*(?\d+)\si.*(?\d+)\se.*"
    # Send a summary of the results to the console
    Write-Host "Passed:     "$matches.right -ForegroundColor Green
    Write-Host "Failed:     "$matches.wrong -ForegroundColor Red
    Write-Host "Ignored:    "$matches.ignores
    Write-Host "Exceptions: "$matches.exception -ForegroundColor Yellow
   }
  }
 }
 # Go back to where we started
 Pop-Location
 # Stop the Fitnesse process so we don't leave things running
 # But only if this script started it, which means $startTries is > 0
 if ($Global:startTries -ge 1){
  Write-Host "`nSince this script started Fitnesse, it'll be shut down now.`n"
  Stop-Process -Id $fitProcess
 }
}

# Check command line values to see what test is running
if ($args -eq "smoke")
{
 $testArray = $smokeArray
 Write-Host "Running Smoke Test.`n"
}
elseif ($args -eq "qa")
{
 $testArray = $qaArray
 Write-Host "Running QA Test Suite.`n"
}
elseif ($args -eq "check")
{
 $testArray = $checkArray
 Write-Host "Running Check Suite.`n"
}
else
{
 myHelp
 exit;
}

# Run main
main

I've moved onto REST but in case anyone else is trying to work with the TestRunner and PowerShell you have something you can start with, and not have to work it all from the beginning like I did.

I've been learning a lot about SharePoint over the past 9 months, the project I am working on is to bring a non-profit health community onto SharePoint and increase the offerings the non-profit has and to expand the social networking.  My work is revolving around testing the conversion of the site, making sure that what we end up with is going to be equivalent with what is currently in place.  It's interesting work and I've had some interesting tasks over the past few months.  Some of the more difficult work has been dealing with SharePoint and its eccentricities in order to assure that what we end up with is what was asked for.

Web Parts are like little applications that SharePoint uses to do all kinds of work, this could be anything from displaying a document library to a search result.  While they are like little applications Web Parts are quite different and aren't very testable on their own, I haven't seen much on them regarding Unit Testing yet and with their reliance on SharePoint data its probably not that easy.  Mostly the plan is to test the Web Parts in an exploratory fashion, when they are ready and added to pages they will be used and abused to put them through their paces.  Not the way I'd like to do it, but I am also pulling some information from the Microsoft page on how to test Web Parts.

Automation with SharePoint is kind of tough, not that its difficult to work with, but the page layouts and Web Parts don't always make things run smooth.  Web Parts can update like Ajax driven pages will, some of the ID's given by SharePoint for page elements can also be cryptic or dynamic.  It's not impossible, I have done tests with Selenium, Fitnium and Fitnesse after some time to work out the issues in calling page elements.  It just seems to take more time than it should.

The Social Networking aspects make things a little hard to automate everything, it would be nice to free myself up for other work but that's more a dream than a reality.  Testing Discussion Groups, Comments and Ratings means eyes need to be on the page as things are done in order to verify that SharePoint responds properly.

When trying to find errors there are two important places to look:

1. ULS Logs - Get the Microsoft ULS Log Viewer, if there is something happening real time, this is the place to see it happen.  Access to the SharePoint server under test is required to get this to work though.
2. Event Log - the Event Log is where alot of the captured errors go, if the Developers have not set this up in the code they should otherwise you lose and important resource.

I'll be putting in more links to some of the material I have captured over the past few months.

I got an IM from a developer I used to work with and he had that good old familiar question "do you have a copy of that graph that shows the cost of fixing a bug?"

Well I didn't at the time, mostly I dealt with this with the book I keep on Quality Assurance, it has the graph and a little blub about it.  Since he needed one online, and pretty much everything is on the internet I did a check and lo and behold, I found a developer centric version.

Rate of Diminishing Returns Of Fixing Software Bugs

He needed an online version because he had a VP who wouldn't drink the water on catching defects early, I guess there are still plenty of them out there.  Either way, I found the link and he went away happy, another QA-centric developer satisfied.

It feels good to help on a Monday.