October 8, 2009 - Checking a List of Sites Using Xenu Link Sleuth

A new member at SQAForums asked:

"I test an online booking website which links to over 60000 client websites.

 

I am looking for a tool that will allow me to check their URL's to ensure that they are still valid.

 

I am trying to find a tool that will allow me to create a script that will reference the URL from a spreadsheet and send a query to that URL and get the HTTP Header response (to see if it returns a 404).

 

I have been looking for tools and the ones that I have found seem to check the link then expands out to check links on that page. I don't need (or want) that to happen. I just need a tool that checks the header response for the URL provided then moves on to the next one in the list."

I use Xenu Link Sleuth for this sort of thing. It's free, but not open source. It's also easy to use, and very fast.

Xenu can be set to check a Maximum Level of 0 - indicating that it should not spider the site, but just check the top-level URLs.

Here's how to do that:

1. Create a text file containing all the URLs you wish to check, with each on a separate line
2. In Xenu, select Option, Preferences... and set Maximum Level = 0 in the Options dialog
3. Set any other Options you choose
4. In Xenu, select File, Check URL List (Test)...
5. In the Open URL List dialog, open the text file containing the URLs that you created in Step 1
6. Your test runs

You can find Xenu Link Sleuth at:
http://home.snafu.de/tilman/xenulink.html

See all the tools in my Tester's Tool Box at:
http://www.sqablogs.com/jstrazzere/55/Tester%26%2339%3Bs+Tool+Box.html