UI Testing SharePoint

As part of an on-going series of ‘Testing SharePoint‘ we intend to share some of what we have learnt when trying to automate the testing of a SharePoint add-on called DocRead. These posts will specifically cover functional and integration testing – not unit testing (which is worth a whole conversation on it’s own).

To get started, I want to tell a story about how I have come to love Selenium, but first a bit of history about who I am. I have used SharePoint seriously since beta 1 of SharePoint 2007 and worked as a developer or architect for organisations such as Collaboris, NHS, Tarmac, BT, Lloyds of London, Lloyds Tsb, Capita, O2 on both versions. I have tried – like lots of other devs – to make unit tests useful and still get the product out on time. I have done lots of performance testing using visual studio web performance tests in my time and actually quite enjoyed it.

Having said that, I have never done functional testing apart from to help out here and there. However, as DocRead is really starting to become the success we hoped for I have found myself looking for ways to get new versions out of the door much faster, cheaper and more efficiently! One area that’s been burning our development teams time is testing. It’s all very organized but very manual. We have really good test coverage, but to run though one set of tests for one version takes 4-5 days. If you think that DocRead ships on SharePoint 2007 (32 and 64) and SharePoint 2010 that’s 15 man days to get a version out! This also assumes that we only have to go through the test cycle once – which never happens. There are always a few iterations of re-testing for either all or part of your applicaiton.

Note. for those of you that don’t know – DocRead is an add-on for SharePoint that allows you to require or recommend that groups of users read a document by a deadline. It has it’s own databases, reporting application, web parts, application pages and features to activate at all levels in SharePoint. It solves a simple problem, but actually requires a very robust solution and has tons of different test scenarios to cater for.

When trying to help the testers come up with an automated solution, my first reaction was to dive in and use Visual Studio Web Tests, but I soon realized that they weren’t going to fulfill our needs. DocRead uses tons of client side code and Ajax calls to make the experience as responsive and smooth as possible. We needed an approach beyond ‘record and play’ that worked at the browser level, not at the http level like web tests do. Enter Selenium.

Firstly, I can honestly say that I have never touched Selenium until 3 weeks ago. I knew a couple of testers that had and only 1 that had used it seriously with SharePoint. In fact, if you Google ‘Selenium and SharePoint’ you very quickly run out of decent results. Why is this ? Let’s think about it. For a tester to automate SharePoint testing they need to know…

1. SharePoint – this in itself is huge!
2. How to code – testing has to be automated in some language.
3. How to test – testing is a real skill, knowing how to structure your tests is very important.
4. A testing Framework/Tool – Selenium, Visual Studio Test, etc.

That’s one hefty skillset – finding testers with these skills is very tough!  In my opinion, testers also get a rough deal. Developers only consider them when they have time, usually as the first bugs come in. Developers don’t write code that easy to test, (e.g. nice use of CSS, ID’s). And all too often with SharePoint, releases are late due to deployment headaches.

The following sections offer some guidance and help explain the problems we had to tackle along the way – (as I write them I will link them!)

• What is Selenium and how can it help me test SharePoint?
• Where to host your SharePoint Test Environment?
• Automating the deployment of SharePoint and setting up your environment.
• Setting up your SharePoint test environment.
• Making SharePoint test cases more readable and less fragile.
• A common testing framework with Selenium SharePoint (ShareTest).
• Testing both SharePoint 2007 and SharePoint 2010 with Selenium.