Software Test Reports for Startups

My test reports have deviated a lot since the early days of testing.

Nowadays, first and foremost I provide an opinion of the software. I make sure I highlight both positives and negatives that I see in the product.

Naturally, I provide a list of bugs I find, but I also provide a list of suggestions on the software, for example new features or ways to improve the Usability. I find when testing, new ideas often come to me about features. I’m happy to provide them with a list. They can take it or leave it.

Finally, I list some recommendations on testing for the next release.

Why all the effort? I do this because my clients love it!  I provide more than just a list of bugs. I see it as adding more value.

Here’s the template pdf version:: Software Test Report Template

I wonder though, is this still testing, or  have I morphed into a new area of work?

In defense of the humble spreadsheet

I wrote a post a while ago on scoping out your tests. In it I described how I use Excel spreadsheets to scope out and estimate what I need to test.
Judging by the comments I got back then, Excel spreadsheets in software testing have the ability to divide the software testing community.

I was once in the ‘down with EXCEL in testing’ tribe. Now I’m in the  ‘Excel is cool” camp.

I used to hate spreadsheets in large organisations. They created havoc on my test methodology. Every every project has their own version of the template that is an improvement on the orginal design. Not only that, but every tester within the project had their own version on their PC. There was no way of really knowing how many bugs had been found.

I couldn’t understand why testers didn’t just use test management tools like Test Director. Why didn’t testers see the benefits these tools could bring?

That was until I started freelancing with startups. Oh my, did things change.

Suddenly I was in an environment where there were no resources to invest in expensive test tools. “No problemo” I thought, “I am an independent free thinking tester,  I will investigate some opensource tools and put forward those as suggestions”. Ha!! Even when I did find a tool that was liked, there was little time to invest in installing and configuring a tool. Furthermore, I knew the minute I left, nobody was going to maintain the scripts etc, so really was there any point in wasting my time installing such applications?

So I turned back to the humble spreadsheet and started estimating,scoping and tracking my results in it. I would send updates to my clients regularily, and sometimes I would even add some pretty metrics to make people feel good about my testing. Now that you can upload spreadsheets online, much of the issues with version control disappear. It works for me.

The great thing about spreadsheets is not its flexibility, its power and its ease of use. The great thing about spreadsheets is this: Everybody knows EXCEL and has the it on their PC. Its available to everyone, needs no training, maintenance or configuration. People understand and are familiar with spreadsheets.

That means the time I would normally need to spend installing a tool, configuring , maintaining and training people on a tool, now can be used in testing. And believe me when your given 3 days to test and report on an application, that additional time really counts!

So Viva the humble spreadsheet, and long may your silvery squares compute.

Is document a dirty word in Exploratory Testing?

I went on James Bach’s Rapid Software Testing (RST) course because some of the concepts and ideas that I had read about exploratory testing and RST appealed to me.   I liked the idea that central to testing is a critical and context-driven approach and I also wanted to put intelligence back into testing.

I was curious though, as on some blogs I read it appeared that exploratory testing and traditional testing were mutually exclusive. You are either a champion of traditional testing techniques and provided multiple test documents or you’re in the maverick camp which threw out all documentation and just ‘tested’.

I was relieved to learn that for rapid software testing, this is just not true. I was relieved because I LIKE DOCUMENTS.  I like them because in certain circumstances I find them helpful.

Because I have to confess, sometimes I forget to test parts of an application. Even worse, sometimes I don’t want to test the hard areas.  

A document gets me to test all areas and to test the parts that I really don’t want to test. It helps me remember the results of what I’ve tested because sometimes I need to know that information.

What James Bach reminded me, was that the ultimate goal of testing is very simple. It’s to test an application with intelligence and thoughtfulness. The goal is not to create endless documents on testing.  Instead, documents can sometimes be handy tools to assist you in testing.

I don’t think I will ever totally give up on my documents. They are my friends. However, I will make sure that in future, these friends can stand the test of relevance, accuracy and brevity. 

Getting ROI from your freelance software tester

Development houses have a right to expect a lot from a freelance tester.

Firstly, without the endless budget of some larger companies, they can ill afford time and money caused by improper scoping and testing.

Secondly, they have recognised the advantage of having an independent review of the product and that in its own right deserves to be appreciated.

In order to get the best return on investment from their tester, communication of priorities and expectations must be passed onto the tester. With knowledge, testers can focus their effort on what developer priorities instead of what they suspect is their priorities.

I’ve created a short questionnaire to clarify to developers what a tester needs. It contains some of these areas:

1) What do you as a developer value most? Consistency, Quality, Breadth of testing,
2) What specifically do you want tested in web testing?
3) What technology are you using?
4) What testing has already been performed?
5) Do you have any specs of any sort?
6) Is this new software, or updated software
7) What sort of feedback do you want? Defect reports, results?

Note: These questions have been created with web testing in mind, but can be changed for any type of testing

Here’s an example of what I use

Customer survey (pdf reader required)

Having this information upfront helps everyone because:
1) There is an agreed understanding of the scope of testing
2) Quotes can be validated through the questionnaire
3) More upfront information maximises your return on investment allowing focus on customer driven testing
4) It provides an insight into the software testing process

The above information is used to create tests in a spreadsheet which I use to track defects and results.

See example:Software Testing Template(pdf reader required)

With this document you have the benefit of evidence of testing which can be useful for contractual purposes. It also assists in future upgrades, streamlining the next round of testing.

Like what your read? Please feel free to use the ideas I have here. Do us a favour though, leave a comment, or digg the post. Thanks !!

Software Test Templates – scoping out your tests

I wrote a blog recently “the value of a software testing process” in which I questioned the traditional benefits of software test scripts based on the re-use and repeatability theory.

My approach to any rapid change software development is to make a some basic assumptions

1) The code is going to change quickly
2) The person who coded won’t necessarily be there for the next release
3) The person who tested the last time, won’t necessarily be there for the next release
4) Test documentation helps to provide structure and is excellent as a guide to ensure adequate coverage
5) Testers are able to think outside the square and aslo be able to fill in the gaps in a test

I use one spreadsheet (if possible) to track the following information;

a) test script number
b) test link (if available which is a reference to requirements etc)
c) test purpose – a clear concise description of what I am planning to test
d) test results – Pass or Fail
e) defect number – I assign a number of use the defect tracking system
f) test estimate – the time I anticipate it will take to test this functionality

I use the document to first scope out what I want to test, a quick and dirty overview of what I am planning. I also use it to estimate how long testing is going to take. This is helpful if I am ever asked to substantiate my estimates.

Once all stakeholders are in agreement onto the scope, I create a concise and descriptive purpose of each test. This is in essence my test script. I use the same spreadsheet to enter results, defect numbers and to calculate simple metrics such as the pass rate.

I have uploaded an example below;

Software Testing Template

I hope you you find this post useful. Please feel free to use the ideas I have here. Do us a favour though, leave a comment, or digg the post. Thanks !!