Tag Archives: startups

Pre Software Testing Checklist for Startups

Posted by on September 7, 2009 No comments

Most Startups will not use large amounts of documentation unless it adds value. In an environment where the one thing you can rely on is change, sometimes it doesn’t pay to put too much detail down in formal software test plans. That’s why I am in such a fan of checklists. A software test checklist is like a cheatsheet for testers. It’s a list of reminders to make sure you’ve covered all the basics. Nothing fancy, I prefer a one sheet, but I’ve seen checklists two or three pages long.

Lots of people in software testing loves checklists. It’s our way of ensuring we don’t forget stuff, well at least that’s why I use them. They work really well with Startups who either don’t have the time, need or inclination to load themselves down with lots of methodologies and process.

The first checklist that I am in favour of handing to my Startup clients is the Pre Assignment checklist. My Pre Assignment Software Testing Checklist provides Startups with a list of essential to do items. Most of my Startup work is short assignments between 3 ~ 5 days as time and budget are limited. So, if they can’t supply these things before I start, I politely suggest that they’re wasting my time and their money.

Proviso:
The assumption in this checklist is that I’m software testing on-site and the assignment is a short term one. Otherwise some of this stuff I would be responsible for. For long term assignments part of these may be written into the software testing assignment.

So here it is in its full glory with added explanations for clarity.

Pre Assignment Software Testing Checklist

Software Test Goals

  • What do you want to get out of the software test effort?

Confidence, Knowledge? Software that works better? The sheer nature of software testing means that a software tester can provide you lots of  interesting information apart from bugs found.

For example, if it’s knowledge,  A software tester can collect lots of data on performance and reliability which may be helpful in marketing your product.

SoftwareTest Reporting

  • Who is going to see this report?

A good software tester can write the same information differently depending on who is going to read the report. For example, you may want a report written in word format for business purposes. If its a developer, then perhaps they have a favourite defect tracking tool they want to use.

  • What sort of reporting do you want?

How often do you want a software tester to update you? Daily, Weekly or just at the end?

Test Strategy

  • Do you have a long term software testing strategy in mind?
    Perhaps you wish to automate your testing in the future, or you want to introduce a process, if you let the software tester know up front, they can perhaps include some structure for you to keep once the software testing is complete.

Software Test environment

  • Determine the Software Test Environment
    Has a decision been made on the system requirements? Software Testing ought to use this as the default test environment. Make some decisions on the following:

    • Operating System
    • Browser
    • Printer
    • Machine
    • Any other supporting software

Don’t forget to decide on a version number for the software

  • Assign Machine
    The developer’s machine won’t do, sorry. You might have a machine in mind, but have:

    • Started it up recently?
    • Made sure no-one else is using it?
    • Does it require updating?

My point is don’t assume that the box you have earmarked is available.

  • Setup Software Test Environment
    I know it works fine on your machine, but its worth ensuring it installs on the test machine.

    • Install all the support software the system needs
    • Install the correct version of the support software
    • DO NOT install the product to be tested if installation and configuration is to be tested
  • Test the Setup
    Start up the software that has been installed to make sure it works properly

Finally and maybe most importantly!

Software Test Support

Ensure:

  • support by development is available to support the tester
    Don’t give the developer a two week holiday for working so hard to complete the software on time. He still needs to support the tester when bugs are found
  • IT support is available if necessary
    If your luck enough to have an IT team, make them aware of the software testing work that is going on and ensure someone is there to help if necessary

This type of work is often overlooked in the eagerness to get the testing started. If you want you’re software testing to start on time and keeping within budget than completing this software testing checklist is essential.

Here’s a link to a scaled down version. Pre-Assignment Software Testing checklist. Feel free to download it, but please leave the reference to Testing Times in either the header or footer.

50 Beta Testers. Not enough, or better than most?

Posted by on September 4, 2009 2 comments

Two software testers talked about the Joe Stump story this week.  One was Elizabeth Hendrickson and the other UTest.  The two attitudes to the same story are very different.

Let me compare two of the views:

First Elizabeth Hendrickson
“Problem #2: Thinking that “50 Beta Testers” and “200 Unit Tests” Constitutes Exhaustive Testing

Having beta testers and unit tests is a good and groovy thing. But it’s not sufficient, as this story shows. What appears to be missing is any kind of rigorous end-to-end testing.”

……….

“So even the most cursory Exploratory Testing by someone with testing skill would have been likely to reveal the problem.”

Now UTest

“Not to lay blame at Stemp’s feet.  We, of all people, know that bugs happen.  Plus he mentions utilizing 50 beta testers and 200 unit tests, so they’re doing more testing than many.  But IF these showstopper bugs had been caught in the initial version or even in the 2nd version of Chess Wars, then Chess Wars wouldn’t have its nose pressed up against the window of the App Store, waiting for the powers-that-be to bless the new version.”

Whilst both blogs encourage more testing, I think I prefer Elizabeth’s attitude to the whole thing. It’s hard to know if 50 beta testers is sufficient or not without the context (last phrase included to avoid references to Unicorn Question in any comments..you know who you are…). Regardless, the attitude that  if you ‘do more testing than many’ somehow vindicates the fact that showstopper bugs were found is I think a misguided one.

As Elizabeth rightly pointed out in her post, good exploratory testing would have picked up this bug and any good tester knows to test for these things.

I hark back to my post on other approaches to testing for Startups. There are better ways to test! Lets not fall into the pitfall of accepting a lower level of testing as sufficient. Just because Startups use Beta Testing as the preferred modus operandi , doesn’t necessarily make it a good approach.

Black and White and Red all over….

Posted by on July 4, 2009 2 comments

There’s an add on telly about this guy who writes his book on a remington typewriter and then the post office sends it and suddenly its in the window of a bookshop. It think the ad is for the Irish Post Office.

Regardless, there is a moment when the author looks into the bookshop window and glows with satisfaction and pride on the achievement.

That’s a bit how a feel like today.

I just have had my first article published in T.E.S.T magazine and I’m on the front cover no less. You can read it here

It’s about one of my favourite topics, software testing and startups.

It’s these types of achievements along with getting work, that keep me working as a independent test consultant because, working on your own can be really tough! It’s a constant act of self promotion, bookeeping, customer liason and oh, occassionaly I get to do some software testing.

There was a post on the software testing club recently on ‘what are the things to consider when outsourcing?” I think the things to be considered are those outside of your area of expertise. Software Testing is only a small percentage of what you will do in order to be successful at freelancing or working as an independent.

Anyhow, back to the article.

As I was reading it, I felt a mixture of pride, satisfaction and surprise. I didn’t realise I could write so well….maybe it’s to do with the fact its a published article, or maybe its because the self satisfaction is putting a rosy tint on the whole thing. I don’t know….I don’t really care.

Now leave me alone whilst I bask in my halo of self content…..