Bizcamp Talk

I gave a talk at Bizcamp this year on how to get the most out of your software testing.

This was an interesting challenge as the I wasn’t speaking to software testers but developers. How could I help developers and small companies test better?

I decided to go down the path of identifying typical traps that testers and developers fall into, e.g testing to little, or testing too much and how to try and get your testing just right.

I used the analogy of Goldilocks to get my point across.

I got lots of questions which was very gratifying.

Walking the walk, but can you talk the talk?

In my experience, selling testing is a lot harder than testing. Despite being Irish, “the gift of the gab”  eludes me, often leaving me tongue tied and twisted when selling my wares.

Its one of the reasons I have got myself a business mentor.  His main focus has been for me to understand my ‘product’ and my customers.  He’s arguing at the moment, that the word ‘testing’ is not sufficient to draw in my customer base. The reason being, is my customers work in a different paradigm and use different language and so fully fail to comprehend the benefits that testing can provide.

We know as testers that testing can have massive benefits right across the company, and that the information and insight (thanks Joe) we provide is useful beyond the IT development lifecyle.  For example, we know that marketing can benefit from data resulting from performance testing, that legal teams can benefit from any compliance testing we perform.

There is a bit of a trend recently to describe our work in clear terms such as testing and bugs. Where before we perhaps worked in QA, we our now testers. We have ceased to use language such as gatekeepers and milestones. But I wonder in using such limiting language such as testing and bugs does it make it harder to sell testing to these stakeholders? I certaintly struggle in finding ways to bridge the language gap between testing and my customers.

Words such as Quality and Assurance have in a way become tainted and I try to avoid such terminology, but really where does this leave me?  I know for effective communication with my customers (who exist outside the IT world), I need to use their language and describe testing in their terms.

I will give you an example.

I attended a BizSpark networking event recently where a number of Venture Capitalists were present. One concept I’ve been toying with is to provide independent technical reviews on software products for investors and the like.  I know, it sounds a lot like testing, but if I used the word test  and bugs, VC’s eyes glaze over.

So I speak to them using words such as due diligence, audits and risk management.

Does this mean I’m not testing anymore?  Am I now selling insurance as opposed to  testing? Personally, I don’t see it that way, to me the core is all about testing. It’s just that the context has changed, and so the language has.

Perhaps I’m over complicating things here. I’m unsure.  I know that placing testing in their context it makes it easier for them to buy into. Is that so wrong?

I’d really like to hear people’s thoughts on this, it’s a topic that fascinates.

Changing the Goal Posts

I got a chance to watch some Gaelic Football recently as my two sons have started playing hurling and gaelic football.  Hurling is a fairly ferocious sport where everyone goes for the ball with hands and sticks. Gaelic Football gives the appearance of being a far kinder sport. However, the Australian version (aka Aussie Rules), is a much more physical and aggressive game.  One thing that differs between the Irish and the Australian version is the goal posts. In Aussie rules points are scored when you kick through one of four poles, Gaelic, you kick into or over a pole. Little differences but really important to know and get right.

hurling

Adam Goucher had a nice post this week on communication and letting your team know where the goalposts are. Well I have had a similar goalpost problem. Except in mine, the goalposts changed. How I scored the goal had changed, and no-one had told me.

I have a very nice client (he’s a client, so he has to be nice, well in this case he actually is very nice) but new to the software development, and a grappling with some of the basics do’s and don’t to mostly people in IT take for granted.  It’s the basic advice they require most. Like, letting people know when things change, so I can relate to Adams comments.

I spent a good amount of time before the project, understanding who the stakeholders were, and the mission of the testing effort. I explained to them the many different types of testing I could perform, such as Usability, Performance, functionality, Installation and Configuration. It was decided to narrow the scope to robustness, it was a prototype after and all and as long as the screens did not crash, that would be satisfactory.

The day before delivery to their customer, there was a review. The stakeholders were dismayed at the state of the screens and the way the functionality worked. But I spluttered, “you told me you didn’t want Usability Testing”  What a frustrating experience. They had changed the goalposts!

The experience left me angry and frustrated. I had approached the whole exercise well. I had involved the stakeholders, I had worked out their mission. Why did the customer have to go and change their minds?

I suddenly felt empathy developers whose requirements continuously change.

It then struck me that in a way I was the problem. I was being too inflexible and rigid in expecting an inexperienced customer not to change their minds.

I’ve resolved to be far more open in my approach to what’s required (for this project). As long as the customer understands the financial implications of such decisions who am I to argue?

After all is not the customer always right?

Test Tools for Covert Operations

I have one of my clients to thank for this tip.

I was performing some software testing recently on a product that had both a software and hardware element to it. The unique setup of the laptop, and  peripheral equipment meant I that there was only one test environment available.  I was fortunate not to live to far away, but the developer lived in England and so would have to upload any fixes and new versions remotely.

The client insisted we use LogMeIn for the remote access.

Now, I like my software test environment to be stable at least for some fixed period. That way, I know that I’m not testing a moving target.

So when I heard that both the developer and I were to have remote access, I was a bit concerned as the developer  could upgrade software at any time without telling me.

Then I had a stroke of luck.

As any tester worth their salt would do, I immediately started checking out the LogMeIn software. I noticed that under Preferences, there was an Advanced Settings. So, I clicked on that.  To my delight, I found a ‘Screen Record’ option which allowed be to enable recording of anything that happens on the remote machine.

To a control freak like me, this was complete heaven. It was a way to not only know if an upgrade occurred, but also I could learn how the developer ‘tweeked’ the system.

I did at times feel like I was on some covert operation until I fessed up and told him I had it enabled.

Anyhow, if you ever in the same situation where perhaps your developer is not the most communicative, I heartily recommend you look at this free application.

In search of the ultimate bug tracking tool

As many of you know, I’ve been on the hunt for a bug tracking tool for a while now and I’m glad to say the hunt is over.

I needed this bug tracking tool to meet some key criteria.

1) It had to be easy to install. Startups don’t always have a reputation of up front planning and I wanted a tool I could recomend that was quick and easy to install.

2) It had to be open source. I didn’t want to pay for a bug tracking tool unless I had too. My clients feel the same way.

3) I wanted to embed the bug tracking tool into my website Testing Times and then provide it as a service for my clients who don’t have or want a bug tracking tool.

4) It had to be easy to use and secure. I wanted each client to have access to only their project.

TRAC got the heads up for being opensource, but was too hard to install in comparison to YouTrack, which beat it hands down. YouTrack only provided a temporary license though and I had trouble getting my hosting company to support Apache Tomcat. The effort to install something I may have to pay for later, made me feel it was pointless to pursue.

Tails was interesting and looked great, and if I had to pay for a tool, their pricing structure suited my needs. There is no messy install as it’s a hosted service but that meant that I wouldn’t be able to embed it onto my site.

Practitest had similar issues for me, and also the pay per month/user pricing structure didn’t suit my needs.

I was at a bit of a loss until BHARATH suggested Redmine.

Redmine is an opensource tracking tool that’s easy to install, easy to configure and secure.  It ticked all the boxes and it’s now sitting on my website. You can take a look at it at  http://bugs.testingtimes.ie if you like.

I have created a public project called Testing Times for those who want to investigate a bit further.

The real test of course will be my clients, their reaction and how much they use it.

So a hearty thanks to Bharath and a sigh of contentment from me.

Software Test Reports for Startups

My test reports have deviated a lot since the early days of testing.

Nowadays, first and foremost I provide an opinion of the software. I make sure I highlight both positives and negatives that I see in the product.

Naturally, I provide a list of bugs I find, but I also provide a list of suggestions on the software, for example new features or ways to improve the Usability. I find when testing, new ideas often come to me about features. I’m happy to provide them with a list. They can take it or leave it.

Finally, I list some recommendations on testing for the next release.

Why all the effort? I do this because my clients love it!  I provide more than just a list of bugs. I see it as adding more value.

Here’s the template pdf version:: Software Test Report Template

I wonder though, is this still testing, or  have I morphed into a new area of work?

Black and White and Red all over….

There’s an add on telly about this guy who writes his book on a remington typewriter and then the post office sends it and suddenly its in the window of a bookshop. It think the ad is for the Irish Post Office.

Regardless, there is a moment when the author looks into the bookshop window and glows with satisfaction and pride on the achievement.

That’s a bit how a feel like today.

I just have had my first article published in T.E.S.T magazine and I’m on the front cover no less. You can read it here

It’s about one of my favourite topics, software testing and startups.

It’s these types of achievements along with getting work, that keep me working as a independent test consultant because, working on your own can be really tough! It’s a constant act of self promotion, bookeeping, customer liason and oh, occassionaly I get to do some software testing.

There was a post on the software testing club recently on ‘what are the things to consider when outsourcing?” I think the things to be considered are those outside of your area of expertise. Software Testing is only a small percentage of what you will do in order to be successful at freelancing or working as an independent.

Anyhow, back to the article.

As I was reading it, I felt a mixture of pride, satisfaction and surprise. I didn’t realise I could write so well….maybe it’s to do with the fact its a published article, or maybe its because the self satisfaction is putting a rosy tint on the whole thing. I don’t know….I don’t really care.

Now leave me alone whilst I bask in my halo of self content…..

Software Testing & Startups Talk in Dublin and Belfast

If anyone is interested there will be a softtest event this month (May) in Dublin and Belfast. I am giving a talk on software testing and startups.

Here’s the abstract

Software Testing in the world of Start Ups

This presentation offers a glimpse into software testing in the world of startups. It looks at the constraints and benefits facing software testers in this unique environment. It examines what it takes to be a software tester in a startup.

It will also be a bit of a myth buster session, looking at some of the myths around testing in startups. It closes by asking are we as a software testing industry doing enough to help startups improve their software.

Dates

Belfast : 26/05/09 4.30 pm, Venue is the Radisson SAS. To register your attendance email Nicola.McManus at momentumni.org

Dublin: 27/05/09 4.30 pm, Venue is the Holiday Inn, Pearse Street. To register your attendance email Anna.Donegan at ibec.ie

Agenda

16:30 – 17:00 Registration

17:00 – 18:00 Professionalism in Testing, John McArdle

18:00 – 19:00 Software Testing in the World of Start Ups, Anne- Marie Charrett, Testing Times

19:00 Close

I think you might need to be a member of softtest which is free, go to the softtest website for more information

I lookforward to seeing you there!

Better than Beta? You Betcha!

Software testing has gone through a few renaissances in the last twenty years or so. From the dark ages of the waterfall process, there has emerged new theories, schools and test tools. We even have online clubs and networks.

One noticeable difference between then and now is  testing was traditionally part of development. Now, most larger companies recognise the need for some form of testing team.

So don’t you think it’s strange when it comes to software testing, startups still remain in the dark ages of developers testing their own code and then skipping straight to a beta test? The rationale is that it’s an easier, faster, cheaper approach than employing a software tester.

I ashamed to say that until recently I bought into this misconception. Its only when I started doing some research into beta testing, that I discovered the amount of work and effort it took to. So I’m doing a Mythbuster session.

Myth 1: Beta Testing is easy

Beta Testing is hard work. Like any major task it needs planning and effort. Successful Beta Testing takes lots of planning and lots of effort. Upfront analysis into the number of Beta testers required and how they will be sourced.  How much time is necessary for the Beta Testing,  how many releases will you have (yes, you need more than one!).  How will Beta testers sign up and will their feedback need to be secure? How will you keep track of the bugs found, and how much time will you need to support the effort?  There is no point having feedback from 200 Beta testers but not having the time to analyse the information, let alone fix the problems. What tools will you need to keep track of the Beta test?  These are just some questions that will need to be answered before you can begin your beta testing phase.

A couple developers talk about their experiences of Beta Testing..Joel Spolsky – founder of FogCreek Software and SyneRyder Journal

Myth 2: Beta Testing is Quick

The length of time beta testing takes depends on the complexity of the software being tested and the number of releases you plan to have. Typical estimates range from four to ten weeks depending it seems on your experience of Beta Testing.  One constant that remains, is that beta testing always takes twice as long as you estimate. That’s because one of the difficulties with Beta Testing is your dependence on people who don’t owe you anything and in that sense are under no obligation to meet any of your deadlines.

Myth 3: Beta Testing is Free

The greatest myth of all.   The man hours spent in planning, setting up, monitoring makes beta testing a time consuming task. Time far better used by a developer in turning the software into a great product. The end result is your developer is distracted and performing tasks that many software testers can perform quicker and cheaper. On top of that, testers know how to test software really well, something you can never guarantee from a Beta tester.

Software Testers do it better

Software testing has really matured in recent years, and there are many ways to provide quality testing that meet the unique demands that startups face. Software testing does not necessarily equate with streams of documents and restrictive process. You only have to look at techniques such as Rapid Software Testing by James Bach and open source tools to know that there are many alternatives to traditional software testing. There are so many good software testers out there, available for hire on a freelance basis, it makes these age old myths about software testing and startups, a bit passed their use by date.

It’s time for startups to do themselves a favour. Hire a software tester, even for a couple of days. It’s worth every cent…