Four Stages of Testing Competence

I have Chris Ashton to thank for this little piece of inspiration. He refers to levels of competence in a recent post of his. I was intrigued by the reference and went off to check out what the levels of competence are all about.

The “conscious competence” is a model that describes how we progress through four psychological states in learning. They are:

  • Unconscious Incompetence
  • Conscious Incompetence
  • Conscious Competence
  • Unconscious Competence

I adore these terms and I can easily relate all but the last state to moments in my testing career. In particular the first one!

Unconscious Testing Incompetence

My keenest recollection of unconscious incompetence was at my first job, compliance testing ETSI standards. I wrote an official report for the standards body, bound it and was ready to send it off. It didn’t occur to me to read through the document to check it was in order. Fortunately, someone else did and found sections of the document were completely illegible.

Conscious Testing Incompetence

Things progressed, I learned how to critique my own work, and was put in charge of a new test team. I learned abourt IEE829, documented tests scripts, how to write a bug report and ISO9000. I felt I had arrived. I now had a wealth of information on top of which I could construct my testing. I have order and reproducibility and I got hung up about process. The process became more important than the testing.  It didn’t occur to me that most of the bugs I found where when designing my test scripts, and executing them was really just a formality.  I started to get bored with testing though, and set my goals on greater and loftier achievements. I wanted to be a test manager!

Conscious Testing Competence

Well, I ended ditching my career as test manager to because guess what? Yes, I got bored.  So I went back to the drawing board and rethought a) what I wanted out of my career and b) what really where my goals in life? I had always wanted to setup a testing consultancy, so that was what I went about doing.

I feel that I’m in a state of conscious testing competence now, not always though. Sometimes I pop back into unconscious incompetence, and on rare occasions I jump into unconscious testing competence.

I can say that because I’ve learned a lot about how to test well and I try to implement that. I’ve made a conscious decision to keep my testing skills keen and relevant, and I find the only way for me to do that is to keep testing. I’m putting myself on courses and training myself up where I need to.  I’ve learned that I will always have much to learn about testing. There are always new tools and  new techniques to try out.

Unconscious Testing Competence

Ah, the holy grail! I see glimpses of this sometimes, but there is so much more to learn about testing and its such a broad field, that I doubt I will ever achieve this for all areas of testing. Sometimes I do feel that exploratory testing is “second nature” to me but to be able to teach it to others? Maybe not so well. I find it hard to believe that I ever felt that I knew everything there was to testing. What arrogance!

So, what  testing competence are you?

Software Testing Training of a different kind

Ever dreamed of being personally trained by Cem Kaner? How about Doug Hoffman? No? Ok, maybe Scott Barber is more your style?

Any tester would, but it would be expensive? I mean these guys would typically charge thousands for this kind of training. Besides, they’re all in the states, so airfare, accommodation.  Your company would never pay for it, right? Especially not now the training budget is cut.

Well, I’m being personally trained by these great testers, right now and even better, I’m doing it for FREE. No that’s not an acronym for  Footnote Really Exorbitantly Expensive,(I know its not very good, come up with a better one and let me know !)  it really is FREE.

I’m on the bug advocacy course run by the association of software testing. (note, the website is currently going an upgrade, but you can still become a member etc).

It’s an online course that’s free to all members. Yes you do have to be a member. Yes it does cost money. $85 US dollars for a year. So, if you want to split hairs, you could argue the course costs $85. So what?

Anyhow, I wanted to talk about this course, because its content is really excellent. First of all, lets deal with the title.


Because our focus as tester’s is not about raising bugs, but ensuring they get fixed. Its true!  Think about it. That way, not only does the software improve, but we as testers gain credibility too.

So, in order for us to get our bugs fixed, we need to make sure we sell them well and we anticipate and preempt any objections people may have about our bug report.

Its about communication effectively and communicating to the right person. Its about creating the ultimate bug report.

There is a lot more than that in the course. I suggest you take the course and find out for yourself.

The recommended hours per week is 6.  I would suggest you allocate more time.   The course content is practical and you are given assessments that involve commenting on actual bug reports. Some of the assessments can take longer if you allow them too. You work with testers around the world, some I knew of because of their blogs.  Personally, I’ve learned as much from other testers feedback as I have from the trainers.

Make no bones about it though, its a demanding and challenging course and if you are just looking for a piece of paper for your resume, this perhaps is not the course for you. On the other hand, if you’re looking to improve your software testing skills and having a highly respected certificate matters to you, then I suggest you join the AST and take some of their courses.

It doesn’t matter what environment you work in, agile or waterfall, process or exploratory, the skills you learn on this course are relative to all testers out there.

So, if your company doesn’t have a large training budget (or even if it does) this is the perfect solution. Your testers get some really great training, and you get kudos to boot.

Note:  This is my own personal opinion, I get nothing out of posting this on my blog.

All the leaves are brown, and the sky is grey..

It was my birthday recently, and I got a brand new Digital SLR as a present. Now my expertise on photography goes as far as knowing that DSLR stands for Digital SLR. Say no more.  Having this camera though, now means I can’t blame bad photos on the equipment.

In Ireland at the moment, its the last of the autumn leaves which is my favourite time of the year.  All those crispy days and beautiful golden colour. I’ve been in Australia for so long, days like this are a real treat. So inspired by the beauty around me, I set off in true Ansell Adams fashion to photograph the local park.

I started down the usual path, and took some average to terrible shots, nothing spectacular.

For some reason, I decided to turn around and look behind me. It was then that I noticed some true beauty. I started clicking away.

Behind Me in St Enda's Park

Behind Me in St Enda's Park

It got me thinking how I tend to always look ahead at things. Often I forget that the best can be simply found by turning around and looking at the same thing with a different perspective.

I think this happens in software testing too. In trying to solve a problem, my thinking is often straight ahead, down the traditional path. Perhaps occasionally, if I looked behind me, I would have a different perspective and find a better answer to my problem?

Anyway, enjoy Dublin in November!

Autumn Leaves

Autumn Leaves

Leaves and Sky

Leaves and Sky

Bug Tracking Tools Explained

Like it or not, all software has bugs.You may not know about them but they are there..lurking in the darkness, ready trip up one of your innocent user who has just gone and purchased your software.

That’s why you employ a software tester. A software tester’s job is to find as many of the bugs and report them to you and the rest of the team before they can do untold damage to you and  your software’s reputation.

You can report bugs in as many ways as there are to communicate. You can write the bugs up in an email, on a piece of paper, in a spreadsheet (see my previous post on spreadsheets). You can even directly speak to the relevant person and directly show them the bug.

Or you can use a bug tracking tool.

So why do you need to track a software bug and whats so great about a bug tracking tool?

Software Bug Workflow

Well, just as a normal bug goes through several stages in its life such as egg, nymph, larvae and finally adult, so to does a software bug go through its own lifecycle or workflow. Redmine the opensource software I use as my online bug tracking tool,  uses the word workflow so I’ll use that term in this post.

Typically a simple bug workflow goes as follows:

Bug WorkFlow

Bug WorkFlow


new is when a tester creates a bug report

open is when a developer  accepts the bug report as valid

fixed is when a developer indicates that the bug is fixed

tested is when a tester indicates the bug has been tested

closed is when a tester accepts the bug has been fixed and the report is now closed

This is a very simple workflow. Alternatives to the workflow can happen when for example,  bugs are rejected by the developer  or a bug fails test and the tester places the bug back to open state. In fact, it can get quite complicated if you let it.

Most bug tracking tools will let you modify or create your own stages and workflows, but if your new to the concept of a bug tracking tool, its better to select a simple existing workflow and use that for a while to get used to what works for you. Redmine, lets you chose a preexisting work-flow.

A bug tracking tool then allows a tester to create a bug report and monitor its progress as it goes through its workflow

So whats the big fuss? Why not just use a spreadsheet or email?

I’ve personally benefited a lot from bug tracking tools. Here are some ways they’ve helped me.

Benefits of a bug tracking tool

1) Its easy to keep track of one bug, but keeping track of many bugs is hard work. A tool helps you easily find out what bugs are still open, fixed, closed. Bug tracking tools normally allow you to sort and filter your bugs and create reports on the bugs.

2) You can track other stuff about bugs, such as how important they are and who is fixing them. This can help you prioritise which bugs are important and require urgent fixing.

3) You can start seeing clusters of bugs which indicates there may be underlying issues in parts of the code

4) Lots of people can see the status of the bugs, not just the tester and the developer. The overall bug status can be quickly reviewed by many avoiding nasty surprise syndrome (NSS) at the end of development/testing.

5) Sometimes not all bugs are fixed in the current release but will be fixed and tested in a future release. This means long after software release the bugs still need to kept open and tracked. A bug tracking tool ensures that these bugs are not overlooked in the future.

6) A bug tracking tool can centralise information. Often a bug tracking tool can be used to track new features as well as issues and can act as a document repository. Redmine offers many additional features such as document repository and wiki.

7) A bug tracking tool can improve productivity by increasing bug awareness and responsiveness. It is also handy when a project is scattered geographically and works across different time zones.

As I’ve mentioned in previous posts, there are many open source options bug tracking tools out there, so if budget is an issue there is no need to spend a lot of money on an top end commercial tool.

And of course, a tool is only as good as the developer and tester using it and will fail miserably if no one is willing to use it, so perhaps if your thinking of adding such a tool to your testing toolbox, speak to the rest of your team first to find out what they are looking for in a tool.

Is automation the new documentation?

Have automation tools become the new templates in software testing? The cornerstone on which all our testing hangs on? Its starting to look that way.

Think about it.

The traditional method of testing was to use a software testing process identifiable by test templates. These templates were (and in many cases still are) the focal point of all testing.

Then some new thinkers introduced concepts such as Exploratory Testing and Context-Driven Testing, placing the tester central to the testing exercise. It rocked the testing world.

Then along came agile with its focus on automation. This has lead to a heavy emphasis on automation instead of  the individual behind the tool.

Traditionally the majority of time in testing was spent writing heavy onerous documents, instead now, we are writing automated test scripts. What has gone wrong?

I was suprised at the emphasis on tools in agile testing talk I went to, as I am suprised at the interest in automation tools by all testers.

My feelings about automation are best described by this post by Kevin Pang where he writes:

The difference between a good developer and a bad developer isn’t whether they use duct tape, it’s how well they can recognize whether a situation calls for it.

There is a similar problem in automation testing.  Automation in itself is not a good or bad thing, its knowing when to use it. That’s whats been overlooked in this whole agile testing excitement.

The skills required to be a good tester are being overlooked in favour of good automation skills.  Instead of “are you a good tester”, you are being asked “what automation tools you know”.

I can understand why, its easier to quantify automation tool experience than quantifying what makes a good tester.

We need take more care about what we discuss in Agile testing. The emphasis on what tools is  too heavy and a rebalance is overdue.

More emphasis needs to be put on how a testers creates a strategy for automation, not just the type of tool they use.

It’s time to reset the balance and put the tester centric to the testing effort.

Rediscover your inner tester

Linda Wilkinson in a recent post called on ‘Experts’ to come down off their ivory towers, and get back in touch with the rest of the ‘hoi polloi’.

Ivory Tower

Ivory Tower

It got me thinking about how in one way or another, we all have an Ivory Tower in testing which we can brag about. Your Ivory Tower (if its anything like mine!) is a nice place to be in. 

We can sit back and feel comfortable there, we know what we are doing and people treat us with certain level of respect.  No-one decided in advance to build these towers, but over the years, as we have specialised, it has become an area we have decided to call our own.

Some Ivory Towers I’ve come across in sofware testing are:

  • The Methodology Tower

    Over the years one process or methodology dominates and closes our minds to  new approaches such as Agile or Exploratory Testing. Or, we are so won over by Agile, that we fail to explore other avenues that may be of benefit.

  • The Trainer’s & Speaker Tower

    After many years of testing at the trenches, the experience acquired is used to help other testers by training and speaking in conferences. Gradually, the speaker/trainer looses touch with the tester on the front and starts finding it hard to identify with issues testers face on a daily basis.

  • The Management Tower

    Climbing up the corporate ladder, the Test Manager spends most days, managing people and projects. Their goals and challenges differ from the tester on their teams. They too start to loose touch on the real issues that testers face.

  • The Manual/Automation Tower

    As a tester, perhaps  whilst performing other tasks, perhaps you have decided to specalise in only certain areas? Manual testers, when did you last try out automation or performance testing?

There is nothing wrong in specialising, or having a niche. But in building these towers, how far have you wandered from the path of testing? You know, the actual testing, where you sit down in front of an application and well look for bugs?

In narrowing your skillset, I think you narrow your mindset, and consequently loose out on all the benefits that testing have to offer.

Puzzles and writing articles are good ways keep up your cognitive skills, but really nothing beats testing a product to remind of the real issues everyday testers face. Try getting your teeth stuck into a good testing problem, and remind yourself of the pleasure and heartache that testing can bring.

Puzzles and forum discussions will never replace or use the sheer number of skills you require in testing such as communication, negotiation, cognitive and written skills.

So, if you really want to get in touch with your inner tester get out and test. That doesn’t mean you have to give up your training or test management or what ever area you have chosen to specialise in, but there is no reason why you can’t contribute to some opensource testing, or volunteer to assist a charity in their software testing. If you can’t see your way to doing that, try keeping  the tester in you alive by going for the many testing challenges that seem to be popping up. I really like the Weekend Tester’s created by  Ajay Balamurugadas. They set themselves challenges and applications to test as a way of improving their testing skills. Matt Heusseur is another person who set a challenge on his blog.

Go on, I dare you to step outside and sniff the air outside your Ivory Tower and rediscover your true inner tester.

Changing the Goal Posts

I got a chance to watch some Gaelic Football recently as my two sons have started playing hurling and gaelic football.  Hurling is a fairly ferocious sport where everyone goes for the ball with hands and sticks. Gaelic Football gives the appearance of being a far kinder sport. However, the Australian version (aka Aussie Rules), is a much more physical and aggressive game.  One thing that differs between the Irish and the Australian version is the goal posts. In Aussie rules points are scored when you kick through one of four poles, Gaelic, you kick into or over a pole. Little differences but really important to know and get right.


Adam Goucher had a nice post this week on communication and letting your team know where the goalposts are. Well I have had a similar goalpost problem. Except in mine, the goalposts changed. How I scored the goal had changed, and no-one had told me.

I have a very nice client (he’s a client, so he has to be nice, well in this case he actually is very nice) but new to the software development, and a grappling with some of the basics do’s and don’t to mostly people in IT take for granted.  It’s the basic advice they require most. Like, letting people know when things change, so I can relate to Adams comments.

I spent a good amount of time before the project, understanding who the stakeholders were, and the mission of the testing effort. I explained to them the many different types of testing I could perform, such as Usability, Performance, functionality, Installation and Configuration. It was decided to narrow the scope to robustness, it was a prototype after and all and as long as the screens did not crash, that would be satisfactory.

The day before delivery to their customer, there was a review. The stakeholders were dismayed at the state of the screens and the way the functionality worked. But I spluttered, “you told me you didn’t want Usability Testing”  What a frustrating experience. They had changed the goalposts!

The experience left me angry and frustrated. I had approached the whole exercise well. I had involved the stakeholders, I had worked out their mission. Why did the customer have to go and change their minds?

I suddenly felt empathy developers whose requirements continuously change.

It then struck me that in a way I was the problem. I was being too inflexible and rigid in expecting an inexperienced customer not to change their minds.

I’ve resolved to be far more open in my approach to what’s required (for this project). As long as the customer understands the financial implications of such decisions who am I to argue?

After all is not the customer always right?

Damn it all, its just not cricket..or is it?

After fifteen years of living in Australia, some things have rubbed off on me. One of the them is cricket.  I have to confess I have spent many a sun drenched day watching the cricket whilst imbibing the odd beverage here and there.

Cricket Image

One thing that I have noticed in the game, is the impact of the new ‘unknown’ player. It suprises me the impact the unknown can have on the game. Michael Clark (or Clarkey to his mates) on his debut in Bangalore went out and scored 151 runs (thats a lot btw).

Here comes the testing bit…

Fergal O’Riordan,  pointed out a similar result in his software testing teams.

When testing is all but hung up and dry, he enlists a new tester.  This tester must be new to the team, and never worked on the application before.  He finds that doing this dramatically increases the number of bugs found in that day.

He breaks the bugs down as follows:

  • The majority are known issues, but previously where not considered to be bugs. The new tester didn’t know that and raised them. Changed circumstances and attitudes now regard these bugs as valid.
  • Most of the other bugs were also known issues, but were either seen as not relevant to the stakeholders or were being dealt with in future releases
  • a small number were new and relevant bugs that required fixing.

So why is it that a new tester is able to have such an impact? He puts it down to the following reasons:

1) These bugs have been seen before but previous experience led testers to believe they were of no relevance to the stakeholders

2) A new set of eyes brings a fresh perspective and outlook to the testing

Doing something different, unpredictable can bring benefits to your team, no matter what the ‘sport’ is.

So there you go, who would have thought that the world of testing and cricket had something in common?

Tea break!

How many 70’s references can you pick up?

OK, I’m really not one to get into bragging, but I just can’t help tell you all about this one, because I am so excited!

I’ve been asked to be a weekly contributor to (QTT to those in the know). I feel, honored, excited and challenged.

Honored: I get to post along side people I admire, like Jonathan Kohl. I respect Jonathan because he backs up his thoughts with experience.

Excited: Its something new, something I have never done before.

Challenged: I want to be able to bring something to TPP that is appreciated and welcomed. I love that challenge.

Thats it! No analogy to testing, no words of advice, just simple elation.

Can you feel it ? !!