Paying Attention to Paying Attention to Detail

One question I often ask a tester in an interview is what they believe are the essential skills that differentiate a good tester from a great tester. One of the common replies is “Paying attention to detail”. But what does that really mean?

Start with detail. What do you mean by detail? Mrs Google offers the explanation “an individual fact or item”, but what sort of detail? Visual detail? Detail from a coding perspective? Any detail or only detail that matter? Is it detail as in ‘small stuff’ as in the Devil is in the detail?

Also what exactly does “paying attention” mean?

Maybe it’s one of these interpretations:

  • A tester is attentive as in a teacher saying “pay attention!”? As in be alert?
  • A tester is only observing detail?
  • A tester is observing detail exists and then noticing they are problems?
  • A tester is noticing detail without a conscious attempt to observe in a methodical manner?

Going deeper, why ‘pay attention to detail’ in the first place?  Is it because you’ve been told to, or because the tester genuinely wants to? Your motivation and purpose play important roles in bug finding. For example what’s your ‘attention to detail’ on being told to complete 40 test scripts in one day, compared to your attention to detail if you get made handsomely for finding a bug or winning a prestigious competition?

Yes, paying attention to detail is important, but so is understanding what you are doing when ‘paying attention to detail’.

In my book, there’s not enough meat on that answer to figure out if someone’s a good fit for my team. I want more. I want my testers to suck the marrow out of that question, to boil it until the flesh falls away from the bone.

Professional testers¹ consciously observes systems using a multiple of models. Their maxim is “the more you see, the more you see”. And that’s only the beginning. Following observation comes evaluation. Now testers get to interrogate information by asking (either explicitly or implicitly) is there a problem? Not content with relying on one oracle they may consciously use many different oracles in their evaluation process.

They then report this information in a way that’s meaningful to the people who want to hear it.

So if you are a  tester coming for an interview on my team and I ask you this question, please have fun with it. Show me your testing process, demonstrate it with diagrams and models and show me a bit of your testing flair. All testers pay attention to detail to some degree. What in particular do you pay attention to?

¹ by professional tester I mean someone who makes an attempt to understand and then improve their personal testing process. They tend to develop a unique approach and style to their testing through years doing, thinking, talking & writing about testing.

This post is a result of a tweet I made which many testers. An interesting aside, my poor spelling was explained away by many as a purposeful trap designed to snare the unwary tester. It wasn’t, though I’m flattered that you would think that of me.

Take the ‘Crappy Work’ Litmus Test

We’ve placed a ban on crappy work within my test team. From now on, we declare that the testing team will refuse to do crappy work.

The challenge is in knowing how your doing good work or crappy work?  This is why I’ve created this simple 5 minute ‘Crappy Work’™ ² test to help any tester to figure out if you’re doing crappy work or not. ¹

1) Do I fully understand what I’ve been asked to do?

2) Why am I doing this task?

3) Who will benefit, and I have spoken to them ?

4) Am I doing this task simply because its part of the process?

5) If I think it’s crappy work, how can I make it valuable?

I’m sure there are plenty more ‘crappy work’  questions that would be useful to add, but my 5 minutes is up.

Why not share your ‘crappy work’ litmus tests? 

¹ Maverick Tester takes no responsibility if you take this quiz and continue to do crappy work

² It’s a 5 minute quiz because it took me 5 minutes to think it up. It’s trademarked so I can sell you a certificate in passing the ‘Crappy Work’ test, thereby making a whole lot of money out of it.

In pursuit of coaching excellence

When you coach a tester you’re working in an environment that dynamically changes as both the student and coach work through a coaching task.  If you look at the diagram below you can see all the different attributes that might change throughout a coaching session.

Coaching Space Bach & Charrett

Also, throughout the coaching session, the student and coach have a mental model of the coach and themselves. They constantly re-evaluate these models as the coaching session progresses.

The coaching I do (and James Bach does) requires that the coach has a testing syllabus that they use to help the student. This is different to life coaching which is non domain specific. Also, our coaching is lot more directive. The relationship between the coach and student is more coach->student than the traditional peer-to-peer relationship you find in life coaching. I see our coaching more like sports coaching, where a coach outside of a game, runs you through drills and challenging exercises to help you improve, often without realising your in need of improvement.

Personally, I’ve experienced good and not so good sports coaches. In my school days I was a bit of a field hockey superstar (I joined the grade A hockey team two years ahead of time, making me the youngest player). My coach however was incredibly overbearing, shouting and yelling at us and telling us how hopeless we were. I’m not sure if we were hopeless or not, but I know we failed to win many games and left the season completely demoralised to the point I gave up hockey for four years. I was persuaded to pick up hockey again and this time we had a different coach. She was quiet, never said too much and let me play my free style. One day she came up to me and be a quietly suggested I move back 10 metres to be able to better angle my shots ( I was in a midfield position). Very quickly I realised the power of such a move, I was in a better position to be able to control the game. I was 16 when that happened and I’ve never forgotten the power of that one statement.

For me that’s what coaching is about and its the type of coach I aspire to be. Its directive but the direction is about the skill and how the student is performing that skill. Where its non directional is that I challenge the student to think for themselves. It’s paradox at play but one that works.

Its also powerful because it’s watchful, ready to tap into what a student is doing at an appropriate time, using pressure and energy as tools to make direction powerful to the student (just like my second sports coach did). The aim is to help the student feel empowered to achieve more.

But the energy is not only in one way. The coach is getting energised by the coaching session too. I’m constantly evaluating my coaching and testing ability. I become a better coach by doing this. My aim is to become the best coach I can be.

I can only do this by coaching lots of testers, evaluating the transcripts and also working with colleagues who inspire and want to become better coaches too. I’ve been doing that this week with James Bach. We’ve been working on our book on coaching, identifying ways in which we coach, syndromes that both student and coaches encounter (we need to do more of this) and also finding ways to better evaluate coaching transcripts.

I think an aspiration of excellence in any field is such a worthy goal. I was watching Ron Ben Israel who is a master baker of sugar dessert flowers. You can see his passion the how is pursuit of excellence has led him to create masterpieces in sugar. Who would have thought that you could become excellent in such a small field?

Excellence I think is different to perfection. Perfection to me sounds more absolute, perhaps a little unrealistic. Excellence however, is within my grasp but also always one step ahead of me. I can be excellent at one point in time, but I can always strive to be more excellent. I think this is a worthy pursuit and a good use of my time and energy.

What are you in pursuit of?

(if you want a coaching session on software testing, please contact me on Skype. My id is charretts, please include the word “coaching request”. I offer free skype coaching to testers as long as you’re willing to allow your transcripts to be used in my research and perhaps in the upcoming book. This means the transcripts may be published, though I do conceal the second name and any potentially sensitive data).

Crossing the bridge over no-man’s land

Indiana Jones in the Temple of Doom  has a “moment of indecision” when half way across a bridge he realises he’s been cornered by Mola Ram(the baddy) and his henchmen.

He looks back and there’s a hoard  of lusty savages baying for his blood, no luck there. He looks ahead only to find an equal challenge ahead. What is poor Indy to do?

Anyone who has introduced change into a corporate environment will empathize with his situation. You know the decision to break away from the past is the right one, you run eagerly and embrace change, only to find half way through the journey, your path to success becomes blocked.

I’ve been working with my test team for a while now moving from a process driven approach to a more of an Exploratory one.

Its not been without its challenges. Some concepts I’ve introduced have been welcomed warmly but the reception to others has been a little icy. In particular I’ve tried to move the team away from a test case management system. This was met with real concern and there was quite a resistance to the idea.

This troubled me as while I understood their concerns, I knew the system was limiting the generation of new testing ideas.

But how could I overcome this resistance? And really was it worth it? Perhaps the changes I had already made would be enough? The company was already more than impressed with the changes I had made so far.

I felt like Indy at the foot of rope bridge, how the hell was I going to solve this one?

So I stood at my crossroads and dithered. Oh God, did I dither. I ummhed and ahhed and pondered what to do. . But worse, I  knew my indecision was making the situation worse, and that the more I dithered, the harder it would be to rid ourselves of the dust bag of tired and well worn ideas.

Indy at this point, decides his only move is to cut the bridge leaving everyone to hang on for their lives.

Fortunately, unlike Indy I had a reliable and trustworthy sidekick.  Together, we setup a task force within the team to attack the problem. After some discussion we decided our approach needed four cornerstones. They were:

1) Creativity.

However we tested, our approach needed to enable us to foster and encourage creativity. With creativity comes new ideas, new models, new tests and so discover new bugs.

We’re covering this one with a number of approaches. One is to improve tester skill through practice and coaching. I’ve also created a folder of ideas for people to draw upon to help trigger new ideas.

2) Visibility

We wanted to be able to provide reporting on any testing we do. The reporting has to be simple yet with sufficient detail to ensure that our stakeholders understand what we have tested and why.

We have our trusty whiteboard which mostly hits the spot. We need to be able to pull up our actual testing including results in an easy to manage way. We’re looking into BBExplorer to handle that.

We will also track any essential test results on wiki in the form of a test report at the end of each iteration.

3) Coverage

We wanted to have some way of ensuring that key functionality/key features are always tested.

We most likely will rely on our test case management system for this, but we’re cleaning out all the dead wood and making the tests lighter and less authoritative.

4) Knowledge

We wanted to create a knowledge base. Our system is complex and it requires in-depth knowledge to test some areas. We want to store that information and knowledge. We also have a serious amount of test data we want everyone to be able to access, modify and improve.

We’ll use our internal wiki for this.

What I really like about what’s happened here is that the team came up with a solution to solve the problem. It’s a team decision which has got to mean easier implementation.

I think a couple of really powerful things have come out of this. I’m listing them here:

1) Change can be scary. Not changing is worse. Get on with it.

2) Use people around you to help bring about change.

3) Never lose site of your goal. This reminds me of Scott Barber’s email signature: “”If you can see it in your mind…you will find it in your life.”

I feel good. I hope my team does too. We faced a challenge. We examined it, questioned it and overcame it and we’ve all come out sharper, enlightened and positive about the changes ahead.

Now that’s what Exploratory Testing is all about.

 

A scrum in Croke Park

I’m attending the SQS conference on Software Testing in Croke Park, Dublin.  I thought it was appropriate to go to an Agile Testing session involving Scrum amongst other techniques in the same hallowed ground where not to recently a game of Rugby was played out between England and Ireland.

As our trainer Mike Scott was English, we tried not to gloat too much.

I won’t bore you with lots of analogies on how Agile is similar to rugby, besides after a day of Agile, I can’t think up too many, I’m sure someone out there can….

But here is what I enjoyed about Agile and its techniques

I liked the concept of the balloon pattern and testing so early that no code has yet been written, only your installation packages. I think thats really smart. You can iron out all your installation and configuration issues up front.

I like the concept that we as testers need to ask lots of questions and not make assumptions, though I think this is not unique to Agile.  A course on  Rapid Software Testing by James Bach also stresses this point.  However,  Agile demands intelligence in testing, where perhaps more traditional methods are less exacting?

There seemed to be a heavy dependency on Test Driven Development (TDD) which I am a big supporter of, though I do question the use of 100% Acceptance Test Automation.  I think in every software testing exercise there is room for both manual and automated testing. Its a question of intelligently planning out what percentage ratio works best for that particular project or environment.

Is Agile faster and cheaper as its sometimes portrayed?  I suspect not, but it does offer a customer greater flexibility and visibility and I like the sound of that!

I’m glad I’m not a piece of software

I joke about my personal development lifecycle (PDLC) as my work in test management often requires consistency & repeatability in the form of a test process. In comparison my own PDLC is often very erratic and ill behaved. In fact its not really a typical lifecycle at all except perhaps that I was born and one day I will die.

My family and I have recently decided to spend some time in Ireland. In November, we are uprooting our family in Melbourne, Australia and dumping ourselves back into a very sodden, albight green sod of turf known as Ireland.

We are doing this without any certainty in jobs, schools, long term accomodation etc and its making me extremely nervous. To make it all the more uncertain, the world has decided to have a global financial crisis. My faith that all will work out is being severly challenged!

As I  continue along my PDLC, my outlook on life has changed. What would have once been an adventure is now looking like a torturous test.

So I repeatably breath my mantra, “it will work out, I will find work in Ireland” . Who says life is dull?

Anyhow, back to software testing and lifecycles.  I suppose the difference between software and people is that we own our lifecycle and thankfully we get to choose how our life turns out.  We can choose to make it unpredictable and we can behave outside a given setup of expectations.  So even though my life at the moment is in turmoil, its my turmoil. I’m glad I’m not a piece of software that has to follow a rigourous testing process, in order to live up to someone’s expectations.

Celebrate your wins in software testing

Too often software testers are the bearers of bad news:

“There is a major defect that will stop us rolling out to production on time”

“We have to increase the budget to ensure that the system is properly tested”

The more I roll software testing concepts to companies, the more I’ve realised the importance of celebrating your wins.

If you don’t who else will?

Small wins are just as important as big ones. I don’t wait for a software testing process to have all the bells and whistles of metrics before I start shouting about the great benefits that testing has provided.

For example; I was rolling out a software testing process that was new to the client.  At every meeting (in particular the senior ones), I declared:

“In addition to completing the system testing, we found seven existing production defects”

I wanted people to know that we have good testers in their company, and also that they were providing a good service.

As testers I think we need to be smarter about how we promote our services.

 

Take the Test…

Want to review your software testing process? Here are some guidelines that I use to review a company’s test process.

Stakeholder support

Before examining software testing in detail, assess your financial and managerial support. Any good review exposes shortcomings and to act on these changes will require good managerial and financial support.

One way of getting the support you need is to present a business case outlining the benefits software testing brings to your business. Software testing because its ‘good process’ is insufficient justification. An accurate and persuasive business case from a business perspective is required.

Common software testing benefits to business are:

  • Confidence in the software’s ability to behave as expected
  • Confidence in the system’s robustness
  • Risk Mitigation
  • Tangible knowledge on the systems performance
  • Overall cost reduction in the software delivery lifecycle
  • Increased customer satisfaction

Align these benefits with any company mission statement and it will re-enforce your case.

Software testing process

A software test process delivers reproducibility of results, repeatability of tests and the consistency across multiple projects. A software test process is the cornerstone for many testing teams.

Review the effectiveness of your software testing process to determine if improvements can be made.

Here are some ways in which you can review your test process;

Perform test process reviews on completed projects. Create a questionnaire on the software testing process used and ask developers, testers, managers and if possible your customers to provide some feedback and suggestions on improvements.

Review the test templates and examine how they where used. Discuss the templates with testers to discover ways to improve the templates.

If your software testing process is informal, discussions with developers, business and managers will help identify and formalise what is perhaps an informal and ad-hoc activity.

Skill and resources

Consider what resources you have available to perform your testing. Dedicated resources have the advantage for the following reasons;

they can focus full time on testing, reducing the test execution time compared to a resource who is performing both testing and business as usual activities

A dedicated resource has more experience in scoping out the test effort. They are familiar with the pit holes and understand the need for proper planning and setup.

A dedicated resource provides an independent and impartial view, taken from a user’s perspective. This often helps find critical defects before the user does.

Dedicated resources may initially seem as an additional cost but they improve the effectiveness of testing and reduce delays in testing caused by lack of resources or inefficient planning.

If testers are included in your resources, examine their skill set and determine ways to improve their knowledge of testing. For example, look at training courses or training on testing tools.

Testing Tools

Testing tools are an excellent way to facilitate reproducibility, repeatability consistency and uniformity. The catch is that to reap the benefits, they require total team participation and be kept up to date.

Automated test tools improve efficiency and reduce time though do not necessarily reduce the resource effort. This is because automated test scripts like any software requires maintenance. Testing tools need to be assessed with your testing requirements in mind, so careful thought is required before purchasing any tool like this.

Typical types of testing tools that are available are;

defect tracking tools

automation tools

performance loading tools

test script repository

An essential testing tool is the defect tracking tool. Defects are the key method of communication between developer, tester, manager and customer.

Also, a central repository that holds all defects are their status facilitates the sharing of this knowledge.

A test script repository is helpful for tracking the test effort and the results as well as centralising where test scripts are kept.

Collaboration

Testing affects developers, business, support and infrastructure and naturally testers themselves so it’s essential to examine ways to improve testing collaboration. Review how communication is performing throughout your software testing process. Effective methods of communication are daily or weekly meetings, defect reviews, documentation such as reports and emails, though not too many!

One method to improve collaboration is to get all parties involved in improving the software testing process. Their input and feedback on the software testing process will help make the process relevant to them and so increase the chance of testing success.

Test environment

Look at your test environment and review it for the following;

Security

Check how much control testing has over the test environment. Test results can be compromised if the test environment is open to all and it’s possible to make code or configuration changes without a tester’s knowledge.

· Suitability

Having your testing environment mimic as closely as possible the end user or production environment increases the confidence that the software will behave as expected.

Planning
Some of the major delays in testing are caused by initial setup problems in the test environment. Good planning reduces the risk of something going wrong. Review how you plan your test environment and ensure that the test environment is ready when you’re ready to start testing.

In conclusion, a software testing process is only useful if it’s in use. If you do require changes, make sure their relevant to your business.