16-truman.w1200.h630

Nuance on leaving testing to the experts

Yesterday I wrote a blog post entitled “Leave Testing to the Experts” to which Maaret quickly reacted online and by a post where she strongly disagreed with the sentiment.  Disagreement is a very useful tool to clarify your thinking, so thanks Maaret!.

It’s allowed me the opportunity to explain in more detail, why I think what I do, and to investigate some of the nuances on this topic.

Maaret, counter-argued with a convincing description of her egalitarian work place where opinions are offered and accepted in good faith. In truth it left me feeling a little envious, and a commitment to work harder on taking ‘being told’ as “bad communication with good intent”.

But I think this post misses a vital point and that is responsibility.

Taking on a role³ of tester comes with a responsibility to have testing performed to your best ability. So, if I listen to an opinion and follow it, knowing its wrong, I call that irresponsible. If I hire a tester based on likability over capability, that’s irresponsible. If I fail to highlight risk for fear of upsetting team members? Well, you get the point¹.
Perhaps you might say that testing is a collective responsibility, and we all own the decision. After all, in an egalitarian team such as Maarets, doesn’t everyone own quality? Isn’t everyone responsible for testing? Well, yes and no. Let’s explore that a bit by taking the pilot example.
It takes a crew to fly a large plane. There’s the engineers, the pilots, the cabin crew and many more. The pilot happily takes on other crew member’s opinions and ideas. Perhaps she listens to the engineer’s advice not to fly above a certain height. Or maybe she nods when a cabin crew member tells her that people like to hear from her voice more often. Perhaps her junior suggests a flight plan for the journey. She hears all this and takes it into consideration, but flying the plane is ultimately her responsibility². Even if part of that decision is to allow the junior to fly the plane for some of the journey. The buck stops with the pilot.
The crew is responsible for the whole flight experience, but the pilot is responsible for flying the plane.
Similarly, in a team, we’re all responsible for quality, each person may even be responsible for their own bit of testing but ultimately the testing buck stops with the tester.
Notice that everyone got to have an opinion? Similarly on a team, many people can perform testing and offer ideas on testing. This can be incredibly invaluable to testing as a whole. For example, developers have a keen insight into technical risk which can really help identify/dismiss areas that require testing. Teams that are test infected and have this testing mindset are often a joy to work with.
You don’t get a testing mindset if you’re prevented from exploring ideas, so I agree with Maaret that people should be allowed to offer opinions and experiment.
I worked with a team who wanted to implement BDD. I was skeptical. BDD seemed to me a bad fit for the context in which we were developing. There’s nothing like self discovery though and so the team went ahead and tried it out. It turned out to be a dud idea and it was dropped. It’s good to experiment and encourage exploration but there needs to be a proviso here and that is when the risk is low and the impact to the company is minimal.  
Back to the pilot. You don’t handover control of a plane to a junior pilot just as turbulent weather conditions hit. You might if they have more experience in flying, but as the senior pilot you will know when to take the wheel and when to allow practise. Someone who knows what they’re doing needs to make these decisions.
Also performing a small experiment when the risk is low is very different to adopting a strategy company wide. Dictating cucumber as a tool across an organisation is very different to performing an experiment within a team. To clarify, my lament on testing experts is in the context of organisations adopting ‘best practise’ strategies.
Maaret talks about everyone on her team ‘tests like an expert’. I think that’s great, her very capable team means she doesn’t have to exercise responsibility. She’s still responsible for the testing though, just as carrying a plane full of pilots doesn’t absolve the actual pilot of responsibility.
I’ll end this post with the comment about sales people. Maaret says she constantly offer views to developers and sales people. The truth is I have done that too. I guess everyone feels in some way their idea is good and has value and its nice to have your opinions heard.
I remember once querying a company’s sales strategy. I couldn’t understand why we only aligned with partners instead of also focusing on location. I brought this up with the sales manager who listened and then dismissed it explaining why. Personally, I still don’t agree with the approach, but I’m fine with his decision. Why? Because ultimately the buck stops with him. I’m cool with that.
Non testers will always have views and opinions on testing. That’s not a bad thing. In fact, part of me is glad they feel responsibility about testing. But it’s a problem when those opinions start impacting my testing in a negative way. Then I’m failing in my responsibility to perform testing to my best ability. You see, the intent of ‘claiming territory’ is to do a job well, the job I was hired to do, not to hold power over people through control.
I hope this makes sense. Like I said, I’m grateful to Maaret for her response and the opportunity to deepen my ideas on the topic.
Footnotes
³I considered adding the word specialist here as the concept still holds. A specialist (of testing) still holds the ultimate responsibility for testing, even if they don’t perceive themselves as a tester.
¹Many of these decisions are nuanced and are rarely black and white. For example, hiring a tester for a team fit is really important, but if you do so over capability you need to be able to back your decision with some sound logic. Context is important here.
²many people inside and outside a team test, so a tester isn’t responsible for doing all the testing, but they are responsible for the strategic direction and process of testing.

Leave the testing to the experts!

Why is it that in a company almost everyone has an opinion and knows the best way to test? What makes so many of these people feel they know how testing ought to be performed?

Business people tell me, the tester:

“You should be writing test cases and reporting against them”

Developers & technical people tell me to:

” automate the lot using cucumber/PACT/Selenium or some other niche technical tool” 

Test Managers (many who have never tested) tell me:

“You should be following a formalised signed-off testing process”

I find this quite extraordinary. I don’t tell you, oh developer how to code your program. I don’t tell you oh, sales person, how to sell your product.

So why do you think its reasonable and perfectly acceptable to tell me how to test software? 

Perhaps its because we all test to some degree. But just because I can build a car or sell one doesn’t mean I can drive it. Just because I know how to drive a car, doesn’t mean I go and tell a rally driver how to race around a track.

The truth is, most developers, project managers and business people don’t understand testing in any deep way. Instead they often resort to shallow imitations, an emperors test suite, if you will.  That’s partly the fault of the testing profession. We’ve failed to claim our turf and instead tugged our forelock, deferring ideology to those with authority but little understanding.

It’s time tester’s to claim some turf back. I suspect (actually I know) for many places it won’t be given up without a struggle. You’re going to need courage backed by confidence in your ability if you want to claim some of the territory. But it’s worth it. With space comes freedom and the autonomy to drive your testing strategy and process in a way that you know will offer value to your organisation and your clients.

Now, please get out of my way and leave the testing to the experts!

Mount Everest

The Base Camp Heuristic

The ‘Base Camp’ heuristic is the work required to find any bug. Typically ‘Base Camp’ bugs tend to be low lying fruit and can be found quickly, especially if a tester is experienced. In the world of Daniel Kahneman, these bugs are found using Systems 1 thinking.

To discover an ‘Everest’ type bug requires Base Camp intuition plus more. For example a tester might need to deliberate over the product in detail, question the diversity of the data, the validity of the oracle(s)under use and the procedure required for testing.It may mean you need to create new tools, or find new ways to track the unknown.

That means you’re going to need mental training to extend the boundaries of your thinking, determination to continue when the easy answers stop flowing and some tactics to help you refocus and come up with new solutions when you run out of ideas.

You may feel dismayed because everyone around you is finding bugs at a rapid pace. You may even encounter skepticism because you appear to be showing no value. But Everest quality bugs are not meant to be easy to find. They require stamina, commitment and courage.

It’s rare to find bugs of this calibre without training and serious commitment.

Maybe I’m not going to climb Mount Everest every day, abut as an aspiration I’d like at least some of my bugs to be of the Everest calibre.

How about you?

Testing Microservices – what do you want to know?

The following is an outline of blog post on Testing Microservices. Does it sound interesting? Vote below to get me to write it or focus on something else :)

Testing Microservices

1) Microservices is a concept a set of patterns, not a set architecture
2) The context in which you testing has a direct impact on how you test
3) Risk changes, allow your strategy to change with it
4) Diversify your testing strategy

If you have a question you want me to answer on testing microservices, please add it in the comments.

No women speakers? No Bother!

That might surprise you if you listened the Eurostar Webinar last week where Fiona Charles and I explored the numbers of women speaking at software testing conferences. If you didn’t hear it you can now  (Spoiler ahead it’s roughly 25%*)

FemaleConferenceSpeakers

 

Considering that there are more women in testing than other technical professions you might be surprised by this.

Unfortunately I’m not surprised. Fortunately, I’m not bothered by this. In fact, I’m not even bothered that in 2015 the percentage of women speakers has dropped in some conferences, or that the number of women on program committees for many conferences is a big fat zero.

Percentage of women on program committees

I’m disappointed of course, but I’m not bothered by it.

That’s because there are some phenomenal women in software testing and they’re taking the matter into their own hands.

Rosie Sherry working with Anna Royzman for example, has dropped the teaser that women makeup ~50% of TestBashNY 2015 using a merit based selection process. They had a lot of women submitting proposals and I think it’s clear why. What female tester wouldn’t want to speak at a conference run by those two phenomenal and respected women?

There’s more though:

Maaret Pyhäjärvi & Adi Bolboacă have decided to create a conference that they would want to attend. It’s called the European Testing Conference 

Mieke Gevers & Nadine Raes run Belgium Testing Days and there’s typically a high percentage of women speakers (30% this year).

I’m sure there are other examples too.

It’s simple folks. When you create an culture where women feel welcome to speak, the submissions come flooding in.  What does that mean? Well, perhaps women no longer need to be concerned about being underrepresented at dinosaur conferences. Instead, women can focus on conferences that are already offering a healthy environment. Conferences were women are compelled to submit.

I suggest that for any conference in software testing, if the trend of  women speakers is decreasing or wildly fluctuating, if the percentage is consistently below 25%, then conference organisers need to rethink how they are attracting talent.

FemaleSpeakersByConferenceTime

In this day and age, I think there is little excuse for poor female representation. Conferences such as CAST have demonstrated it can be done. CAST has high calibre talks and a high percentage of female speakers. Who would have thought?

How about you, do you think conferences organisers need to rethink how they attract talent?

*figures were taken based on information on websites. We may have made errors in counting, but we think they are fairly accurate representation. Please let us know if we’ve made any glaring stuff ups. 

Agile Australia and Intent based Leadership

Today I went to Agile Australia as an attendee. That’s a first for me, usually, I’m speaking. It was wonderful to go with a different purpose, which is to learn from others. Typically, if I’m speaking I’m so focused on ‘my talk’ and getting ready for ‘my talk’ that I don’t feel very open to learning new ideas.


I’m glad I went, because I had the opportunity to listen to David Marquet’s (nuclear submarine commander) talk about the abdicating control (not responsibility) to the people who worked for him. In a military organisation I can’t imagine the mindset shift & the struggle his crew must have felt!


There were many aspects of the talk I enjoyed, how subtle changes in  language was used to help people think differently. For example instead of his crew requesting permission, they informed authority by using the phrase “I intend”. He emphasised the need for constant training and drills to help keep people prepared for when the inevitable mistakes were made. I also liked the concept of pushing ‘authority down’, allowing his crew to control their decision making.


The biggest resonator though was his perspective that you can’t change people, all you can do is provide an environment where people can change themselves. After all “When a flower doesn’t bloom, you fix the environment in which it grows, not the flower.” I have this ideology too in building & developing test teams. In 2013 I spoke at Tasting Let’s Test about building a context driven test team. In it, I talk about creating a habitat where testers can thrive and grow, with a focus on coaching to support that work. I’m working hard to create a habitat like that in Tyro Payments.


The underlying belief he had was that “people are fine the way they are stop trying to fix them” struck deeply with me. If you start with that premise you come to coaching with a different mindset. And just like I came to Agile Australia with a mindset of learning this year as opposed to speaking, I can approach coaching with intent to listen instead of teach. Instead of focusing on where a person needs to be, you think of were a person is. The spotlight is placed on the person not your perception of that person.  Perhaps it’s subtle differences like this that can make all the difference in coaching. Or maybe not. I don’t know. It’s an experiment, but one I’m performing on myself rather than on other people.


And I think that’s exactly how we all like to learn.

Nice girls don’t get the Keynote

Keynotes are something to be earned and rightly so. Typically a keynote comes from someone who likes to speak, who is good at speaking, and has something interesting to talk about.

I know many women testers who meet the above criteria. They have the requisite ‘merit’ required to be a keynoter. They have thoughtful ideas, are good speakers and are well respected in the testing community.

So why are they not keynoting?

Could it be that to be a keynote takes more than ‘merit’?  Could it be that keynoting is also about your place in the community and your connections within that community?

My experience as a conference organiser and being on program committees is that ‘who you know’ is equally as important as ‘what they know and how they speak’. I don’t apologise for that. It’s important for me to base my decisions not only on an abstract but on a speaker’s reputation.

A keynoter needs to have some of the following:

1) They have interesting topics to talk about
2) They are practiced/skilled at speaking
3) They are involved in some way in the community
4) They network within and outside of their ‘tribe’
5) They make a point of asking for keynotes

I know many practised women speakers with fresh and innovative talks and we are well covered on points one and two.

And we’re not so bad at organising and volunteering for conferences & community either, so….

Could it be, points four and five is what’s holding us back?

Keynotes are typically by invite. That means, the program committee gets to decide who keynotes. Those on the program committee tend to have a good network. It’s probably one of the reasons they’ve been chosen to be on the program.

How are your networking skills?  I’m not talking about card swapping nonsense, but ask yourself: do you have a genuine interest in engaging with people who you respect? Importantly, do you know your network will openly advocate on your behalf? Which brings me to point number five…

Be open to asking to keynote. Is it just me, or is there’s this weird unwritten rule in our community that prevents us asking to keynote? It seems to me that to ‘make it’ we have to Marlene Dietrich like, sit nonchalantly in the corner waiting to be asked for our moment in the spotlight.

Maybe that’s just me and not a general experience. But for those of us who are less direct I’d like to suggest you make it clear to your close network that you want to keynote?

What’s more, go direct to the program committee and ask them for a spot. If the say no, ask why not? and what does it take to keynote,? Who knows you might gain some valuable insights into the process.

Because no matter what skill you have, or how good you are at speaking, or how charismatic you are, there’s dozens of great speakers who can do what you do. It takes more than merit to get noticed. It takes courage to ask and you need allies who will support you along the way. Have you got what it takes?

If you’re reading this post, and you’ve keynoted in the past with a story to share about how you got to keynote, Speak Easy would love to hear it as it may help us understand what it takes to ‘get there’. I’d post a link to your blog post on the Speak Easy website (or I can post a blog).

*Title misappropriated from a book I found useful on this topic “nice girls don’t get the corner office’

Paying Attention to Paying Attention to Detail

One question I often ask a tester in an interview is what they believe are the essential skills that differentiate a good tester from a great tester. One of the common replies is “Paying attention to detail”. But what does that really mean?

Start with detail. What do you mean by detail? Mrs Google offers the explanation “an individual fact or item”, but what sort of detail? Visual detail? Detail from a coding perspective? Any detail or only detail that matter? Is it detail as in ‘small stuff’ as in the Devil is in the detail?

Also what exactly does “paying attention” mean?

Maybe it’s one of these interpretations:

  • A tester is attentive as in a teacher saying “pay attention!”? As in be alert?
  • A tester is only observing detail?
  • A tester is observing detail exists and then noticing they are problems?
  • A tester is noticing detail without a conscious attempt to observe in a methodical manner?

Going deeper, why ‘pay attention to detail’ in the first place?  Is it because you’ve been told to, or because the tester genuinely wants to? Your motivation and purpose play important roles in bug finding. For example what’s your ‘attention to detail’ on being told to complete 40 test scripts in one day, compared to your attention to detail if you get made handsomely for finding a bug or winning a prestigious competition?

Yes, paying attention to detail is important, but so is understanding what you are doing when ‘paying attention to detail’.

In my book, there’s not enough meat on that answer to figure out if someone’s a good fit for my team. I want more. I want my testers to suck the marrow out of that question, to boil it until the flesh falls away from the bone.

Professional testers¹ consciously observes systems using a multiple of models. Their maxim is “the more you see, the more you see”. And that’s only the beginning. Following observation comes evaluation. Now testers get to interrogate information by asking (either explicitly or implicitly) is there a problem? Not content with relying on one oracle they may consciously use many different oracles in their evaluation process.

They then report this information in a way that’s meaningful to the people who want to hear it.

So if you are a  tester coming for an interview on my team and I ask you this question, please have fun with it. Show me your testing process, demonstrate it with diagrams and models and show me a bit of your testing flair. All testers pay attention to detail to some degree. What in particular do you pay attention to?

¹ by professional tester I mean someone who makes an attempt to understand and then improve their personal testing process. They tend to develop a unique approach and style to their testing through years doing, thinking, talking & writing about testing.

This post is a result of a tweet I made which many testers. An interesting aside, my poor spelling was explained away by many as a purposeful trap designed to snare the unwary tester. It wasn’t, though I’m flattered that you would think that of me.

Bobbing for Embedded Apples

One of my first recollections of physics is the image of Archimedes running down the street, dripping wet from his bath and shouting: “Eureka I found it!” on discovering his principle of fluid displacement to measure the force.

Add to that Isaac Newton who quietly sat under an apple tree, musing the day away until he discovered his laws of motion, one which states;

For every action there is an equal and opposite reaction..

Happy days…

Many companies want to improve their testing, particularly if that improvement is translated into reduced time or cost.

To achieve this, there’s a big trend to embed testers into development teams. There’s no longer ‘us’ and ‘them’ but an harmonious ‘we’. The wall of independence is torn down replaced by a gleaming whiteboard emblazoned with the banner ‘Quality is a team responsibility’.

The reality though is that just like Archimedes in his bath, or Newton and his Apple, embedding testers into a team of developers is going to create displacement and cause a reaction.

This is common sense. Put one tester into a team of multiple developers and it’s going to change the team dynamics. Add to the mix, that the tester will most likely become a point of constraint disrupting what was perhaps a seamless flow of stories into the Done column and it’s no wonder the apple cart gets upset.

For example, developers discover they need to get more involved in testing and decision making.That can be a big mindset change if traditionally developers are used to handing work over to testers (as opposed to receiving it from them).

Testers need to change and adapt too. They have to let go control over all the testing and entrust the developers to help them test. Again big mindset change for testers who traditionally rely on empirical evidence than a developers word to evaluate quality.

Why then, do we seemed surprised when we face resistance and/or confusion when these changes are attempted?

This is our brave new world. I wonder where we will all end up?  Wether a team develops a true and meaningful relationship or simply an uneasy alliance is I guess up to how much both testers and developers are prepared to appreciate each other’s challenges.

One way we can help each other out is to be supportive as opposed to defensive when people face the impact of changes and perhaps feel displaced and confused. How much we are prepared to let go our assumptions and prejudices about each other will also play a big part in how ‘successful’ these experiments are deemed to be.

Exciting times indeed!

 

How’s your signal to noise ratio going?

It’s not only engineers that need to deal with S/N (Signal to Noise Ratio). 

Test Managers have to deal with lots of noise, in managing projects, dealing with stakeholders, identifying potential future risk, maintaining test environments, hiring testers, and meetings,meetings & meetings. Traditionally that’s what the role has involved. We deal with the noise, so testers can get on with doing their job – testing.

But is this really what is required? The reporting, the stats, the work allocation, the test strategy work and did I mention the meetings? It’s all STUFF. There’s always STUFF to do as a test manager. If we don’t have STUFF to do, we go out and look for STUFF. We roll our eyes at all the STUFF we have to do, but secretly we like it. It makes us feel important and keeps us busy.

Recently, I did a peer review with my team. I asked them to anonymously review my work. One question I asked them was “what I should start doing more of?”.

The resounding signal was loud and clear.  They wanted me to increase the amount of coaching and training within the team.

So I stopped doing STUFF. I dropped meetings that were optional, I’ve distributed recruitment and I worry less about planning for ‘the future’.

Instead I test and I encourage other testers to pair with me as I test, or to invite me over to test with them.

That way I’m sharing my knowledge and expertise.

It’s been hard to let go the high level STUFF but in some ways, these management type activities can be performed by most managers. Very few people can train and coach testers.

How’s your S/N ratio going?