https://pbs.twimg.com/media/C5xvGA9VAAAtxPy.jpg

Test Leadership is here to stay

In his article “What leaders do”, J.P Kotter makes the following distinction between management and leadership:

Managers promote stability, leaders press for change

For example:

  • Management involves planning and budgeting. Leadership involves setting direction.
  • Management involves organizing and staffing. Leadership involves aligning people.
  • Management provides control and solves problems. Leadership provides motivation.

The left hand side sounds a whole lot like what test managers traditionally do or have done in the past.

But, as organisations move to a more agile methodology tasks associated with these responsibilities are becoming redundant.

For example:

No more Test Team

Testers frequently sit within an agile  team alongside UX people, a product manager, developers and operations. Often they report to a senior person within the team. Large independent test teams no longer exist. The need for a test manager from a people management perspective is no longer required.

No more Big Test Strategy

Small agile teams working on small stories reduce the size and complexity of work involved. Any decisions on testing are typically made at a team level as autonomy and decision making is pushed down into the teams. There’s little need for a formal test strategy for work at a team level, and little need for a test manager to develop a large test plan or test strategy that outlines testing and resources required over an extended period of time.

No more Formal Test Process

You have 10 days to think and complete testing for a story. If you’re lucky, five days of that may be actually testing. Who the hell is going to worry about writing test cases and a formal test report? The nearest you get to a formal test process is a Jira Workflow. So bye bye formal test process. Bye Bye Test Manager to enforce test process.

No more Formal Testing Metrics

Metrics are becoming more team focused. Teams are using metrics such as MTTR (Mean Time to Recover), or LTTR (Lead Time to Deployment). Interesting ideas, and I’m glad to see a shift perceptions of quality. Again though, we don’t specifically need a test manager to report on these metrics.

No more Gatekeeping

Testing has traditionally (and unfairly) been the Gatekeeper to ensuring quality.  The business sees the testing department as a method of control. I suspect the entire testing community is heaving a quiet sigh of relief at the demise of this one!  Ensuring quality is a fool’s game, as there is no way you can win. It’s akin to forcing someone to sign a pre-nup to ensure love.  Good luck with that.

No more Test Managers then?

Yes and No. The role of test manager as we know it is dissapearing. There’s evidence of the demise of these roles in many of the large enterprise organisations I work with.

But while there’s less need for test managers that doesn’t mean that the thorny challenge of testing has disappeared. And while deploying in bitesized pieces reduces risk, it doesn’t it remove entirely. Often the risk is displaced to somewhere we are not used to investigating.

Something AWS discovered much to their dismay.


Some of the testing challenges many companies are facing:

  1. How do we test large scale systems and architectures (think Microservices)
  2. How can testing be an activity that all perform?
  3. How can we develop the testing capability within teams?
  4. How can we get better diversify our thinking in our testing?
  5. How can we think about testability as part of systems design and architecture?
  6. What part do testers and testing play within our teams?
  7. What skillsets do I need testers to have? Is it just one type of skillset?
  8. How can you test in production?
  9. How can we better understand and provide information on quality?
  10. How can we better respond to change, that is either in and out of our control?

 

Let me explain that final point.

Software Testing doesn’t have a great reputation for being flexible and adaptive to change. The way we test is deterministic. Think gated process and scripted test cases. But then neither is Test automation.

Like it or not, change happens. Sure, the extent to which change happens and the nature of that change may depend on the industry and technology you work with. What becomes important though, is how we handle that change. And, how we equip our teams with the ability to adapt to change becomes the task of a test leader.

We need confident self possessed software engineers, equipped with the ability to think critically through whatever challenge crosses their path. We need to help them work together to solve whatever testing problem comes their way. This requires a positive and safe environment where teams become ‘test-infected’ and have a culture of continuously thinking about improving their testing.

Test Leaders provide this breathing space for a testing culture to grow. They achieve this by placing an emphasis on vision, strategy and motivation and alignment. Examples of this are coaching, training, knowledge sharing and making information visible to the rest of the organisation.

A testing culture that provides this space itself generates test leaders. That’s important, because to some degree every tester is an advocate of quality and that in itself requires a degree of test leadership.

The skill set required in test leadership is different to test management. It’s more aligned to coaching, connecting people and driving a shared vision of quality for your company. It requires revisiting ideas on why we test and what software testing actually is and how we can continue to deliver value in a different context.

But I don’t see uncertainty going away, in fact I only see it increasing. And so, neither is test leadership.

Mount Everest

The Base Camp Heuristic

The ‘Base Camp’ heuristic is the work required to find any bug. Typically ‘Base Camp’ bugs tend to be low lying fruit and can be found quickly, especially if a tester is experienced. In the world of Daniel Kahneman, these bugs are found using Systems 1 thinking.

To discover an ‘Everest’ type bug requires Base Camp intuition plus more. For example a tester might need to deliberate over the product in detail, question the diversity of the data, the validity of the oracle(s)under use and the procedure required for testing.It may mean you need to create new tools, or find new ways to track the unknown.

That means you’re going to need mental training to extend the boundaries of your thinking, determination to continue when the easy answers stop flowing and some tactics to help you refocus and come up with new solutions when you run out of ideas.

You may feel dismayed because everyone around you is finding bugs at a rapid pace. You may even encounter skepticism because you appear to be showing no value. But Everest quality bugs are not meant to be easy to find. They require stamina, commitment and courage.

It’s rare to find bugs of this calibre without training and serious commitment.

Maybe I’m not going to climb Mount Everest every day, abut as an aspiration I’d like at least some of my bugs to be of the Everest calibre.

How about you?

Agile Australia and Intent based Leadership

Today I went to Agile Australia as an attendee. That’s a first for me, usually, I’m speaking. It was wonderful to go with a different purpose, which is to learn from others. Typically, if I’m speaking I’m so focused on ‘my talk’ and getting ready for ‘my talk’ that I don’t feel very open to learning new ideas.


I’m glad I went, because I had the opportunity to listen to David Marquet’s (nuclear submarine commander) talk about the abdicating control (not responsibility) to the people who worked for him. In a military organisation I can’t imagine the mindset shift & the struggle his crew must have felt!


There were many aspects of the talk I enjoyed, how subtle changes in  language was used to help people think differently. For example instead of his crew requesting permission, they informed authority by using the phrase “I intend”. He emphasised the need for constant training and drills to help keep people prepared for when the inevitable mistakes were made. I also liked the concept of pushing ‘authority down’, allowing his crew to control their decision making.


The biggest resonator though was his perspective that you can’t change people, all you can do is provide an environment where people can change themselves. After all “When a flower doesn’t bloom, you fix the environment in which it grows, not the flower.” I have this ideology too in building & developing test teams. In 2013 I spoke at Tasting Let’s Test about building a context driven test team. In it, I talk about creating a habitat where testers can thrive and grow, with a focus on coaching to support that work. I’m working hard to create a habitat like that in Tyro Payments.


The underlying belief he had was that “people are fine the way they are stop trying to fix them” struck deeply with me. If you start with that premise you come to coaching with a different mindset. And just like I came to Agile Australia with a mindset of learning this year as opposed to speaking, I can approach coaching with intent to listen instead of teach. Instead of focusing on where a person needs to be, you think of were a person is. The spotlight is placed on the person not your perception of that person.  Perhaps it’s subtle differences like this that can make all the difference in coaching. Or maybe not. I don’t know. It’s an experiment, but one I’m performing on myself rather than on other people.


And I think that’s exactly how we all like to learn.

Nice girls don’t get the Keynote

Keynotes are something to be earned and rightly so. Typically a keynote comes from someone who likes to speak, who is good at speaking, and has something interesting to talk about.

I know many women testers who meet the above criteria. They have the requisite ‘merit’ required to be a keynoter. They have thoughtful ideas, are good speakers and are well respected in the testing community.

So why are they not keynoting?

Could it be that to be a keynote takes more than ‘merit’?  Could it be that keynoting is also about your place in the community and your connections within that community?

My experience as a conference organiser and being on program committees is that ‘who you know’ is equally as important as ‘what they know and how they speak’. I don’t apologise for that. It’s important for me to base my decisions not only on an abstract but on a speaker’s reputation.

A keynoter needs to have some of the following:

1) They have interesting topics to talk about
2) They are practiced/skilled at speaking
3) They are involved in some way in the community
4) They network within and outside of their ‘tribe’
5) They make a point of asking for keynotes

I know many practised women speakers with fresh and innovative talks and we are well covered on points one and two.

And we’re not so bad at organising and volunteering for conferences & community either, so….

Could it be, points four and five is what’s holding us back?

Keynotes are typically by invite. That means, the program committee gets to decide who keynotes. Those on the program committee tend to have a good network. It’s probably one of the reasons they’ve been chosen to be on the program.

How are your networking skills?  I’m not talking about card swapping nonsense, but ask yourself: do you have a genuine interest in engaging with people who you respect? Importantly, do you know your network will openly advocate on your behalf? Which brings me to point number five…

Be open to asking to keynote. Is it just me, or is there’s this weird unwritten rule in our community that prevents us asking to keynote? It seems to me that to ‘make it’ we have to Marlene Dietrich like, sit nonchalantly in the corner waiting to be asked for our moment in the spotlight.

Maybe that’s just me and not a general experience. But for those of us who are less direct I’d like to suggest you make it clear to your close network that you want to keynote?

What’s more, go direct to the program committee and ask them for a spot. If the say no, ask why not? and what does it take to keynote,? Who knows you might gain some valuable insights into the process.

Because no matter what skill you have, or how good you are at speaking, or how charismatic you are, there’s dozens of great speakers who can do what you do. It takes more than merit to get noticed. It takes courage to ask and you need allies who will support you along the way. Have you got what it takes?

If you’re reading this post, and you’ve keynoted in the past with a story to share about how you got to keynote, Speak Easy would love to hear it as it may help us understand what it takes to ‘get there’. I’d post a link to your blog post on the Speak Easy website (or I can post a blog).

*Title misappropriated from a book I found useful on this topic “nice girls don’t get the corner office’

Paying Attention to Paying Attention to Detail

One question I often ask a tester in an interview is what they believe are the essential skills that differentiate a good tester from a great tester. One of the common replies is “Paying attention to detail”. But what does that really mean?

Start with detail. What do you mean by detail? Mrs Google offers the explanation “an individual fact or item”, but what sort of detail? Visual detail? Detail from a coding perspective? Any detail or only detail that matter? Is it detail as in ‘small stuff’ as in the Devil is in the detail?

Also what exactly does “paying attention” mean?

Maybe it’s one of these interpretations:

  • A tester is attentive as in a teacher saying “pay attention!”? As in be alert?
  • A tester is only observing detail?
  • A tester is observing detail exists and then noticing they are problems?
  • A tester is noticing detail without a conscious attempt to observe in a methodical manner?

Going deeper, why ‘pay attention to detail’ in the first place?  Is it because you’ve been told to, or because the tester genuinely wants to? Your motivation and purpose play important roles in bug finding. For example what’s your ‘attention to detail’ on being told to complete 40 test scripts in one day, compared to your attention to detail if you get made handsomely for finding a bug or winning a prestigious competition?

Yes, paying attention to detail is important, but so is understanding what you are doing when ‘paying attention to detail’.

In my book, there’s not enough meat on that answer to figure out if someone’s a good fit for my team. I want more. I want my testers to suck the marrow out of that question, to boil it until the flesh falls away from the bone.

Professional testers¹ consciously observes systems using a multiple of models. Their maxim is “the more you see, the more you see”. And that’s only the beginning. Following observation comes evaluation. Now testers get to interrogate information by asking (either explicitly or implicitly) is there a problem? Not content with relying on one oracle they may consciously use many different oracles in their evaluation process.

They then report this information in a way that’s meaningful to the people who want to hear it.

So if you are a  tester coming for an interview on my team and I ask you this question, please have fun with it. Show me your testing process, demonstrate it with diagrams and models and show me a bit of your testing flair. All testers pay attention to detail to some degree. What in particular do you pay attention to?

¹ by professional tester I mean someone who makes an attempt to understand and then improve their personal testing process. They tend to develop a unique approach and style to their testing through years doing, thinking, talking & writing about testing.

This post is a result of a tweet I made which many testers. An interesting aside, my poor spelling was explained away by many as a purposeful trap designed to snare the unwary tester. It wasn’t, though I’m flattered that you would think that of me.

Bobbing for Embedded Apples

One of my first recollections of physics is the image of Archimedes running down the street, dripping wet from his bath and shouting: “Eureka I found it!” on discovering his principle of fluid displacement to measure the force.

Add to that Isaac Newton who quietly sat under an apple tree, musing the day away until he discovered his laws of motion, one which states;

For every action there is an equal and opposite reaction..

Happy days…

Many companies want to improve their testing, particularly if that improvement is translated into reduced time or cost.

To achieve this, there’s a big trend to embed testers into development teams. There’s no longer ‘us’ and ‘them’ but an harmonious ‘we’. The wall of independence is torn down replaced by a gleaming whiteboard emblazoned with the banner ‘Quality is a team responsibility’.

The reality though is that just like Archimedes in his bath, or Newton and his Apple, embedding testers into a team of developers is going to create displacement and cause a reaction.

This is common sense. Put one tester into a team of multiple developers and it’s going to change the team dynamics. Add to the mix, that the tester will most likely become a point of constraint disrupting what was perhaps a seamless flow of stories into the Done column and it’s no wonder the apple cart gets upset.

For example, developers discover they need to get more involved in testing and decision making.That can be a big mindset change if traditionally developers are used to handing work over to testers (as opposed to receiving it from them).

Testers need to change and adapt too. They have to let go control over all the testing and entrust the developers to help them test. Again big mindset change for testers who traditionally rely on empirical evidence than a developers word to evaluate quality.

Why then, do we seemed surprised when we face resistance and/or confusion when these changes are attempted?

This is our brave new world. I wonder where we will all end up?  Wether a team develops a true and meaningful relationship or simply an uneasy alliance is I guess up to how much both testers and developers are prepared to appreciate each other’s challenges.

One way we can help each other out is to be supportive as opposed to defensive when people face the impact of changes and perhaps feel displaced and confused. How much we are prepared to let go our assumptions and prejudices about each other will also play a big part in how ‘successful’ these experiments are deemed to be.

Exciting times indeed!

 

How’s your signal to noise ratio going?

It’s not only engineers that need to deal with S/N (Signal to Noise Ratio). 

Test Managers have to deal with lots of noise, in managing projects, dealing with stakeholders, identifying potential future risk, maintaining test environments, hiring testers, and meetings,meetings & meetings. Traditionally that’s what the role has involved. We deal with the noise, so testers can get on with doing their job – testing.

But is this really what is required? The reporting, the stats, the work allocation, the test strategy work and did I mention the meetings? It’s all STUFF. There’s always STUFF to do as a test manager. If we don’t have STUFF to do, we go out and look for STUFF. We roll our eyes at all the STUFF we have to do, but secretly we like it. It makes us feel important and keeps us busy.

Recently, I did a peer review with my team. I asked them to anonymously review my work. One question I asked them was “what I should start doing more of?”.

The resounding signal was loud and clear.  They wanted me to increase the amount of coaching and training within the team.

So I stopped doing STUFF. I dropped meetings that were optional, I’ve distributed recruitment and I worry less about planning for ‘the future’.

Instead I test and I encourage other testers to pair with me as I test, or to invite me over to test with them.

That way I’m sharing my knowledge and expertise.

It’s been hard to let go the high level STUFF but in some ways, these management type activities can be performed by most managers. Very few people can train and coach testers.

How’s your S/N ratio going?

“Nobody — calls me — chicken.”

One of the morals of the Back to The Future series is be able to walk away from a fight. It took three movies for Marty McFly to learn this self control.

Biff Tannen calling Marty “chicken” in 1955.

One thing that the scriptwriters failed to predict was the birth of speaking your mind in 140 characters, aka Twitter.  If they had, it would be easy to see how Marty might have been pulled in to a twitter diatribe and feel unable to resist  responding.

Screenshot 2014-10-14 07.47.54

 

It’s easy to engage on Twitter. It’s fun and easy to throw out ideas, thoughts and its mildly useful for debate. My personal experience though is that its much harder to disengage. I  feel compelled to reply to a challenge. It’s as if each tweet directed at me is a personal call to respond (sometimes it is). As much as I want to ignore and avoid, I find myself compelled to return and <sigh> to respond. My reasoning mind tells me, desist, desist, but my ego overrides this. I must have the final word!

There comes a point in any twitter engagement when you realise the conversation is little about debate and more about chest beating. When I’ve come to this realisation, I’ve lost the battle. I’m no longer in control regardless of having the last word or not. There’s little dignity in wining such a battle.

We need to learn the art of disengagement, the ability to walk away without feeling somehow less of a person. It’s important to engage, but its also important to be able to disengage.

I’ve come up with some heuristics to help me battle the McFly syndrome.

Yes, And Heuristic
I got this from Lynne Cazeallys book Create Change. Instead of think ‘yes but’, think ‘Yes and..’. .it goes some way to appearing collaborative, perhaps reducing the antagonism in the twitter conversation.

How about this? Heuristic
Another Lynne Cazeally suggestion. It uses a lot of characters but you can apply the sentiment to help sound more congenial.

Rule of Three Heuristic
Sometimes, attempting to understand other reasons why someone might write something helps you to walk away. Taken from Jerry Weinberg’s rule of three.

Out to Lunch Heuristic
I’ve seen people use this occasionally where they excuse the challenge to go out and walk the dog, feed the kids etc. I haven’t tried it myself but its a method of walking away with some grace. Even better if the reason is valid.

Blank Wall
Simply don’t respond. Oh to have the will power to apply this, especially in the heat of battle!

Blocking Heuristic.
A crude yet highly effective way of not knowing is someone is responding to you or not. You are now blissfully unaware. The downside is that you will never hear anything that this person has to say. You may consider this a good thing.

Turn off Twitter 
The ultimate solution and something a few people have resorted to. Not only does twitter seem to bring out the worst in people it’s incredibly time consuming. Ask yourself do you really need it?

What tactics do you use to help you disengage from a twitter war?

Take the ‘Crappy Work’ Litmus Test

We’ve placed a ban on crappy work within my test team. From now on, we declare that the testing team will refuse to do crappy work.

The challenge is in knowing how your doing good work or crappy work?  This is why I’ve created this simple 5 minute ‘Crappy Work’™ ² test to help any tester to figure out if you’re doing crappy work or not. ¹

1) Do I fully understand what I’ve been asked to do?

2) Why am I doing this task?

3) Who will benefit, and I have spoken to them ?

4) Am I doing this task simply because its part of the process?

5) If I think it’s crappy work, how can I make it valuable?

I’m sure there are plenty more ‘crappy work’  questions that would be useful to add, but my 5 minutes is up.

Why not share your ‘crappy work’ litmus tests? 

¹ Maverick Tester takes no responsibility if you take this quiz and continue to do crappy work

² It’s a 5 minute quiz because it took me 5 minutes to think it up. It’s trademarked so I can sell you a certificate in passing the ‘Crappy Work’ test, thereby making a whole lot of money out of it.

My Testing Manifesto 2014

It’s my first day in permanent employment for, oh, about 20 years. I feel a little giddy, like someone’s first day of school – nervous but excited.

Unlike my first day of school, I have some clear goals and ideas I want to implement. I thought I might outline them here to help remind me of what I want to achieve and also to see how different it turns out.

Testing Philosophies

Testing is an skilled activity(not a phase) that all, to some degree can acquire.
Testers need autonomy to make decisions, to develop and perform excellent testing.
Quality is something we all care about, though it means different things to different people.
Every test has a cost (design, building, maintaining, reporting)

Goals for Testing

To develop a company wide reputation for excellent testing
To develop a test team that is able to handle testing problems with courage, skill and humility.
To coach and help develop the skill of testing with whoever may need it
To identify where testing is occurring and help augment that
To develop and the grow the test team in size, skill and expertise.
To engage with and support the testing community

Personal Goals

To acquire knowledge in testing within a continuous deployment, delivery environment
To learn more about functional programming
To be able to identify how to repair code
To work with integrity and within the bounds of what I consider ethical