https://pbs.twimg.com/media/C5xvGA9VAAAtxPy.jpg
by Anne-Marie

Test Leadership is here to stay

Test Leadership is so important is because it’s needed to help testing be responsive to change. The complexity and level of uncertainty within which we now test means we can’t hold rigidly to doctrine or external process.

In his article “What leaders do”, J.P Kotter makes the following distinction between management and leadership:

Managers promote stability, leaders press for change

For example:

  • Management involves planning and budgeting. Leadership involves setting direction.
  • Management involves organizing and staffing. Leadership involves aligning people.
  • Management provides control and solves problems. Leadership provides motivation.

The left hand side sounds a whole lot like what test managers traditionally do or have done in the past.

But, as organisations move to a more agile methodology tasks associated with these responsibilities are becoming redundant.

For example:

No more Test Team

Testers frequently sit within an agile  team alongside UX people, a product manager, developers and operations. Often they report to a senior person within the team. Large independent test teams no longer exist. The need for a test manager from a people management perspective is no longer required.

No more Big Test Strategy

Small agile teams working on small stories reduce the size and complexity of work involved. Any decisions on testing are typically made at a team level as autonomy and decision making is pushed down into the teams. There’s little need for a formal test strategy for work at a team level, and little need for a test manager to develop a large test plan or test strategy that outlines testing and resources required over an extended period of time.

No more Formal Test Process

You have 10 days to think and complete testing for a story. If you’re lucky, five days of that may be actually testing. Who the hell is going to worry about writing test cases and a formal test report? The nearest you get to a formal test process is a Jira Workflow. So bye bye formal test process. Bye Bye Test Manager to enforce test process.

No more Formal Testing Metrics

Metrics are becoming more team focused. Teams are using metrics such as MTTR (Mean Time to Recover), or LTTR (Lead Time to Deployment). Interesting ideas, and I’m glad to see a shift perceptions of quality. Again though, we don’t specifically need a test manager to report on these metrics.

No more Gatekeeping

Testing has traditionally (and unfairly) been the Gatekeeper to ensuring quality.  The business sees the testing department as a method of control. I suspect the entire testing community is heaving a quiet sigh of relief at the demise of this one!  Ensuring quality is a fool’s game, as there is no way you can win. It’s akin to forcing someone to sign a pre-nup to ensure love.  Good luck with that.

No more Test Managers then?

Yes and No. The role of test manager as we know it is dissapearing. There’s evidence of the demise of these roles in many of the large enterprise organisations I work with.

But while there’s less need for test managers that doesn’t mean that the thorny challenge of testing has disappeared. And while deploying in bitesized pieces reduces risk, it doesn’t it remove entirely. Often the risk is displaced to somewhere we are not used to investigating.

Something AWS discovered much to their dismay.


Some of the testing challenges many companies are facing:

  1. How do we test large scale systems and architectures (think Microservices)
  2. How can testing be an activity that all perform?
  3. How can we develop the testing capability within teams?
  4. How can we get better diversify our thinking in our testing?
  5. How can we think about testability as part of systems design and architecture?
  6. What part do testers and testing play within our teams?
  7. What skillsets do I need testers to have? Is it just one type of skillset?
  8. How can you test in production?
  9. How can we better understand and provide information on quality?
  10. How can we better respond to change, that is either in and out of our control?

 

Let me explain that final point.

Software Testing doesn’t have a great reputation for being flexible and adaptive to change. The way we test is deterministic. Think gated process and scripted test cases. But then neither is Test automation.

Like it or not, change happens. Sure, the extent to which change happens and the nature of that change may depend on the industry and technology you work with. What becomes important though, is how we handle that change. And, how we equip our teams with the ability to adapt to change becomes the task of a test leader.

We need confident self possessed software engineers, equipped with the ability to think critically through whatever challenge crosses their path. We need to help them work together to solve whatever testing problem comes their way. This requires a positive and safe environment where teams become ‘test-infected’ and have a culture of continuously thinking about improving their testing.

Test Leaders provide this breathing space for a testing culture to grow. They achieve this by placing an emphasis on vision, strategy and motivation and alignment. Examples of this are coaching, training, knowledge sharing and making information visible to the rest of the organisation.

A testing culture that provides this space itself generates test leaders. That’s important, because to some degree every tester is an advocate of quality and that in itself requires a degree of test leadership.

The skill set required in test leadership is different to test management. It’s more aligned to coaching, connecting people and driving a shared vision of quality for your company. It requires revisiting ideas on why we test and what software testing actually is and how we can continue to deliver value in a different context.

But I don’t see uncertainty going away, in fact I only see it increasing. And so, neither is test leadership.

Quality Workshop
by Anne-Marie

Quality Workshop

How do you help a team to agree on a working definition of quality? I developed this quality workshop for this purpose, but also to help them apply it in a concrete way.

One sure way to liven up a meeting is to ask attendees for a definition of quality. My experience has been that this is most entertaining particularly if you have both devops and testing in the room.

It turns out that many people feel very passionate about Quality. It also turns out that many have different ideas on what quality actually is. Who knew? Diversity of ideas are not bad but it can impact a teams understanding of quality. The consequence is that it could possibly hinder a team’s ability to delivery quality product.

To help people understand the challenges around quality and to try clarify what quality might be in their context, I developed the following quality workshop.

Purpose of Quality Workshop

a) to get a working definition of quality that most people are comfortable
b) to outline a manifestation of quality in terms of quality attributes
c) to identify ways in which we can these quality attributes can be acted on

Who should attend

Depends on what your trying to achieve, but if you’re a team delivering product, I would suggest your key stakeholders come along. That may mean product owner, developers, testers, Ops. If you can get your business there too that’s a big plus.

Tools for Quality Workshop

  • A set of Quality Attributes (I’ve known these also to be called Quality Characteristics, or CFR’s – Cross Functional Requirements) printed out and placed on a wall.
  • PostItNotes (of course!)
  • some pens.

Process for Quality Workshop

  1. Ask people to write down on a post-it note their understanding of quality
  2. Place the definitions on the wall and ask people to read them (or read them out)
  3. Note the diversity (if it exists, it typically does). Bring up the subjectivity of quality and how difficult it is to agree on and quantify.
  4. Ask what the implications of this might be throughout the delivery lifecycle (e.g Design, Development, Ops)
  5. You can dot vote to try and get some sort of agreement on a working definition of quality*.

* My working definition of quality is the Jerry Weinberg one “Quality is value to some person”.  If no-one puts this definition up on the board, I like to add it as it encourages a discussion on the subjectivity of quality. 

If you’ve got this far without some major outburst or spat, congratulations! Next step is to try and parse what quality might look like.

  1. Ask people to read the quality attributes on the wall
  2. Ask them to dot vote 3 what they consider to be the most important quality attributes
  3. Collect the top 3 quality attributes and put them on a whiteboard
  4. Ask the group to identify 3 tasks for each particular quality attribute and to write them on a post-it note.*
  5. Place the post it notes on the whiteboard along side the quality attribute
  6. Run through the post-it notes. Discuss the viability of these and how to make these tasks happen.
  7. Visualise and share this information to the broader organisation.

* I avoid using the word ‘testing’ as I want people to think about other factors that influence quality. Testing is important, but testing helps us evaluate quality.  Unless its acted upon, it doesn’t help us build quality. Sometimes I like to prime people by providing a couple of non testing examples such as Code Reviews,  Monitoring and Security by Design. 

I’ve found these sessions generate a lot of discussion and can become quite heated.

I wouldn’t run a quality workshop all the time, but its really useful iron out at perhaps an epic or feature level, what quality might be and how a team can work together to implement that vision.

(Learn more on this topic by attending my Quality Matters Workshop)

Let me know how you get on!

Photo Attribute:  Quality by jason Taellious https://www.flickr.com/photos/dreamsjung/

by Anne-Marie

What Teenagers and MVP have in common

Teenage children can be the classic MVP’s (Minimal Viable Products) in action. Here’s why:

  1. They tend to be  self deterministic
  2. They tend to experiment constantly
  3. They don’t attempt to predict the future but learn from their experiments
  4. They typically have no endstate in mind that they wish to be made explicit
  5. They have courage to outline their path determined by their feedback
  6. They have faith in the outcome even if the outcome is unknown

Ok, so I made a lot of that up. But hear me out…

Teenagers can the quintessential examples of MVP’s in progress. If you have one in your house and you want to understand more about MVP.  Observe and Learn from them. They can teach you wonderful insights into known, knowns and unknown, unknowns. Even more, you will have insight to the  ‘knowns I wish I could now be unknown’. Something I’m not aware the Cynefin model caters for?

As a finale, here’s a voyeuristic view into what my teenagers do right now…

A post shared by @nikolai.charrett on

— love your work boys…

Update: Since I wrote this, I’ve thought a bit deeper on the topic. To some degree we are all MVP’s regardless of age. To some degree we are all completed products, but also we’re often minimum versions of the end state. At least I am. I’m still trying to figure out who I want to be, what I might want to do, and what that might look like.

building the quality inn
by Anne-Marie

Building the Quality Inn

Building Quality In integral part of moving to a DevOps culture, at least according to the State of DevOps 2016. But what exactly does that mean?

Sure, the report cites Deming, in particular

 “Cease dependence on inspection to achieve quality. Eliminate the need for inspection on a mass basis by building quality into the product in the first place.”

And while the concept is not new, and seems to make sense at least at an ideological point of view, what in practice would it look like? This was what Matthew Santon Rutherford, the Quality Guild Doctor at IAG asked me.

He also mentioned that when people spoke of ‘Building Quality In’ it immediately conjured up this image of a run down motel somewhere remote, with a pink flamingo sign, swaying slightly in the wind.  We laughed and played around with this image for a while, adding an empty swimming pool and trying to imagine what each room might look like and who its customers might be.

I dedicate this series of “Building Quality INN” blog posts to Matthew, who has forever burned this image in my mind. What would your version of Quality Inn look like? Do you have a picture that might represent it? Please share !

16-truman.w1200.h630
by Anne-Marie

Nuance on leaving testing to the experts

Non testers will always have views and opinions on testing. But it’s a problem when those opinions start impacting my testing in a negative way. Then I’m failing in my responsibility to perform testing to my best ability. The intent of control is to do a job well, not to hold power over people.

Yesterday I wrote a blog post entitled “Leave Testing to the Experts” to which Maaret quickly reacted online and by a post where she strongly disagreed with the sentiment.  Disagreement is a very useful tool to clarify your thinking, so thanks Maaret!.

It’s allowed me the opportunity to explain in more detail, why I think what I do, and to investigate some of the nuances on this topic.

Maaret, counter-argued with a convincing description of her egalitarian work place where opinions are offered and accepted in good faith. In truth it left me feeling a little envious, and a commitment to work harder on taking ‘being told’ as “bad communication with good intent”.

But I think this post misses a vital point and that is responsibility.

Taking on a role³ of tester comes with a responsibility to have testing performed to your best ability. So, if I listen to an opinion and follow it, knowing its wrong, I call that irresponsible. If I hire a tester based on likability over capability, that’s irresponsible. If I fail to highlight risk for fear of upsetting team members? Well, you get the point¹.
Perhaps you might say that testing is a collective responsibility, and we all own the decision. After all, in an egalitarian team such as Maarets, doesn’t everyone own quality? Isn’t everyone responsible for testing? Well, yes and no. Let’s explore that a bit by taking the pilot example.
It takes a crew to fly a large plane. There’s the engineers, the pilots, the cabin crew and many more. The pilot happily takes on other crew member’s opinions and ideas. Perhaps she listens to the engineer’s advice not to fly above a certain height. Or maybe she nods when a cabin crew member tells her that people like to hear from her voice more often. Perhaps her junior suggests a flight plan for the journey. She hears all this and takes it into consideration, but flying the plane is ultimately her responsibility². Even if part of that decision is to allow the junior to fly the plane for some of the journey. The buck stops with the pilot.
The crew is responsible for the whole flight experience, but the pilot is responsible for flying the plane.
Similarly, in a team, we’re all responsible for quality, each person may even be responsible for their own bit of testing but ultimately the testing buck stops with the tester.
Notice that everyone got to have an opinion? Similarly on a team, many people can perform testing and offer ideas on testing. This can be incredibly invaluable to testing as a whole. For example, developers have a keen insight into technical risk which can really help identify/dismiss areas that require testing. Teams that are test infected and have this testing mindset are often a joy to work with.
You don’t get a testing mindset if you’re prevented from exploring ideas, so I agree with Maaret that people should be allowed to offer opinions and experiment.
I worked with a team who wanted to implement BDD. I was skeptical. BDD seemed to me a bad fit for the context in which we were developing. There’s nothing like self discovery though and so the team went ahead and tried it out. It turned out to be a dud idea and it was dropped. It’s good to experiment and encourage exploration but there needs to be a proviso here and that is when the risk is low and the impact to the company is minimal.  
Back to the pilot. You don’t handover control of a plane to a junior pilot just as turbulent weather conditions hit. You might if they have more experience in flying, but as the senior pilot you will know when to take the wheel and when to allow practise. Someone who knows what they’re doing needs to make these decisions.
Also performing a small experiment when the risk is low is very different to adopting a strategy company wide. Dictating cucumber as a tool across an organisation is very different to performing an experiment within a team. To clarify, my lament on testing experts is in the context of organisations adopting ‘best practise’ strategies.
Maaret talks about everyone on her team ‘tests like an expert’. I think that’s great, her very capable team means she doesn’t have to exercise responsibility. She’s still responsible for the testing though, just as carrying a plane full of pilots doesn’t absolve the actual pilot of responsibility.
I’ll end this post with the comment about sales people. Maaret says she constantly offer views to developers and sales people. The truth is I have done that too. I guess everyone feels in some way their idea is good and has value and its nice to have your opinions heard.
I remember once querying a company’s sales strategy. I couldn’t understand why we only aligned with partners instead of also focusing on location. I brought this up with the sales manager who listened and then dismissed it explaining why. Personally, I still don’t agree with the approach, but I’m fine with his decision. Why? Because ultimately the buck stops with him. I’m cool with that.
Non testers will always have views and opinions on testing. That’s not a bad thing. In fact, part of me is glad they feel responsibility about testing. But it’s a problem when those opinions start impacting my testing in a negative way. Then I’m failing in my responsibility to perform testing to my best ability. You see, the intent of ‘claiming territory’ is to do a job well, the job I was hired to do, not to hold power over people through control.
I hope this makes sense. Like I said, I’m grateful to Maaret for her response and the opportunity to deepen my ideas on the topic.
Footnotes
³I considered adding the word specialist here as the concept still holds. A specialist (of testing) still holds the ultimate responsibility for testing, even if they don’t perceive themselves as a tester.
¹Many of these decisions are nuanced and are rarely black and white. For example, hiring a tester for a team fit is really important, but if you do so over capability you need to be able to back your decision with some sound logic. Context is important here.
²many people inside and outside a team test, so a tester isn’t responsible for doing all the testing, but they are responsible for the strategic direction and process of testing.
by Anne-Marie

Leave the testing to the experts!

The truth is, most developers, project managers and business people don’t understand testing in any deep way. Instead they often resort to shallow imitations, an emperors test suite, if you will.

Why is it that in a company almost everyone has an opinion and knows the best way to test? What makes so many of these people feel they know how testing ought to be performed?

Business people tell me, the tester:

“You should be writing test cases and reporting against them”

Developers & technical people tell me to:

” automate the lot using cucumber/PACT/Selenium or some other niche technical tool” 

Test Managers (many who have never tested) tell me:

“You should be following a formalised signed-off testing process”

I find this quite extraordinary. I don’t tell you, oh developer how to code your program. I don’t tell you oh, sales person, how to sell your product.

So why do you think its reasonable and perfectly acceptable to tell me how to test software? 

Perhaps its because we all test to some degree. But just because I can build a car or sell one doesn’t mean I can drive it. Just because I know how to drive a car, doesn’t mean I go and tell a rally driver how to race around a track.

The truth is, most developers, project managers and business people don’t understand testing in any deep way. Instead they often resort to shallow imitations, an emperors test suite, if you will.  That’s partly the fault of the testing profession. We’ve failed to claim our turf and instead tugged our forelock, deferring ideology to those with authority but little understanding.

It’s time tester’s to claim some turf back. I suspect (actually I know) for many places it won’t be given up without a struggle. You’re going to need courage backed by confidence in your ability if you want to claim some of the territory. But it’s worth it. With space comes freedom and the autonomy to drive your testing strategy and process in a way that you know will offer value to your organisation and your clients.

Now, please get out of my way and leave the testing to the experts!

Mount Everest
by Anne-Marie

The Base Camp Heuristic

The ‘Base Camp’ heuristic is the work required to find any bug. Typically ‘Base Camp’ bugs tend to be low lying fruit and can be found quickly, especially if a tester is experienced. In the world of Daniel Kahneman, these bugs are found using Systems 1 thinking.

To discover an ‘Everest’ type bug requires Base Camp intuition plus more. For example a tester might need to deliberate over the product in detail, question the diversity of the data, the validity of the oracle(s)under use and the procedure required for testing.It may mean you need to create new tools, or find new ways to track the unknown.

That means you’re going to need mental training to extend the boundaries of your thinking, determination to continue when the easy answers stop flowing and some tactics to help you refocus and come up with new solutions when you run out of ideas.

You may feel dismayed because everyone around you is finding bugs at a rapid pace. You may even encounter skepticism because you appear to be showing no value. But Everest quality bugs are not meant to be easy to find. They require stamina, commitment and courage.

It’s rare to find bugs of this calibre without training and serious commitment.

Maybe I’m not going to climb Mount Everest every day, abut as an aspiration I’d like at least some of my bugs to be of the Everest calibre.

How about you?

by Anne-Marie

Testing Microservices – what do you want to know?

Want to hear about testing microservices, take the short poll!

The following is an outline of blog post on Testing Microservices. Does it sound interesting? Vote below to get me to write it or focus on something else :)

Testing Microservices

1) Microservices is a concept a set of patterns, not a set architecture
2) The context in which you testing has a direct impact on how you test
3) Risk changes, allow your strategy to change with it
4) Diversify your testing strategy

If you have a question you want me to answer on testing microservices, please add it in the comments.

by Anne-Marie

No women speakers? No Bother!

That might surprise you if you listened the Eurostar Webinar last week where Fiona Charles and I explored the numbers of women speaking at software testing conferences. If you didn’t hear it you can now  (Spoiler ahead it’s roughly 25%*)

FemaleConferenceSpeakers

 

Considering that there are more women in testing than other technical professions you might be surprised by this.

Unfortunately I’m not surprised. Fortunately, I’m not bothered by this. In fact, I’m not even bothered that in 2015 the percentage of women speakers has dropped in some conferences, or that the number of women on program committees for many conferences is a big fat zero.

Percentage of women on program committees

I’m disappointed of course, but I’m not bothered by it.

That’s because there are some phenomenal women in software testing and they’re taking the matter into their own hands.

Rosie Sherry working with Anna Royzman for example, has dropped the teaser that women makeup ~50% of TestBashNY 2015 using a merit based selection process. They had a lot of women submitting proposals and I think it’s clear why. What female tester wouldn’t want to speak at a conference run by those two phenomenal and respected women?

There’s more though:

Maaret Pyhäjärvi & Adi Bolboacă have decided to create a conference that they would want to attend. It’s called the European Testing Conference 

Mieke Gevers & Nadine Raes run Belgium Testing Days and there’s typically a high percentage of women speakers (30% this year).

I’m sure there are other examples too.

It’s simple folks. When you create an culture where women feel welcome to speak, the submissions come flooding in.  What does that mean? Well, perhaps women no longer need to be concerned about being underrepresented at dinosaur conferences. Instead, women can focus on conferences that are already offering a healthy environment. Conferences were women are compelled to submit.

I suggest that for any conference in software testing, if the trend of  women speakers is decreasing or wildly fluctuating, if the percentage is consistently below 25%, then conference organisers need to rethink how they are attracting talent.

FemaleSpeakersByConferenceTime

In this day and age, I think there is little excuse for poor female representation. Conferences such as CAST have demonstrated it can be done. CAST has high calibre talks and a high percentage of female speakers. Who would have thought?

How about you, do you think conferences organisers need to rethink how they attract talent?

*figures were taken based on information on websites. We may have made errors in counting, but we think they are fairly accurate representation. Please let us know if we’ve made any glaring stuff ups. 

by Anne-Marie

Agile Australia and Intent based Leadership

Today I went to Agile Australia as an attendee. That’s a first for me, usually, I’m speaking. It was wonderful to go with a different purpose, which is to learn from others. Typically, if I’m speaking I’m so focused on ‘my talk’ and getting ready for ‘my talk’ that I don’t feel very open to learning new ideas.


I’m glad I went, because I had the opportunity to listen to David Marquet’s (nuclear submarine commander) talk about the abdicating control (not responsibility) to the people who worked for him. In a military organisation I can’t imagine the mindset shift & the struggle his crew must have felt!


There were many aspects of the talk I enjoyed, how subtle changes in  language was used to help people think differently. For example instead of his crew requesting permission, they informed authority by using the phrase “I intend”. He emphasised the need for constant training and drills to help keep people prepared for when the inevitable mistakes were made. I also liked the concept of pushing ‘authority down’, allowing his crew to control their decision making.


The biggest resonator though was his perspective that you can’t change people, all you can do is provide an environment where people can change themselves. After all “When a flower doesn’t bloom, you fix the environment in which it grows, not the flower.” I have this ideology too in building & developing test teams. In 2013 I spoke at Tasting Let’s Test about building a context driven test team. In it, I talk about creating a habitat where testers can thrive and grow, with a focus on coaching to support that work. I’m working hard to create a habitat like that in Tyro Payments.


The underlying belief he had was that “people are fine the way they are stop trying to fix them” struck deeply with me. If you start with that premise you come to coaching with a different mindset. And just like I came to Agile Australia with a mindset of learning this year as opposed to speaking, I can approach coaching with intent to listen instead of teach. Instead of focusing on where a person needs to be, you think of were a person is. The spotlight is placed on the person not your perception of that person.  Perhaps it’s subtle differences like this that can make all the difference in coaching. Or maybe not. I don’t know. It’s an experiment, but one I’m performing on myself rather than on other people.


And I think that’s exactly how we all like to learn.