16-truman.w1200.h630

Nuance on leaving testing to the experts

Yesterday I wrote a blog post entitled “Leave Testing to the Experts” to which Maaret quickly reacted online and by a post where she strongly disagreed with the sentiment.  Disagreement is a very useful tool to clarify your thinking, so thanks Maaret!.

It’s allowed me the opportunity to explain in more detail, why I think what I do, and to investigate some of the nuances on this topic.

Maaret, counter-argued with a convincing description of her egalitarian work place where opinions are offered and accepted in good faith. In truth it left me feeling a little envious, and a commitment to work harder on taking ‘being told’ as “bad communication with good intent”.

But I think this post misses a vital point and that is responsibility.

Taking on a role³ of tester comes with a responsibility to have testing performed to your best ability. So, if I listen to an opinion and follow it, knowing its wrong, I call that irresponsible. If I hire a tester based on likability over capability, that’s irresponsible. If I fail to highlight risk for fear of upsetting team members? Well, you get the point¹.
Perhaps you might say that testing is a collective responsibility, and we all own the decision. After all, in an egalitarian team such as Maarets, doesn’t everyone own quality? Isn’t everyone responsible for testing? Well, yes and no. Let’s explore that a bit by taking the pilot example.
It takes a crew to fly a large plane. There’s the engineers, the pilots, the cabin crew and many more. The pilot happily takes on other crew member’s opinions and ideas. Perhaps she listens to the engineer’s advice not to fly above a certain height. Or maybe she nods when a cabin crew member tells her that people like to hear from her voice more often. Perhaps her junior suggests a flight plan for the journey. She hears all this and takes it into consideration, but flying the plane is ultimately her responsibility². Even if part of that decision is to allow the junior to fly the plane for some of the journey. The buck stops with the pilot.
The crew is responsible for the whole flight experience, but the pilot is responsible for flying the plane.
Similarly, in a team, we’re all responsible for quality, each person may even be responsible for their own bit of testing but ultimately the testing buck stops with the tester.
Notice that everyone got to have an opinion? Similarly on a team, many people can perform testing and offer ideas on testing. This can be incredibly invaluable to testing as a whole. For example, developers have a keen insight into technical risk which can really help identify/dismiss areas that require testing. Teams that are test infected and have this testing mindset are often a joy to work with.
You don’t get a testing mindset if you’re prevented from exploring ideas, so I agree with Maaret that people should be allowed to offer opinions and experiment.
I worked with a team who wanted to implement BDD. I was skeptical. BDD seemed to me a bad fit for the context in which we were developing. There’s nothing like self discovery though and so the team went ahead and tried it out. It turned out to be a dud idea and it was dropped. It’s good to experiment and encourage exploration but there needs to be a proviso here and that is when the risk is low and the impact to the company is minimal.  
Back to the pilot. You don’t handover control of a plane to a junior pilot just as turbulent weather conditions hit. You might if they have more experience in flying, but as the senior pilot you will know when to take the wheel and when to allow practise. Someone who knows what they’re doing needs to make these decisions.
Also performing a small experiment when the risk is low is very different to adopting a strategy company wide. Dictating cucumber as a tool across an organisation is very different to performing an experiment within a team. To clarify, my lament on testing experts is in the context of organisations adopting ‘best practise’ strategies.
Maaret talks about everyone on her team ‘tests like an expert’. I think that’s great, her very capable team means she doesn’t have to exercise responsibility. She’s still responsible for the testing though, just as carrying a plane full of pilots doesn’t absolve the actual pilot of responsibility.
I’ll end this post with the comment about sales people. Maaret says she constantly offer views to developers and sales people. The truth is I have done that too. I guess everyone feels in some way their idea is good and has value and its nice to have your opinions heard.
I remember once querying a company’s sales strategy. I couldn’t understand why we only aligned with partners instead of also focusing on location. I brought this up with the sales manager who listened and then dismissed it explaining why. Personally, I still don’t agree with the approach, but I’m fine with his decision. Why? Because ultimately the buck stops with him. I’m cool with that.
Non testers will always have views and opinions on testing. That’s not a bad thing. In fact, part of me is glad they feel responsibility about testing. But it’s a problem when those opinions start impacting my testing in a negative way. Then I’m failing in my responsibility to perform testing to my best ability. You see, the intent of ‘claiming territory’ is to do a job well, the job I was hired to do, not to hold power over people through control.
I hope this makes sense. Like I said, I’m grateful to Maaret for her response and the opportunity to deepen my ideas on the topic.
Footnotes
³I considered adding the word specialist here as the concept still holds. A specialist (of testing) still holds the ultimate responsibility for testing, even if they don’t perceive themselves as a tester.
¹Many of these decisions are nuanced and are rarely black and white. For example, hiring a tester for a team fit is really important, but if you do so over capability you need to be able to back your decision with some sound logic. Context is important here.
²many people inside and outside a team test, so a tester isn’t responsible for doing all the testing, but they are responsible for the strategic direction and process of testing.

Agile Australia and Intent based Leadership

Today I went to Agile Australia as an attendee. That’s a first for me, usually, I’m speaking. It was wonderful to go with a different purpose, which is to learn from others. Typically, if I’m speaking I’m so focused on ‘my talk’ and getting ready for ‘my talk’ that I don’t feel very open to learning new ideas.


I’m glad I went, because I had the opportunity to listen to David Marquet’s (nuclear submarine commander) talk about the abdicating control (not responsibility) to the people who worked for him. In a military organisation I can’t imagine the mindset shift & the struggle his crew must have felt!


There were many aspects of the talk I enjoyed, how subtle changes in  language was used to help people think differently. For example instead of his crew requesting permission, they informed authority by using the phrase “I intend”. He emphasised the need for constant training and drills to help keep people prepared for when the inevitable mistakes were made. I also liked the concept of pushing ‘authority down’, allowing his crew to control their decision making.


The biggest resonator though was his perspective that you can’t change people, all you can do is provide an environment where people can change themselves. After all “When a flower doesn’t bloom, you fix the environment in which it grows, not the flower.” I have this ideology too in building & developing test teams. In 2013 I spoke at Tasting Let’s Test about building a context driven test team. In it, I talk about creating a habitat where testers can thrive and grow, with a focus on coaching to support that work. I’m working hard to create a habitat like that in Tyro Payments.


The underlying belief he had was that “people are fine the way they are stop trying to fix them” struck deeply with me. If you start with that premise you come to coaching with a different mindset. And just like I came to Agile Australia with a mindset of learning this year as opposed to speaking, I can approach coaching with intent to listen instead of teach. Instead of focusing on where a person needs to be, you think of were a person is. The spotlight is placed on the person not your perception of that person.  Perhaps it’s subtle differences like this that can make all the difference in coaching. Or maybe not. I don’t know. It’s an experiment, but one I’m performing on myself rather than on other people.


And I think that’s exactly how we all like to learn.

Hi, my name is Done and I’m conflicted

If there is ever one word that highlights the difference between a tester and a developer’s mindset it has to be the word done.

Developers¹ tend think of done as a form of sign off. A story is done when the code is complete. For many developers, it’s when the code is complete, tested and deployed. Done is a sign of completion, of moving onto something else.

Testers² look at Done as the beginning of a quest, an exploration. Done is to be prodded, probed, explored and dissected. Under what conditions can ‘done’ fail? What data tests the limits of Done? How about if two Done’s are put together, how will they behave?

With these two different mindsets it’s no surprise that at times there is conflict and disagreement.

Most testers (most people for that matter) shy away from conflict in order to maintain team harmony. Instead, they try to gain agreement upfront on Done. The goal is clarity and scope definition. Make done clear upfront (before coding begins) and it lessens the conflict later.

In many companies, conflict is seen as a ‘bad thing’. Being in conflict suggests that you’re not a team player. But conflict is a not always bad. According to the 5 dysfunctions of a team by Patrick Lencioni conflict is healthy indicator of trust and ought to be encouraged.

And testers must not forget that our primary role in a team is to raise uncertainty about Done. This has to come ahead of wanting to be everyone’s friend and being accepted. The reality is, if we are doing our job we will be testing the limits of preconceptions of Done.

Because as appealing as it is to think of Done in terms of black and white, zero and one, the reality is a lot different. Done is subtle and muted with hidden dimensions and unknown crevices. Exploring and shining a light³ on these dark corners generates new information. That information helps mould and develop our understanding of Done. The more we test, the better this becomes. Whether this new information turns out to be accepted or rejected is to some extent irrelevant, both contribute to fleshing out our understanding.

So sure, have your discussions on ‘Done’ before code is written, but let’s realise that it’s only the beginning not something final written in concrete. Allow testing and testers to expands that understanding.

In fact, I would go as far to say that true harmony is having a mutual goal of building our understanding of what Done means. Who knows? Maybe through that mutual goal can real trust and respect be built within a team.

¹ ² generalisations

³ kudos James Bach

Why waterfall kicks ass

I read a blog post about why waterfall is NEVER the right approach and I feel compelled to respond to what’s touted as the waterfall mindset. Here’s a copy of the paragraph, but you can read the whole post on the above link to get a better sense of context.

I actually don’t believe adopting waterfall as an approach is ever a good choice.  Waterfall comes with the following mindset:

  • we don’t need feedback between the steps of requirements, analysis, design, code, test
  • we can hand work off
  • big batches are ok since they enable us to be more efficient
  • specialized skills working only on their specialty is good
  • we can understand the work to be done before we do it
  • written requirements can specify what we need

Putting aside for now, the use of absolutes, lets address this waterfall mindset:

1) we don’t need feedback between the steps of requirements, analysis, design, code, test

I’ve worked in both waterfall and agile over the years. In those ‘bad old days’ where no-one appreciated collaboration, we used to extensively review requirements. This meant that testers offered valuable input into requirements before any piece of code was developed. Since the invention of ‘agile’ almost everyone has discovered the 3 amigos, but honestly, this is not an agile concept, it existed way before agile was even thought of.

2) we can hand work off

Honestly, I don’t understand this sentence. I’m serious. I can think of many reasons why handing work to others is a good thing. For instance, if I’ve got too much work I’m in danger of becoming a bottleneck, and I hand some of my work over to someone else. If that’s a waterfall mindset, its one I like to have.

3) big batches are ok since they enable us to be more efficient

What do you mean by efficient here? Does it mean quicker, better quality, less waste? Efficient in what? Design, Writing code? Testing? Support? If I wish to carry 20 oranges from point A to point B, is it more efficient to carry them one at a time, or do I get a bag and carry them all together?Try delivering to a customer 1/4 of an IC circuit and request feedback? Sometimes delivering something in one batch IS the the more efficient way to proceed.

In waterfall, we did have teams, and work was allocated into smaller isolated tasks. It was rare one person developed the whole product. In fact, when I worked on telecommunication systems in the nineties, the concept of frameworks was being introduced, segregating data from its transportation method, allowing for people to work on separate parts of a system in parallel to each other.

4) specialized skills working only on their specialty is good

When I started working at Nortel Networks in 1994 it was all waterfall, except we didn’t call it waterfall then, we called it software development. Nortel Networks had a policy that encouraged developers and testers to spend time working in each others ‘specialisation’ area. For six months I became a developer working to deliver software. I was taught C++ and object oriented design principles, so I’m uncertain why you think this is a waterfall mindset?

5) we can understand the work to be done before we do it

Why is being able to understand work before you begin considered a ‘bad’ idea? I think this phrase is too ambiguous to be able to discuss with any merit.

6) written requirements can specify what we need

Is it the word requirement, or is it the fact that it’s written that makes this a ‘bad’ mindset? Yes we do have written requirements in waterfall. We also have written user stories in agile, so what is the point? Just because stories are written in confluence doesn’t make them any less written.

In the bad old waterfall days, I facilitated workshops with the business and IT teams to determine and understand risk. Lots of verbal collaboration, lots of whiteboard discussions.  I’ve worked in ‘agile’ teams were little communication takes place, no stand-ups nothing. In fact, one developer spent a week working on the wrong story without realising it.

I’ve worked on some fantastic waterfall projects that blew the socks of some really crappy agile teams I’ve worked with recently. In these waterfall projects, I’ve worked in harmony with great developers and testers, working closely together. We were afforded sufficient time to examine the system as a whole instead of its parts, something that these days appears to be a bit of a luxury. I’ve worked in environments that develop hardware, firmware and software where upfront design and ‘big batch’ systems thinking helped us understand that we were not merely developing code, but we were attempting to solve a problem.

Its easy to conflate poor software development practices with waterfall, just as its easy to conflate an agile approach with ‘good’ practice. To me the biggest change since those days are technological ones which allows us develop, integrate and compile it quickly and relatively cheaply. It’s the technology that has allowed us to develop in these small tasks that we are familiar with today. In the early nineties the concept of layering and isolating according to purpose was coming into play but we simply didn’t have the sophisticated systems that allowed us to develop in a way we wanted to.

A second important change was the conception of the agile manifesto. To me the agile manifesto is a stroke of genius. People who had the courage to espouse ideology that placed people over process, tools or artefacts. For many, this has changed how we think about developing software.

But lets not forget, that the agile manifesto was developed by people who worked in what you call a waterfall environment.  It seems to me these people had quite a different mindset than the one suggested above. It seems to me, those who developed the agile manifesto did so with ideas of collaboration and an emphasis on the ‘humanness’   of developing software. Do those ideas come about as a result of what was done badly in waterfall or because of what they saw was working well?

I suspect it was a little of both.

Don’t get me wrong. Lots of mistakes were made pre agile days. There was an idea of segregation between developer and tester in an attempt to avoid bias from developers. I’m glad we’ve gotten over that idea. But many of the mistakes we made in those days we’re still making now. Most software teams fail to understand testing and attempt to measure it using means that appear to have no direct correlation to quality. Many teams use measurement to lock down estimates as opposed to using them as a source of information for change. Go to an agile conference and count the number of talks on process and methodologies and frameworks. Look at today’s obsession with continuous delivery as a process. What happened to the people folks?

It’s easy to understand why. The agile manifesto is bloody hard to implement. Its much easier to point at a tool or a process and say “thats what we do” because its explicit, it’s easy to see. Programming & testing are human activities and are much harder to identify and talk about. It’s hard to describe and transfer these skills. The best way I know of is to actually perform the tasks.

We need a little more humility in acknowledging the great shoulders that agile stands on. Its simplistic to identify in hindsight a ‘waterfall’ mindset. Such a thing did not exist. Instead lets view them as the agile manifesto encourages us to do, to view them as people attempting to deliver quality software just like we are attempting to do today.

Where have all the good jobs gone?

If you follow and believe the twitter conversations, it seems that the main reason for getting ISTQB certification is to pass the screening when applying for work. No ISTQB? No interview!

My personal experience has been that yes, on one or two occasions I’ve failed to be interviewed based on lack of certification. Rather then see this as a negative, I see this as a blessing. After all, if your idea of a tester is that narrow, then I’m probably not suited to your company. On occasion I’ve cited ‘NO CERT’! to justify why I can’t apply for a role. “I’m so sorry, I’m not ISTQB certified…” as I edge my way to the door.

I think it’s naive to rely on a certification as a means to getting work. Especially if you are a thoughtful and intelligent tester who cares about the quality of your work, and who wants to be taken seriously in the industry.

But forget about certification for one minute. Are you seriously willingly going to put your career into a strangers hand who then decides your fate by a keyword? There are cleverer ways to play this game.

Have you not noticed that the way companies recruit is rapidly changing? To get a job that interests you, its not good enough to send in your resume or hold a silly piece of paper with a over ornate stamp on it. Those days are long gone. Now you need passion, you need to keep up to date with whats happening in your field, you need to be committed to keeping yourself relevant.

Our family has experienced this first hand when my husband was looking for work last year. He suddenly discovered at the ‘old age’ of 42 that he was unemployable. He is intelligent and personable(yes I am biased) with a first class degree in Electrical Engineering and he had made the assumption that good people always find work. But that’s not enough. Today, if you want a job that’s worthy of you, you need make sure you earn the companies respect.

This is not only based on my personal experience, I’ve spoken to many recruiters in the last few months, and all seem to have similar stories. Companies are more reluctant to use recruitment agencies to find their staff. They want recruiters who know and understand the specific skills they are seeking. I’m seeing recruiters leave the industry, or re-invent themselves as specialists in one field. Other ways the industry has changed is that many recruiting companies work for one company and are in effect the procurement arm of the company.

Of course, the testing industry has changed significantly too. Now that the major consultancies have successfully sold testing as a commodity, testing (or an excuse for testing) is being performed wherever people are cheapest. The adoption of Agile as a development process and its dependency on automation has also reduced the need for testers (though this is not necessarily a bad thing). The fact is, there are less testing jobs out there.

That doesn’t mean there are less quality testing jobs though. While its true that the crumbs from the table need to be shared among more, there is still plenty of meat and gravy at the table. The question is, have you earned a spot there? Here’s a fact. You are not going to earn a spot on this table with a certificate. The path to this table is through credibility and reputation.

My first bit of advice is if you want a worthy job, then you need to be worthy. Examine the work that you have done to date. Does it reflect your skill and perhaps more importantly, your ability? What about your attitude? Does your work reflect that of someone who is passionate and who loves testing? You don’t need rockstar status, but you do need to be an eager apprentice. If you have aspirations to get a great testing job, but you’re not prepared to put in the hard work, then why should you deserve a great role?

Here’s something I do that has proven to be very useful. When I start a new role, I ask myself two questions. The first is “If I leave, how do I want people to remember me”? and second is “what legacy do I want to leave behind me”? It may sound ruthless to think about an exit strategy when starting a role, but the reality is, NO job is permanent so why treat it as one?

So, you are now a worthy tester, the next step is to be able to demonstrate this worthiness and please, put away that tired old resume! I’m talking about blogging, and speaking and contributing to the testing community. Contributing to the community is a great way of meeting local and international people and you learn so much. Regarding speaking, this doesn’t have to be large conferences, there are plenty of small local meetups that offer you a space to speak. No tester meetups near you? Why not create one, or speak at a developer meetup.

Thirdly you need to network. I can’t emphasis this enough. This is where the jobs are. You need to consider two types of networking, local and online. Local is essential if you want to find work in your area. This means meeting people face to face at the local meetup. Yes, I know masterchef is on a Tuesday night and this clashes with the meetup, but hey, do you want a great job or not? Online networking is important too because it allows you to connect with like minded people, plus its a great source of learning. Many jobs come from both online and local networks.

You also need to research. Find out the good companies, speak to people through your network (not agencies) about the ‘good places’ to work, and make a plan on how you are going to work there. Having an online presence helps a lot here, but so does face to face networking. And be patient, great jobs don’t just drop off trees and fall into your lap.

Yes, ultimately, these jobs don’t come easy. They require hard work, and a willingness to put yourself out there. It comes at a cost to your personal lifestyle. but hey! It’s all about choice. Great jobs are around, but its about seizing the day and making the opportunity instead of relying on an agent to do it for you. This is a good thing. Trust me. As I said at the start, why should you put your fate into someone’s hands?

The good news is more than ever before, companies who recognise and value their staff, who recognise and value quality testing, are recruiting in a grass roots way. If you want these types of jobs, it’s easier to get them.

Now maybe this all seems like too much work and you know? I can live with that! Seriously, its your call. But don’t tell me that certificate is mandatory to work in testing, because its not true. Many companies who ‘get’ testing will hire you without a cert. The question that is probably more pertinent is: “Do they want you?”

So get out there, work your butt off and then market your fabulous testing skills. If you stop putting your pearls before swine, one day that dream job will be yours!

Speed Kills

Some of my testers have become embedded on a newly formed agile team. Its been a roller coaster ride for sure. Lots of fun, thrills and a few scary points in time where I thought for certain we were not going to make it, or we were heading in the wrong direction.

From a testing perspective, we were quietly confident. Our testing is exploratory by nature, and so it lends itself easily to being flexible and adaptable.

We’re testing before the acceptance criteria are developed by attending the workshop and asking questions about the upcoming feature, learning why its needed and what problem its trying to solve.

And when it comes to testing the feature, we all jump head first  into the deep end, swimming with long powerful strokes through the water of doubt and complexity and only surfacing for air to congratulate and high five each other on another completed testing charter.

Its been a wet wild ride, but not without some uncomfortable moments.

While we’re revelling in the warm waters of upfront information and involvement, we’ve also noticed the cooler waters of reduced timeframes. Shorter iterations has placed a lot of pressure on the testers. There’s a feeling that we’re unable to pause and reflect during our testing, the pressure to be done within the iteration lends to the temptation of minimalist testing.

We had to change something to make sure we were testing well enough.

So, we slowed down.

We made sure we spent time to think critically about our testing, coming up with test ideas, understanding what done meant, and most importantly sharing these ideas with each other (including developers).

Now we test like this:

We still jump into the deep end, splash out, learn some stuff about the product but before we continue with some serious swimming we stop and confer. We reflect on our learning so far, we discuss our ideas and how we know we are done. Only then do we continue with our swimming, our strokes confident and strong.

Its been a revelation for some testers that are new to exploratory testing. They thought the objective was to test as fast as you can without stopping for a breath.

We still have the same time pressure on us, but that stop, that moment of pause and reflection has been sufficient to gain confidence in our approach and confident in delivering the best testing that we can.