badass tester marshmallow charrett

Badass Tester Marshmallow

When Keith worked at Barclays he held me up as the pinnacle of what a badass tester was. I found this funny as only the other week I had been called a marshmallow and so the phrase ‘badass tester marshmallow’ was born.  Why was I given the name ‘badass tester’?  You’ll have to listen to the podcast to find out.

I asked Trish Khoo to come up with a design for a T-shirt based on the phrase. Now you can get your very own badass tester marshmallow T-shirt.

Badass Tester Marshmallow T-Shirt

Badass tester marshmallow

Badass tester marshmallow



Dumbfounded by Or Hilch

A different perspective on Boundary Testing

The book ‘Boundaries after a pathological relationship‘ by Adele Birch has a load of practical advice on boundaries in relationships. As Trish Khoo who recommended it to me said: “even if you don’t think you’ve had a pathological relationship, this is a good guide to setting and enforcing personal boundaries”, and it is. It’s full of practical advice and tips on understanding, setting and managing boundaries.

This book applies to work too. Setting boundaries is vital to maintain healthy relationships within your workplace. Thinking about and setting boundaries between yourself, your peers, business partners, your managers and if you’re in a leadership position, people who report to you will help you better cope when one of these boundaries is violated.

When you are a minority on a team, as  testers frequently are this becomes even more important. Because unless your team works hard at creating a safe team environment, speaking up as a minority can be difficult and costly. It can be hard to be heard in a retro when your dot has to work against 7 other collective dots focused on other priorities.

Work has other challenges too. We are all familiar with the ‘delegator’ who happily shoves responsibility onto other’s shoulders as if its the most natural place for it to reside (it isn’t).Without clear boundaries, many are likely to take on responsibilities that are not actually theirs. There’s and old but wonderful book called the One Minute Manager meets Monkey recommended to me by Graham Lea on this topic. Worth a read if you’re can’t understand why you seem to have too much on your plate.

Without firm boundaries, you can end up carrying the workload, while your team pats themselves on the back at the consistent work flow produced.

The book describes different types of boundaries you might want to consider with examples of what these might be. One exercise is to describe the personalities and traits of your ideal partner. This probably isn’t that useful for a work relationship, we don’t get to pick and chose who we work with. Instead of thinking of one partner, replace it with company values. Know what you will or will not tolerate in a working relationship.

At some point in your working career some one will intentionally or unintentionally cross one of your boundaries.  Knowing what they are, and what you are prepared to tolerate (and not tolerate) will prevent feelings of violation and powerlessness. Being able to uphold your boundaries can be frightening for many of us, but when you manage it (and I believe you will) it’s empowering and liberating.


It doesn’t mean that you will be impervious to boundaries being broken. In my experience, some boundaries can only be discovered after being crossed. We are after human, and our boundaries will probably shift as we journey through life. Hopefully, though with a little bit of practise, you will find it easier to maintain healthy boundaries and move to building solid team relationships.

I’ll leave you with a mindmap of the lessons I learned from this book and applied to the work context.

Boundaries at work by A Charrett

Proviso: I’m not an expert in this field by any means. I’ve written this post based on my own work experiences and the advice may not relate in any way to your context.

The Toyota Engagement Equation By Tracey San

Quality and the Toyota Production Model

I saw this tweet the other day prompting this post on quality, visualisation and quality at pace. 

Understanding Quality

“Quality is value to some person” is a Jerry Weinberg quote emphasising the subjective nature of quality.   Tapping into an understanding of what your customers uphold, believe in and need goes a long way to creating a great product. It’s not easy to tap into what your customer’s value, but once you do, once you discover that “x-factor” you generate customer loyalty.   Michael Bolton added “Quality is value to some person who matters”. I and others have added “at some point in time” to indicate the transient nature of quality. This is the beauty and terror of quality. Its notoriously difficult to understand, let alone measure and quantify. Often we resort to quality attributes in an attempt to know and measure it.  How then does quality relate to the above tweet? What struck me was the underlined phrase: “Its about making problems….visible to workers and management, and addressing them as close to the source as possible” Breaking that down…

“Making problems visible….

Understanding and knowing what quality is for your team, your company and customers, makes it significantly easier to recognise problems that threaten its value. The difficulty is in knowing where these problems lie. If we knew where problems existed, their nature and size and their impact, making it visible would be much easier. Software testing helps us identify and shine a light on such problems. It gives us the knowledge we need to “know” if we are reaching the quality our stakeholders want. Poor old software testing though. Perceived as expensive and time consuming with every test having a cost. A cost in thinking about it, developing it, using it, evaluating the output, fixing any bugs related to running it, and retesting. But when performed as an investigation to discover threats to quality, software testing excels. It can discover problems no-one had dreamed about, one reason why many software testers are seen to have super bug finding powers. The reality is, we spend all are days thinking about how things might go wrong. And, like any skilled person it’s become a well honed craft.  By broadening the scope of our risk analysis beyond stories and product functionality, software testers can begin to exercise their powers of investigation and experimentation to many other areas. Many already help in story analysis to identify risk, but there are other places that potential problems lie. For example, performance, infrastructure, operations, test environment deployments, build times, bloated regressions test suites, flakey tests are all types of risks that potentially risk investigators can explore. I have often challenged and encouraged software testers on my teams to look beyond their bounded context and think of different types of risk. To create small experiments to investigate if a perceived risk is a real threat, and if it is, hand over to engineering to solve. The vital role of risk investigator starts to become real.  By rebranding a software tester to that of risk investigator, we can begin to see how this skill might be useful in the context of the Toyota Production Model where consistent visibility of problems is desired.

…to workers and management

It’s not enough for risk investigators to conduct experiments in isolation. That information has to be delivered to many different types of people whose concept of quality will probably differ and even conflict. Having data and being able to portray it in a way that is useful to that team is a real challenge. Conducting small experiments and providing as much technical data to other engineers is one thing. Facilitating senior management decision making is a different ball game (In the article Finishing Exploring I describe the subtle difference in providing data and supplying valuable information). I’ve always liked the idea of having a UX person create senior management infographic. I explored this concept at Tyro Payments when I was Head of Engineering there.

Visualizing Data by Anne-Marie Charrett Copyright 2017

Visualizing Data by Anne-Marie Charrett Copyright 2017

For senior management, think about how you can portray information in a visible, colourful and easy to read manner. Use BLUF (Bottom line up front) and KIS (Keep it simple) principles avoiding dense text.

“..addressing them as close to the source as possible”

Early Feedback in software testing is not a “new” thing. The sound principle of unit testing has been around since I walked the hallowed aisles of Nortel Networks as a developer and a tester in the early nineteen nineties. What has changed is the technological advances that have enabled us to provide this feedback faster than before, and we want more of it. Long feedback loops introduced by end to end performance testing, or end to end GUI regression testing performed long after code has been written impact delivery. Unplanned work such as fixing bugs we didn’t know of disrupt this flow.   We’ve tried in the past to use automation in an attempt to speed up the feedback loop. That has worked to some degree, but its added additional work (read cost), and the value of some of these automated tests is debatable. Put it this way. If your end to end tests are not finding bugs that could be a problem. But if they are, that could be a problem too! By looking at alternatives to software testing that provide visual indicators of threats to quality throughout the life cycle we can begin to better understand the state of quality throughout delivery. I call this quality at pace.  More on that later

Test Leadership is here to stay

In his article “What leaders do”, J.P Kotter makes the following distinction between management and leadership:

Managers promote stability, leaders press for change

For example:

  • Management involves planning and budgeting. Leadership involves setting direction.
  • Management involves organizing and staffing. Leadership involves aligning people.
  • Management provides control and solves problems. Leadership provides motivation.

The left hand side sounds a whole lot like what test managers traditionally do or have done in the past.

But, as organisations move to a more agile methodology tasks associated with these responsibilities are becoming redundant.

For example:

No more Test Team

Testers frequently sit within an agile  team alongside UX people, a product manager, developers and operations. Often they report to a senior person within the team. Large independent test teams no longer exist. The need for a test manager from a people management perspective is no longer required.

No more Big Test Strategy

Small agile teams working on small stories reduce the size and complexity of work involved. Any decisions on testing are typically made at a team level as autonomy and decision making is pushed down into the teams. There’s little need for a formal test strategy for work at a team level, and little need for a test manager to develop a large test plan or test strategy that outlines testing and resources required over an extended period of time.

No more Formal Test Process

You have 10 days to think and complete testing for a story. If you’re lucky, five days of that may be actually testing. Who the hell is going to worry about writing test cases and a formal test report? The nearest you get to a formal test process is a Jira Workflow. So bye bye formal test process. Bye Bye Test Manager to enforce test process.

No more Formal Testing Metrics

Metrics are becoming more team focused. Teams are using metrics such as MTTR (Mean Time to Recover), or LTTR (Lead Time to Deployment). Interesting ideas, and I’m glad to see a shift perceptions of quality. Again though, we don’t specifically need a test manager to report on these metrics.

No more Gatekeeping

Testing has traditionally (and unfairly) been the Gatekeeper to ensuring quality.  The business sees the testing department as a method of control. I suspect the entire testing community is heaving a quiet sigh of relief at the demise of this one!  Ensuring quality is a fool’s game, as there is no way you can win. It’s akin to forcing someone to sign a pre-nup to ensure love.  Good luck with that.

No more Test Managers then?

Yes and No. The role of test manager as we know it is dissapearing. There’s evidence of the demise of these roles in many of the large enterprise organisations I work with.

But while there’s less need for test managers that doesn’t mean that the thorny challenge of testing has disappeared. And while deploying in bitesized pieces reduces risk, it doesn’t it remove entirely. Often the risk is displaced to somewhere we are not used to investigating.

Something AWS discovered much to their dismay.

Some of the testing challenges many companies are facing:

  1. How do we test large scale systems and architectures (think Microservices)
  2. How can testing be an activity that all perform?
  3. How can we develop the testing capability within teams?
  4. How can we get better diversify our thinking in our testing?
  5. How can we think about testability as part of systems design and architecture?
  6. What part do testers and testing play within our teams?
  7. What skillsets do I need testers to have? Is it just one type of skillset?
  8. How can you test in production?
  9. How can we better understand and provide information on quality?
  10. How can we better respond to change, that is either in and out of our control?


Let me explain that final point.

Software Testing doesn’t have a great reputation for being flexible and adaptive to change. The way we test is deterministic. Think gated process and scripted test cases. But then neither is Test automation.

Like it or not, change happens. Sure, the extent to which change happens and the nature of that change may depend on the industry and technology you work with. What becomes important though, is how we handle that change. And, how we equip our teams with the ability to adapt to change becomes the task of a test leader.

We need confident self possessed software engineers, equipped with the ability to think critically through whatever challenge crosses their path. We need to help them work together to solve whatever testing problem comes their way. This requires a positive and safe environment where teams become ‘test-infected’ and have a culture of continuously thinking about improving their testing.

Test Leaders provide this breathing space for a testing culture to grow. They achieve this by placing an emphasis on vision, strategy and motivation and alignment. Examples of this are coaching, training, knowledge sharing and making information visible to the rest of the organisation.

A testing culture that provides this space itself generates test leaders. That’s important, because to some degree every tester is an advocate of quality and that in itself requires a degree of test leadership.

The skill set required in test leadership is different to test management. It’s more aligned to coaching, connecting people and driving a shared vision of quality for your company. It requires revisiting ideas on why we test and what software testing actually is and how we can continue to deliver value in a different context.

But I don’t see uncertainty going away, in fact I only see it increasing. And so, neither is test leadership.

Quality Workshop

Quality Workshop

One sure way to liven up a meeting is to ask attendees for a definition of quality. My experience has been that this is most entertaining particularly if you have both devops and testing in the room.

It turns out that many people feel very passionate about Quality. It also turns out that many have different ideas on what quality actually is. Who knew? Diversity of ideas are not bad but it can impact a teams understanding of quality. The consequence is that it could possibly hinder a team’s ability to delivery quality product.

To help people understand the challenges around quality and to try clarify what quality might be in their context, I developed the following quality workshop.

Purpose of Quality Workshop

a) to get a working definition of quality that most people are comfortable
b) to outline a manifestation of quality in terms of quality attributes
c) to identify ways in which we can these quality attributes can be acted on

Who should attend

Depends on what your trying to achieve, but if you’re a team delivering product, I would suggest your key stakeholders come along. That may mean product owner, developers, testers, Ops. If you can get your business there too that’s a big plus.

Tools for Quality Workshop

  • A set of Quality Attributes (I’ve known these also to be called Quality Characteristics, or CFR’s – Cross Functional Requirements) printed out and placed on a wall.
  • PostItNotes (of course!)
  • some pens.

Process for Quality Workshop

  1. Ask people to write down on a post-it note their understanding of quality
  2. Place the definitions on the wall and ask people to read them (or read them out)
  3. Note the diversity (if it exists, it typically does). Bring up the subjectivity of quality and how difficult it is to agree on and quantify.
  4. Ask what the implications of this might be throughout the delivery lifecycle (e.g Design, Development, Ops)
  5. You can dot vote to try and get some sort of agreement on a working definition of quality*.

* My working definition of quality is the Jerry Weinberg one “Quality is value to some person”.  If no-one puts this definition up on the board, I like to add it as it encourages a discussion on the subjectivity of quality. 

If you’ve got this far without some major outburst or spat, congratulations! Next step is to try and parse what quality might look like.

  1. Ask people to read the quality attributes on the wall
  2. Ask them to dot vote 3 what they consider to be the most important quality attributes
  3. Collect the top 3 quality attributes and put them on a whiteboard
  4. Ask the group to identify 3 tasks for each particular quality attribute and to write them on a post-it note.*
  5. Place the post it notes on the whiteboard along side the quality attribute
  6. Run through the post-it notes. Discuss the viability of these and how to make these tasks happen.
  7. Visualise and share this information to the broader organisation.

* I avoid using the word ‘testing’ as I want people to think about other factors that influence quality. Testing is important, but testing helps us evaluate quality.  Unless its acted upon, it doesn’t help us build quality. Sometimes I like to prime people by providing a couple of non testing examples such as Code Reviews,  Monitoring and Security by Design. 

I’ve found these sessions generate a lot of discussion and can become quite heated.

I wouldn’t run a quality workshop all the time, but its really useful iron out at perhaps an epic or feature level, what quality might be and how a team can work together to implement that vision.

(Learn more on this topic by attending my Quality Matters Workshop)

Let me know how you get on!

Photo Attribute:  Quality by jason Taellious

What Teenagers and MVP have in common

Teenage children can be the classic MVP’s (Minimal Viable Products) in action. Here’s why:

  1. They tend to be  self deterministic
  2. They tend to experiment constantly
  3. They don’t attempt to predict the future but learn from their experiments
  4. They typically have no endstate in mind that they wish to be made explicit
  5. They have courage to outline their path determined by their feedback
  6. They have faith in the outcome even if the outcome is unknown

Ok, so I made a lot of that up. But hear me out…

Teenagers can the quintessential examples of MVP’s in progress. If you have one in your house and you want to understand more about MVP.  Observe and Learn from them. They can teach you wonderful insights into known, knowns and unknown, unknowns. Even more, you will have insight to the  ‘knowns I wish I could now be unknown’. Something I’m not aware the Cynefin model caters for?

As a finale, here’s a voyeuristic view into what my teenagers do right now…

A post shared by @nikolai.charrett on

— love your work boys…

Update: Since I wrote this, I’ve thought a bit deeper on the topic. To some degree we are all MVP’s regardless of age. To some degree we are all completed products, but also we’re often minimum versions of the end state. At least I am. I’m still trying to figure out who I want to be, what I might want to do, and what that might look like.

building the quality inn

Building the Quality Inn

Building Quality In integral part of moving to a DevOps culture, at least according to the State of DevOps 2016. But what exactly does that mean?

Sure, the report cites Deming, in particular

 “Cease dependence on inspection to achieve quality. Eliminate the need for inspection on a mass basis by building quality into the product in the first place.”

And while the concept is not new, and seems to make sense at least at an ideological point of view, what in practice would it look like? This was what Matthew Santon Rutherford, the Quality Guild Doctor at IAG asked me.

He also mentioned that when people spoke of ‘Building Quality In’ it immediately conjured up this image of a run down motel somewhere remote, with a pink flamingo sign, swaying slightly in the wind.  We laughed and played around with this image for a while, adding an empty swimming pool and trying to imagine what each room might look like and who its customers might be.

I dedicate this series of “Building Quality INN” blog posts to Matthew, who has forever burned this image in my mind. What would your version of Quality Inn look like? Do you have a picture that might represent it? Please share !


Nuance on leaving testing to the experts

Yesterday I wrote a blog post entitled “Leave Testing to the Experts” to which Maaret quickly reacted online and by a post where she strongly disagreed with the sentiment.  Disagreement is a very useful tool to clarify your thinking, so thanks Maaret!.

It’s allowed me the opportunity to explain in more detail, why I think what I do, and to investigate some of the nuances on this topic.

Maaret, counter-argued with a convincing description of her egalitarian work place where opinions are offered and accepted in good faith. In truth it left me feeling a little envious, and a commitment to work harder on taking ‘being told’ as “bad communication with good intent”.

But I think this post misses a vital point and that is responsibility.

Taking on a role³ of tester comes with a responsibility to have testing performed to your best ability. So, if I listen to an opinion and follow it, knowing its wrong, I call that irresponsible. If I hire a tester based on likability over capability, that’s irresponsible. If I fail to highlight risk for fear of upsetting team members? Well, you get the point¹.
Perhaps you might say that testing is a collective responsibility, and we all own the decision. After all, in an egalitarian team such as Maarets, doesn’t everyone own quality? Isn’t everyone responsible for testing? Well, yes and no. Let’s explore that a bit by taking the pilot example.
It takes a crew to fly a large plane. There’s the engineers, the pilots, the cabin crew and many more. The pilot happily takes on other crew member’s opinions and ideas. Perhaps she listens to the engineer’s advice not to fly above a certain height. Or maybe she nods when a cabin crew member tells her that people like to hear from her voice more often. Perhaps her junior suggests a flight plan for the journey. She hears all this and takes it into consideration, but flying the plane is ultimately her responsibility². Even if part of that decision is to allow the junior to fly the plane for some of the journey. The buck stops with the pilot.
The crew is responsible for the whole flight experience, but the pilot is responsible for flying the plane.
Similarly, in a team, we’re all responsible for quality, each person may even be responsible for their own bit of testing but ultimately the testing buck stops with the tester.
Notice that everyone got to have an opinion? Similarly on a team, many people can perform testing and offer ideas on testing. This can be incredibly invaluable to testing as a whole. For example, developers have a keen insight into technical risk which can really help identify/dismiss areas that require testing. Teams that are test infected and have this testing mindset are often a joy to work with.
You don’t get a testing mindset if you’re prevented from exploring ideas, so I agree with Maaret that people should be allowed to offer opinions and experiment.
I worked with a team who wanted to implement BDD. I was skeptical. BDD seemed to me a bad fit for the context in which we were developing. There’s nothing like self discovery though and so the team went ahead and tried it out. It turned out to be a dud idea and it was dropped. It’s good to experiment and encourage exploration but there needs to be a proviso here and that is when the risk is low and the impact to the company is minimal.  
Back to the pilot. You don’t handover control of a plane to a junior pilot just as turbulent weather conditions hit. You might if they have more experience in flying, but as the senior pilot you will know when to take the wheel and when to allow practise. Someone who knows what they’re doing needs to make these decisions.
Also performing a small experiment when the risk is low is very different to adopting a strategy company wide. Dictating cucumber as a tool across an organisation is very different to performing an experiment within a team. To clarify, my lament on testing experts is in the context of organisations adopting ‘best practise’ strategies.
Maaret talks about everyone on her team ‘tests like an expert’. I think that’s great, her very capable team means she doesn’t have to exercise responsibility. She’s still responsible for the testing though, just as carrying a plane full of pilots doesn’t absolve the actual pilot of responsibility.
I’ll end this post with the comment about sales people. Maaret says she constantly offer views to developers and sales people. The truth is I have done that too. I guess everyone feels in some way their idea is good and has value and its nice to have your opinions heard.
I remember once querying a company’s sales strategy. I couldn’t understand why we only aligned with partners instead of also focusing on location. I brought this up with the sales manager who listened and then dismissed it explaining why. Personally, I still don’t agree with the approach, but I’m fine with his decision. Why? Because ultimately the buck stops with him. I’m cool with that.
Non testers will always have views and opinions on testing. That’s not a bad thing. In fact, part of me is glad they feel responsibility about testing. But it’s a problem when those opinions start impacting my testing in a negative way. Then I’m failing in my responsibility to perform testing to my best ability. You see, the intent of ‘claiming territory’ is to do a job well, the job I was hired to do, not to hold power over people through control.
I hope this makes sense. Like I said, I’m grateful to Maaret for her response and the opportunity to deepen my ideas on the topic.
³I considered adding the word specialist here as the concept still holds. A specialist (of testing) still holds the ultimate responsibility for testing, even if they don’t perceive themselves as a tester.
¹Many of these decisions are nuanced and are rarely black and white. For example, hiring a tester for a team fit is really important, but if you do so over capability you need to be able to back your decision with some sound logic. Context is important here.
²many people inside and outside a team test, so a tester isn’t responsible for doing all the testing, but they are responsible for the strategic direction and process of testing.

Leave the testing to the experts!

Why is it that in a company almost everyone has an opinion and knows the best way to test? What makes so many of these people feel they know how testing ought to be performed?

Business people tell me, the tester:

“You should be writing test cases and reporting against them”

Developers & technical people tell me to:

” automate the lot using cucumber/PACT/Selenium or some other niche technical tool” 

Test Managers (many who have never tested) tell me:

“You should be following a formalised signed-off testing process”

I find this quite extraordinary. I don’t tell you, oh developer how to code your program. I don’t tell you oh, sales person, how to sell your product.

So why do you think its reasonable and perfectly acceptable to tell me how to test software? 

Perhaps its because we all test to some degree. But just because I can build a car or sell one doesn’t mean I can drive it. Just because I know how to drive a car, doesn’t mean I go and tell a rally driver how to race around a track.

The truth is, most developers, project managers and business people don’t understand testing in any deep way. Instead they often resort to shallow imitations, an emperors test suite, if you will.  That’s partly the fault of the testing profession. We’ve failed to claim our turf and instead tugged our forelock, deferring ideology to those with authority but little understanding.

It’s time tester’s to claim some turf back. I suspect (actually I know) for many places it won’t be given up without a struggle. You’re going to need courage backed by confidence in your ability if you want to claim some of the territory. But it’s worth it. With space comes freedom and the autonomy to drive your testing strategy and process in a way that you know will offer value to your organisation and your clients.

Now, please get out of my way and leave the testing to the experts!

Mount Everest

The Base Camp Heuristic

The ‘Base Camp’ heuristic is the work required to find any bug. Typically ‘Base Camp’ bugs tend to be low lying fruit and can be found quickly, especially if a tester is experienced. In the world of Daniel Kahneman, these bugs are found using Systems 1 thinking.

To discover an ‘Everest’ type bug requires Base Camp intuition plus more. For example a tester might need to deliberate over the product in detail, question the diversity of the data, the validity of the oracle(s)under use and the procedure required for testing.It may mean you need to create new tools, or find new ways to track the unknown.

That means you’re going to need mental training to extend the boundaries of your thinking, determination to continue when the easy answers stop flowing and some tactics to help you refocus and come up with new solutions when you run out of ideas.

You may feel dismayed because everyone around you is finding bugs at a rapid pace. You may even encounter skepticism because you appear to be showing no value. But Everest quality bugs are not meant to be easy to find. They require stamina, commitment and courage.

It’s rare to find bugs of this calibre without training and serious commitment.

Maybe I’m not going to climb Mount Everest every day, abut as an aspiration I’d like at least some of my bugs to be of the Everest calibre.

How about you?