Dumbfounded by Or Hilch

A different perspective on Boundary Testing

The book ‘Boundaries after a pathological relationship‘ by Adele Birch has a load of practical advice on boundaries in relationships. As Trish Khoo who recommended it to me said: “even if you don’t think you’ve had a pathological relationship, this is a good guide to setting and enforcing personal boundaries”, and it is. It’s full of practical advice and tips on understanding, setting and managing boundaries.

This book applies to work too. Setting boundaries is vital to maintain healthy relationships within your workplace. Thinking about and setting boundaries between yourself, your peers, business partners, your managers and if you’re in a leadership position, people who report to you will help you better cope when one of these boundaries is violated.

When you are a minority on a team, as  testers frequently are this becomes even more important. Because unless your team works hard at creating a safe team environment, speaking up as a minority can be difficult and costly. It can be hard to be heard in a retro when your dot has to work against 7 other collective dots focused on other priorities.

Work has other challenges too. We are all familiar with the ‘delegator’ who happily shoves responsibility onto other’s shoulders as if its the most natural place for it to reside (it isn’t).Without clear boundaries, many are likely to take on responsibilities that are not actually theirs. There’s and old but wonderful book called the One Minute Manager meets Monkey recommended to me by Graham Lea on this topic. Worth a read if you’re can’t understand why you seem to have too much on your plate.

Without firm boundaries, you can end up carrying the workload, while your team pats themselves on the back at the consistent work flow produced.

The book describes different types of boundaries you might want to consider with examples of what these might be. One exercise is to describe the personalities and traits of your ideal partner. This probably isn’t that useful for a work relationship, we don’t get to pick and chose who we work with. Instead of thinking of one partner, replace it with company values. Know what you will or will not tolerate in a working relationship.

At some point in your working career some one will intentionally or unintentionally cross one of your boundaries.  Knowing what they are, and what you are prepared to tolerate (and not tolerate) will prevent feelings of violation and powerlessness. Being able to uphold your boundaries can be frightening for many of us, but when you manage it (and I believe you will) it’s empowering and liberating.

Ringo

It doesn’t mean that you will be impervious to boundaries being broken. In my experience, some boundaries can only be discovered after being crossed. We are after human, and our boundaries will probably shift as we journey through life. Hopefully, though with a little bit of practise, you will find it easier to maintain healthy boundaries and move to building solid team relationships.

I’ll leave you with a mindmap of the lessons I learned from this book and applied to the work context.

Boundaries at work by A Charrett

Proviso: I’m not an expert in this field by any means. I’ve written this post based on my own work experiences and the advice may not relate in any way to your context.

The Toyota Engagement Equation By Tracey San

Quality and the Toyota Production Model

I saw this tweet the other day prompting this post on quality, visualisation and quality at pace. 

Understanding Quality

“Quality is value to some person” is a Jerry Weinberg quote emphasising the subjective nature of quality.   Tapping into an understanding of what your customers uphold, believe in and need goes a long way to creating a great product. It’s not easy to tap into what your customer’s value, but once you do, once you discover that “x-factor” you generate customer loyalty.   Michael Bolton added “Quality is value to some person who matters”. I and others have added “at some point in time” to indicate the transient nature of quality. This is the beauty and terror of quality. Its notoriously difficult to understand, let alone measure and quantify. Often we resort to quality attributes in an attempt to know and measure it.  How then does quality relate to the above tweet? What struck me was the underlined phrase: “Its about making problems….visible to workers and management, and addressing them as close to the source as possible” Breaking that down…

“Making problems visible….

Understanding and knowing what quality is for your team, your company and customers, makes it significantly easier to recognise problems that threaten its value. The difficulty is in knowing where these problems lie. If we knew where problems existed, their nature and size and their impact, making it visible would be much easier. Software testing helps us identify and shine a light on such problems. It gives us the knowledge we need to “know” if we are reaching the quality our stakeholders want. Poor old software testing though. Perceived as expensive and time consuming with every test having a cost. A cost in thinking about it, developing it, using it, evaluating the output, fixing any bugs related to running it, and retesting. But when performed as an investigation to discover threats to quality, software testing excels. It can discover problems no-one had dreamed about, one reason why many software testers are seen to have super bug finding powers. The reality is, we spend all are days thinking about how things might go wrong. And, like any skilled person it’s become a well honed craft.  By broadening the scope of our risk analysis beyond stories and product functionality, software testers can begin to exercise their powers of investigation and experimentation to many other areas. Many already help in story analysis to identify risk, but there are other places that potential problems lie. For example, performance, infrastructure, operations, test environment deployments, build times, bloated regressions test suites, flakey tests are all types of risks that potentially risk investigators can explore. I have often challenged and encouraged software testers on my teams to look beyond their bounded context and think of different types of risk. To create small experiments to investigate if a perceived risk is a real threat, and if it is, hand over to engineering to solve. The vital role of risk investigator starts to become real.  By rebranding a software tester to that of risk investigator, we can begin to see how this skill might be useful in the context of the Toyota Production Model where consistent visibility of problems is desired.

…to workers and management

It’s not enough for risk investigators to conduct experiments in isolation. That information has to be delivered to many different types of people whose concept of quality will probably differ and even conflict. Having data and being able to portray it in a way that is useful to that team is a real challenge. Conducting small experiments and providing as much technical data to other engineers is one thing. Facilitating senior management decision making is a different ball game (In the article Finishing Exploring I describe the subtle difference in providing data and supplying valuable information). I’ve always liked the idea of having a UX person create senior management infographic. I explored this concept at Tyro Payments when I was Head of Engineering there.

Visualizing Data by Anne-Marie Charrett Copyright 2017

Visualizing Data by Anne-Marie Charrett Copyright 2017

For senior management, think about how you can portray information in a visible, colourful and easy to read manner. Use BLUF (Bottom line up front) and KIS (Keep it simple) principles avoiding dense text.

“..addressing them as close to the source as possible”

Early Feedback in software testing is not a “new” thing. The sound principle of unit testing has been around since I walked the hallowed aisles of Nortel Networks as a developer and a tester in the early nineteen nineties. What has changed is the technological advances that have enabled us to provide this feedback faster than before, and we want more of it. Long feedback loops introduced by end to end performance testing, or end to end GUI regression testing performed long after code has been written impact delivery. Unplanned work such as fixing bugs we didn’t know of disrupt this flow.   We’ve tried in the past to use automation in an attempt to speed up the feedback loop. That has worked to some degree, but its added additional work (read cost), and the value of some of these automated tests is debatable. Put it this way. If your end to end tests are not finding bugs that could be a problem. But if they are, that could be a problem too! By looking at alternatives to software testing that provide visual indicators of threats to quality throughout the life cycle we can begin to better understand the state of quality throughout delivery. I call this quality at pace.  More on that later

https://pbs.twimg.com/media/C5xvGA9VAAAtxPy.jpg

Test Leadership is here to stay

In his article “What leaders do”, J.P Kotter makes the following distinction between management and leadership:

Managers promote stability, leaders press for change

For example:

  • Management involves planning and budgeting. Leadership involves setting direction.
  • Management involves organizing and staffing. Leadership involves aligning people.
  • Management provides control and solves problems. Leadership provides motivation.

The left hand side sounds a whole lot like what test managers traditionally do or have done in the past.

But, as organisations move to a more agile methodology tasks associated with these responsibilities are becoming redundant.

For example:

No more Test Team

Testers frequently sit within an agile  team alongside UX people, a product manager, developers and operations. Often they report to a senior person within the team. Large independent test teams no longer exist. The need for a test manager from a people management perspective is no longer required.

No more Big Test Strategy

Small agile teams working on small stories reduce the size and complexity of work involved. Any decisions on testing are typically made at a team level as autonomy and decision making is pushed down into the teams. There’s little need for a formal test strategy for work at a team level, and little need for a test manager to develop a large test plan or test strategy that outlines testing and resources required over an extended period of time.

No more Formal Test Process

You have 10 days to think and complete testing for a story. If you’re lucky, five days of that may be actually testing. Who the hell is going to worry about writing test cases and a formal test report? The nearest you get to a formal test process is a Jira Workflow. So bye bye formal test process. Bye Bye Test Manager to enforce test process.

No more Formal Testing Metrics

Metrics are becoming more team focused. Teams are using metrics such as MTTR (Mean Time to Recover), or LTTR (Lead Time to Deployment). Interesting ideas, and I’m glad to see a shift perceptions of quality. Again though, we don’t specifically need a test manager to report on these metrics.

No more Gatekeeping

Testing has traditionally (and unfairly) been the Gatekeeper to ensuring quality.  The business sees the testing department as a method of control. I suspect the entire testing community is heaving a quiet sigh of relief at the demise of this one!  Ensuring quality is a fool’s game, as there is no way you can win. It’s akin to forcing someone to sign a pre-nup to ensure love.  Good luck with that.

No more Test Managers then?

Yes and No. The role of test manager as we know it is dissapearing. There’s evidence of the demise of these roles in many of the large enterprise organisations I work with.

But while there’s less need for test managers that doesn’t mean that the thorny challenge of testing has disappeared. And while deploying in bitesized pieces reduces risk, it doesn’t it remove entirely. Often the risk is displaced to somewhere we are not used to investigating.

Something AWS discovered much to their dismay.


Some of the testing challenges many companies are facing:

  1. How do we test large scale systems and architectures (think Microservices)
  2. How can testing be an activity that all perform?
  3. How can we develop the testing capability within teams?
  4. How can we get better diversify our thinking in our testing?
  5. How can we think about testability as part of systems design and architecture?
  6. What part do testers and testing play within our teams?
  7. What skillsets do I need testers to have? Is it just one type of skillset?
  8. How can you test in production?
  9. How can we better understand and provide information on quality?
  10. How can we better respond to change, that is either in and out of our control?

 

Let me explain that final point.

Software Testing doesn’t have a great reputation for being flexible and adaptive to change. The way we test is deterministic. Think gated process and scripted test cases. But then neither is Test automation.

Like it or not, change happens. Sure, the extent to which change happens and the nature of that change may depend on the industry and technology you work with. What becomes important though, is how we handle that change. And, how we equip our teams with the ability to adapt to change becomes the task of a test leader.

We need confident self possessed software engineers, equipped with the ability to think critically through whatever challenge crosses their path. We need to help them work together to solve whatever testing problem comes their way. This requires a positive and safe environment where teams become ‘test-infected’ and have a culture of continuously thinking about improving their testing.

Test Leaders provide this breathing space for a testing culture to grow. They achieve this by placing an emphasis on vision, strategy and motivation and alignment. Examples of this are coaching, training, knowledge sharing and making information visible to the rest of the organisation.

A testing culture that provides this space itself generates test leaders. That’s important, because to some degree every tester is an advocate of quality and that in itself requires a degree of test leadership.

The skill set required in test leadership is different to test management. It’s more aligned to coaching, connecting people and driving a shared vision of quality for your company. It requires revisiting ideas on why we test and what software testing actually is and how we can continue to deliver value in a different context.

But I don’t see uncertainty going away, in fact I only see it increasing. And so, neither is test leadership.

Leave the testing to the experts!

Why is it that in a company almost everyone has an opinion and knows the best way to test? What makes so many of these people feel they know how testing ought to be performed?

Business people tell me, the tester:

“You should be writing test cases and reporting against them”

Developers & technical people tell me to:

” automate the lot using cucumber/PACT/Selenium or some other niche technical tool” 

Test Managers (many who have never tested) tell me:

“You should be following a formalised signed-off testing process”

I find this quite extraordinary. I don’t tell you, oh developer how to code your program. I don’t tell you oh, sales person, how to sell your product.

So why do you think its reasonable and perfectly acceptable to tell me how to test software? 

Perhaps its because we all test to some degree. But just because I can build a car or sell one doesn’t mean I can drive it. Just because I know how to drive a car, doesn’t mean I go and tell a rally driver how to race around a track.

The truth is, most developers, project managers and business people don’t understand testing in any deep way. Instead they often resort to shallow imitations, an emperors test suite, if you will.  That’s partly the fault of the testing profession. We’ve failed to claim our turf and instead tugged our forelock, deferring ideology to those with authority but little understanding.

It’s time tester’s to claim some turf back. I suspect (actually I know) for many places it won’t be given up without a struggle. You’re going to need courage backed by confidence in your ability if you want to claim some of the territory. But it’s worth it. With space comes freedom and the autonomy to drive your testing strategy and process in a way that you know will offer value to your organisation and your clients.

Now, please get out of my way and leave the testing to the experts!

Agile Australia and Intent based Leadership

Today I went to Agile Australia as an attendee. That’s a first for me, usually, I’m speaking. It was wonderful to go with a different purpose, which is to learn from others. Typically, if I’m speaking I’m so focused on ‘my talk’ and getting ready for ‘my talk’ that I don’t feel very open to learning new ideas.


I’m glad I went, because I had the opportunity to listen to David Marquet’s (nuclear submarine commander) talk about the abdicating control (not responsibility) to the people who worked for him. In a military organisation I can’t imagine the mindset shift & the struggle his crew must have felt!


There were many aspects of the talk I enjoyed, how subtle changes in  language was used to help people think differently. For example instead of his crew requesting permission, they informed authority by using the phrase “I intend”. He emphasised the need for constant training and drills to help keep people prepared for when the inevitable mistakes were made. I also liked the concept of pushing ‘authority down’, allowing his crew to control their decision making.


The biggest resonator though was his perspective that you can’t change people, all you can do is provide an environment where people can change themselves. After all “When a flower doesn’t bloom, you fix the environment in which it grows, not the flower.” I have this ideology too in building & developing test teams. In 2013 I spoke at Tasting Let’s Test about building a context driven test team. In it, I talk about creating a habitat where testers can thrive and grow, with a focus on coaching to support that work. I’m working hard to create a habitat like that in Tyro Payments.


The underlying belief he had was that “people are fine the way they are stop trying to fix them” struck deeply with me. If you start with that premise you come to coaching with a different mindset. And just like I came to Agile Australia with a mindset of learning this year as opposed to speaking, I can approach coaching with intent to listen instead of teach. Instead of focusing on where a person needs to be, you think of were a person is. The spotlight is placed on the person not your perception of that person.  Perhaps it’s subtle differences like this that can make all the difference in coaching. Or maybe not. I don’t know. It’s an experiment, but one I’m performing on myself rather than on other people.


And I think that’s exactly how we all like to learn.

Nice girls don’t get the Keynote

Keynotes are something to be earned and rightly so. Typically a keynote comes from someone who likes to speak, who is good at speaking, and has something interesting to talk about.

I know many women testers who meet the above criteria. They have the requisite ‘merit’ required to be a keynoter. They have thoughtful ideas, are good speakers and are well respected in the testing community.

So why are they not keynoting?

Could it be that to be a keynote takes more than ‘merit’?  Could it be that keynoting is also about your place in the community and your connections within that community?

My experience as a conference organiser and being on program committees is that ‘who you know’ is equally as important as ‘what they know and how they speak’. I don’t apologise for that. It’s important for me to base my decisions not only on an abstract but on a speaker’s reputation.

A keynoter needs to have some of the following:

1) They have interesting topics to talk about
2) They are practiced/skilled at speaking
3) They are involved in some way in the community
4) They network within and outside of their ‘tribe’
5) They make a point of asking for keynotes

I know many practised women speakers with fresh and innovative talks and we are well covered on points one and two.

And we’re not so bad at organising and volunteering for conferences & community either, so….

Could it be, points four and five is what’s holding us back?

Keynotes are typically by invite. That means, the program committee gets to decide who keynotes. Those on the program committee tend to have a good network. It’s probably one of the reasons they’ve been chosen to be on the program.

How are your networking skills?  I’m not talking about card swapping nonsense, but ask yourself: do you have a genuine interest in engaging with people who you respect? Importantly, do you know your network will openly advocate on your behalf? Which brings me to point number five…

Be open to asking to keynote. Is it just me, or is there’s this weird unwritten rule in our community that prevents us asking to keynote? It seems to me that to ‘make it’ we have to Marlene Dietrich like, sit nonchalantly in the corner waiting to be asked for our moment in the spotlight.

Maybe that’s just me and not a general experience. But for those of us who are less direct I’d like to suggest you make it clear to your close network that you want to keynote?

What’s more, go direct to the program committee and ask them for a spot. If the say no, ask why not? and what does it take to keynote,? Who knows you might gain some valuable insights into the process.

Because no matter what skill you have, or how good you are at speaking, or how charismatic you are, there’s dozens of great speakers who can do what you do. It takes more than merit to get noticed. It takes courage to ask and you need allies who will support you along the way. Have you got what it takes?

If you’re reading this post, and you’ve keynoted in the past with a story to share about how you got to keynote, Speak Easy would love to hear it as it may help us understand what it takes to ‘get there’. I’d post a link to your blog post on the Speak Easy website (or I can post a blog).

*Title misappropriated from a book I found useful on this topic “nice girls don’t get the corner office’

Paying Attention to Paying Attention to Detail

One question I often ask a tester in an interview is what they believe are the essential skills that differentiate a good tester from a great tester. One of the common replies is “Paying attention to detail”. But what does that really mean?

Start with detail. What do you mean by detail? Mrs Google offers the explanation “an individual fact or item”, but what sort of detail? Visual detail? Detail from a coding perspective? Any detail or only detail that matter? Is it detail as in ‘small stuff’ as in the Devil is in the detail?

Also what exactly does “paying attention” mean?

Maybe it’s one of these interpretations:

  • A tester is attentive as in a teacher saying “pay attention!”? As in be alert?
  • A tester is only observing detail?
  • A tester is observing detail exists and then noticing they are problems?
  • A tester is noticing detail without a conscious attempt to observe in a methodical manner?

Going deeper, why ‘pay attention to detail’ in the first place?  Is it because you’ve been told to, or because the tester genuinely wants to? Your motivation and purpose play important roles in bug finding. For example what’s your ‘attention to detail’ on being told to complete 40 test scripts in one day, compared to your attention to detail if you get made handsomely for finding a bug or winning a prestigious competition?

Yes, paying attention to detail is important, but so is understanding what you are doing when ‘paying attention to detail’.

In my book, there’s not enough meat on that answer to figure out if someone’s a good fit for my team. I want more. I want my testers to suck the marrow out of that question, to boil it until the flesh falls away from the bone.

Professional testers¹ consciously observes systems using a multiple of models. Their maxim is “the more you see, the more you see”. And that’s only the beginning. Following observation comes evaluation. Now testers get to interrogate information by asking (either explicitly or implicitly) is there a problem? Not content with relying on one oracle they may consciously use many different oracles in their evaluation process.

They then report this information in a way that’s meaningful to the people who want to hear it.

So if you are a  tester coming for an interview on my team and I ask you this question, please have fun with it. Show me your testing process, demonstrate it with diagrams and models and show me a bit of your testing flair. All testers pay attention to detail to some degree. What in particular do you pay attention to?

¹ by professional tester I mean someone who makes an attempt to understand and then improve their personal testing process. They tend to develop a unique approach and style to their testing through years doing, thinking, talking & writing about testing.

This post is a result of a tweet I made which many testers. An interesting aside, my poor spelling was explained away by many as a purposeful trap designed to snare the unwary tester. It wasn’t, though I’m flattered that you would think that of me.

Hi, my name is Done and I’m conflicted

If there is ever one word that highlights the difference between a tester and a developer’s mindset it has to be the word done.

Developers¹ tend think of done as a form of sign off. A story is done when the code is complete. For many developers, it’s when the code is complete, tested and deployed. Done is a sign of completion, of moving onto something else.

Testers² look at Done as the beginning of a quest, an exploration. Done is to be prodded, probed, explored and dissected. Under what conditions can ‘done’ fail? What data tests the limits of Done? How about if two Done’s are put together, how will they behave?

With these two different mindsets it’s no surprise that at times there is conflict and disagreement.

Most testers (most people for that matter) shy away from conflict in order to maintain team harmony. Instead, they try to gain agreement upfront on Done. The goal is clarity and scope definition. Make done clear upfront (before coding begins) and it lessens the conflict later.

In many companies, conflict is seen as a ‘bad thing’. Being in conflict suggests that you’re not a team player. But conflict is a not always bad. According to the 5 dysfunctions of a team by Patrick Lencioni conflict is healthy indicator of trust and ought to be encouraged.

And testers must not forget that our primary role in a team is to raise uncertainty about Done. This has to come ahead of wanting to be everyone’s friend and being accepted. The reality is, if we are doing our job we will be testing the limits of preconceptions of Done.

Because as appealing as it is to think of Done in terms of black and white, zero and one, the reality is a lot different. Done is subtle and muted with hidden dimensions and unknown crevices. Exploring and shining a light³ on these dark corners generates new information. That information helps mould and develop our understanding of Done. The more we test, the better this becomes. Whether this new information turns out to be accepted or rejected is to some extent irrelevant, both contribute to fleshing out our understanding.

So sure, have your discussions on ‘Done’ before code is written, but let’s realise that it’s only the beginning not something final written in concrete. Allow testing and testers to expands that understanding.

In fact, I would go as far to say that true harmony is having a mutual goal of building our understanding of what Done means. Who knows? Maybe through that mutual goal can real trust and respect be built within a team.

¹ ² generalisations

³ kudos James Bach

Take the ‘Crappy Work’ Litmus Test

We’ve placed a ban on crappy work within my test team. From now on, we declare that the testing team will refuse to do crappy work.

The challenge is in knowing how your doing good work or crappy work?  This is why I’ve created this simple 5 minute ‘Crappy Work’™ ² test to help any tester to figure out if you’re doing crappy work or not. ¹

1) Do I fully understand what I’ve been asked to do?

2) Why am I doing this task?

3) Who will benefit, and I have spoken to them ?

4) Am I doing this task simply because its part of the process?

5) If I think it’s crappy work, how can I make it valuable?

I’m sure there are plenty more ‘crappy work’  questions that would be useful to add, but my 5 minutes is up.

Why not share your ‘crappy work’ litmus tests? 

¹ Maverick Tester takes no responsibility if you take this quiz and continue to do crappy work

² It’s a 5 minute quiz because it took me 5 minutes to think it up. It’s trademarked so I can sell you a certificate in passing the ‘Crappy Work’ test, thereby making a whole lot of money out of it.

How will Continuous Delivery affect network traffic?

I was recently in New York and had a chance to walk along the High Line. The High Line is a disused overhead train line converted into a walkway and park. It’s a really lovely walk.

High Line by David Berkowitz

High Line by David Berkowitz

I was interested to learn that the train line had been built in the 1930’s but had become disused by 1980’s. It’s main purpose had been to transport meat and produce to manhattan from the upper west side. With the advent of trucks the train line fell into disuse. It was about to be torn down when a small group of inspired locals advocated it be turned into a park. The movement grew and now the disused train line is a lovely walk and respite from the busy traffic.

I think it’s interesting that the train line fell into disuse in the first place. Why were so many companies eager to drop the train line in place of trucks? My guess is they wanted the ability to freight cargo when they wanted. Rather then delivering in one bulk they delivered smaller amounts more frequently. That way they could become more responsive to their customers needs.

That’s what we’re aiming to do with continuous delivery. We want to   be able to deliver in a faster way, to be more responsive to our customers needs.

7454479488_9cf64433d6_z

World Class Traffic Jam by bk

I wonder if our existing network will be able to handle it though? Look at the traffic jam that is NYC and it makes me wonder that with the ability over faster and more frequent delivery comes the cost of greater traffic on our networks. Remember, if it was just your company wanting to deliver, of course there’s sufficient bandwidth, but when everyone has the same idea will our current network be able to handle it? I  guess only time will tell!