How I use modelling in software testing

Posted by on January 25, 2013 7 comments

My testing can look like  ‘fly by the seat of my pants” testing at times.  There’s nothing more I like to do than grab a new product and jump straight into testing. No namby pamby reading of requirements by me! No, I want to get straight to the source of truth. I want to find out what the product *actually* does. (This approach may seem seem rash, but its not. Its a considered decision, read on).

But this type of testing only takes me so far. I get to the point where my testing starts to be limited. There seems to be nothing new about the information I’m getting and my learning about the product takes an exponential dive down.

I take this as a sign to stop and take a step back. I’ve obtained as much information as I can from playing around, but now its time to get my hands really dirty. Its time to start studying and researching the product. I start learning more about the products structure such as the database, the products architecture and the interfaces. I explore the intent of the product and find out who the users are. I start talking to product owners & developers to gain information about the product.

I then go back and test more, but this time my testing has taken a different turn.  With new information and ‘new eyes’ I’m looking at the product in a different way. I start learning new things again and the curve of learning and finding new information goes up again.

All this time I’ve been modelling and testing.  In software testing I model a product to understand it. This might be a little different to the way architects model a building. They model to demonstrate to others what the final product will look like, though I can imagine creating a physical model helps to clarify thinking.

Modelling isn’t always explicit. We all have a mental model – a representation of the world in our head and testing makes use of it heavily (Go to a Rapid Software Testing Class taught by James Bach, Michael Bolton or Paul Holland to find out more on this). Sometimes I find it helpful to make the models explicit though. I do this to help me reason through the information. Ordering  information through drawing it or writing it down seems to help me recognize gaps in my thinking.

When I jump in and test, I’m actually creating a model of what the product does. I prefer to model the product first *before* reading requirements etc so I have good understanding of what product really does. My understanding of the product is unfettered by any biases and assumptions that I might gain from speaking to people or reading requirements. Having a solid model of what the product does grounds my testing in the reality of what is. As a tester, I want to bring a different perspective to the table.

Once I’ve modeled what the product does, its time to find out more. I model how people perceive the product to be. I read the requirements and any other documentation I can find. I talk to people and create squiggly and messy diagrams on whiteboards that normally only I can read (and sometimes struggle to understand!)

All the time I’m modelling to understand the product better.

I’m still testing though. I don’t perform modelling in isolation to other cognitive activities. In fact, I test and model, model and test. This might appear counter-intuitive. After all, how can you test without knowing what people want? How will you know if there’s a problem without requirements?

That goes back to oracles (you know, those things that help you recognise problems). When I test, I purposefully use a diversity of oracles to help me recognise different problems. When I use “Plunge In & Quit”* heuristic, I am testing. My oracles of choice are: Previous Testing Experience, Knowledge of Similar Products, World Experience. You don’t have to have explicit requirements to recognise problems.

So I model and test, test and model.  For example, as I’m creating models, I’m testing them. Think of whiteboard scenario where you are formulating models with a developer. As the model is being created, its being tested. That’s how gaps get recognised. When I’m testing, I’m  challenging my models to see if there’s a problem.

I’ve consolidated my thinking on models into this short video.

Here’s what I’ve learned so far:

1) Modelling is integral to testing regardless of it being performed consciously or unconsciously

2) You can have mental, formal and physical types of models.

3) Creating formal models can help reason through a product

4) Modelling a product by first “playing around” can help bias your testing in a good way

5) Modelling and testing take place simultaneously.

6) Different Models are a source of new questions to ask the product

I’m going to wrap up with George Box’s advice:

“all models are wrong, some models are useful”

Creating this model of models has proved useful to me, but its not complete. What are your ideas on modeling and software testing?

*” The Plunge in & Quit” heuristic was identified and named by James Bach. It’s an approach many experienced testers use to quickly learn about a product.

 

 

Courage in Exploratory Testing

Posted by on October 16, 2012 4 comments

Exploratory Testing takes software testing skill. It also requires the tester be courageous. Let me explain.

Exploratory testing is an approach, not a technique. Exploratory Testing is simultaneous learning, design and execution. What information we learn about a product, helps dictate our tests providing us with information that we can share with people around us.

Exploratory Testing is tester centric, meaning the tester is central to the testing taking place. The tester has the autonomy and the responsibility to make decisions about what to test, how to test, how much to test and when to stop. This may seem blatantly obvious to some, but its surprising the number of test teams where this is not the case.

Its all powerful stuff, generating an environment where the tester must constantly reflect upon the changing project and product environments, enabling the tester to be engaged and mindful as they test.

I honestly can’t think of a better approach to software testing.

But there is a catch. Exploratory Testing also demands great courage.

When you start to take responsibility for your testing it’s not done in isolation. Testing requires interaction with many different people such as developers, project managers, scrum masters, product owners, customer support and business people. We share information that’s not always good news. Its in the form of bugs found and the possible impact of this information on business. The resulting consequences of the information we share often leads to delays in releasing, changes in work load, context switching and revising strategies.

In scripted testing, testers have artifacts which they measure and count giving an illusion of of certainty but really this is smoke and mirror reporting and generally offers little genuine information. “We have reached 78% test coverage, with a DDR of 85%”

Exploratory Testing doesn’t have ‘dutch courage’ to rely on. It requires us to have conversations about our information in potentially hostile environments. Sometimes we can feel like the lone fish swimming against the tide of the silent majority. It can be tough and as testers we need to learn to develop how to speak about our testing, how to tell a story (James Bach and Michael Bolton have both written on this). courage in Exploratory Testing

Here’s a list of ways that have helped a quaking knock kneed tester like myself discover her backbone:

Speak to someone you trust about your concern. Vocalisng a fear helps to make it tangible and sometimes gives strength when you discover its a shared concern.

Be coached or mentored on how to speak about testing with confidence

Take small steps. Speak to people sympathetic to your cause, sound out ideas. See if other people can help.

Try not to lose faith, be persistent. Keep your eyes on the goal, even if sometimes you fail to speak out.

Emotions are your toolbox. Anger and frustration can be very useful emotions! Use your emotion to give you courage to speak out. (I learned that at PSL this year..thanks to Jerry, Johanna & Esther)

Sometimes you need help. Be humble enough to know that sometimes change is out of your capabilities. See if you can find help through the testing community or see if you can bring someone in to help affect the change.

But mostly, its about practice. Courage breeds courage. Standing up to little things helps give you courage to stand up to greater things in the future. Be brave. Be strong.

What drives me most of all is that I want to be able to walk away from a situation with my head held high in the knowledge that I may not have changed the world, but I’ve had my say.

Now that’s a good days work.

 

 

 

 

Growing your own

Posted by on September 23, 2012 12 comments

So, I’m sitting here in my office on a beautiful  Australian spring day. The sun is shining brightly, the air is slightly fresh sending wafts of scent from the spring flowers. Its a good time to be alive and its a good time to be thinking of growth and change.

Potatoes in Spring

True to form, I trotted down to the local garden center and bought back a truck load of seeds and ideas on what I can do in the garden. I feel good this year, the potatoes are growing well, and I’ve managed to grow snow peas for the first time. Having invested a good amount of time in the garden, I feel content enough to sit in my office and allow myself to explore ideas on software testing and training.

And I’ve come up with the crazy idea, wonderful idea. For some time I’ve wanted to invest in online training in software testing. Its a model that I think will suit me well. I live far far away from the rest of the world and as much I as enjoy meeting new testers from around the globe, continuous travel is not for me.

To date, I’ve struggled with the concept of online learning. When I learn, I like to get my hands gritty, experience the stuff I’m wrestling with. With my online coaching, I make sure I include a task of some sort but thats one on one. Is it possible to offer online experiential learning to many people?

And then I read about these guys at Venture Lab. The courses are highly experiential & require collaboration to succeed. I sniffed a model that I could possibly work with.

But I’m taking it one step further. I want to make the students the designers of the course. Its the students who will work out what needs to be learned and how that will be achieved, and how they will know they achieved it. There will be some external structure, perhaps in the form of exercises, some philosophies to abide by, but basically, its the students who will dictate the content & the pace. In fact a lot of these ideas are from the coaching model James Bach & I have worked on holding onto the concept that learning requires real desire from the student, and to do that the student needs to dictate the learning (with the teacher offering space and direction to learn).

I see this courses as being a permission giver. We’re so drilled to think of learning as something we have to sign up for, like its impossible for us to learn outside a course. In this way, I’m helping overcome that little hurdle and get into some real meaty learning.

I’m very excited about these ideas and what will come of them. I’m not sure where they will lead and I guess that’s half the fun! If you want to join me on the crazy, wacky journey, feel free to contact me on Skype at charretts, or else add a comment below.

I’ll be adding information on this model as it progresses.