Categories
innovation quality engineering software testing

2020: The year of dumping old heuristics

When software testing, it’s handy to use mnemonics/heuristics to refer to. They can be useful as generators of test ideas, or reminders to test in a particular way, or to consider some aspect.

What’s a heuristic? Think of a heuristic as a rule of thumb that helps you make a decision about a problem you’re trying to solve. How do we get to the fireworks for New Years Eve? Rule of thumb says its easier to take public transport. But that’s not always the case. Maybe with 10 small children, its safer to go by taxi-bus.

Heuristics are great for one you need to make a decision where there’s a whole lot of uncertainty and complexity and you don’t have all the time you want. It stands to reason that the software testing community has been using heuristics since the previous century. In an software engineering profession where complexity and uncertainty rules, they have become a tester’s best friend.

Heuristics help simplify decision-making by providing patterns that can be useful in some situations, some times.

Notice how I put a whole load of provisos in that sentence? That’s because its useful to treat heuristics with a certain amount of disrespect. Heuristics can’t be applied like a checklist, something you diligently tick off like your shopping list at the supermarket. Think instead of a laundry list*, only tick the ones you absolutely need because it’s so damn bloody expensive to get clothes cleaned at a hotel. I mean, sure who wouldn’t love their underpants ironed and pressed, but do you really need it?

Heuristics can be amazing tools but they can also stop you thinking about other options. Many, many testers are familiar with the San Francisco DePOT heuristic. SFIDPOT** is a mnemonic to describe a set of heuristics to identify product coverage. But here’s the thing. SFDIPOT is not necessarily useful for testing stories because it explores at wrong level of abstraction. Stories are not products, so trying to apply that type of coverage to a slice of a product can prove to be an ineffective method of identifying valuable story coverage.

When I work on testing a story, I like to break it down as much as possible. Think of a typical story structure…As <user x> I want to do <some action> so that I can <achieve some outcome>. To break that down I use the following questions:

Heuristics for Testing a Story

Heuristics can be really useful to us in testing. Sometimes they help us think of new ways to test, prompting us to ask new questions about the story (or the product) and the context in which we’re testing.

But they can also bias us, channelling our ideas about testing into narrow ruts that over time make it hard to know any other route.

Here’s some questions to ask yourself that may prevent those soft malleable laundry lists to becoming hard concrete checklists:

  • Ask yourself what has changed outside my frame of reference and how might that impact me and my testing?
  • Ask yourself, given a difference in context, what ideology could I retire?
  • Ask others testing (technical and/or product) what they think are good patterns to investigate
  • Ask team members what product wise concerns them the most
  • Ask team members what’s stopping them from doing their work optimally?
  • Ask your head of engineering what information would they ideally like to have?
  • Ask operations what’s keeping them up at night (literally)

I find that there are consistent themes in quality that most companies wish to adopt and retain. For example, we want products that do what they are supposed to do. We want websites that perform under heavy than usual load. Increasingly we want other quality attributes that are becoming essential to contemporary engineering practises. Security is key. Speed to Market is key. Smaller & key. Adopting newer technologies to mitigate risk is key.

Sometimes, just sometimes, we in software testing love to challenge others in software engineering about handling uncertainty and complexity, but our own behavior fails to reflect that.

How about in 2020 we ask ourselves this question? What ideas from people outside of software testing might I learn about and might be worthwhile embracing? I feel if we can embrace this ideology, then we will bring so much value an benefit into our lovely community.

Anyway, that’s it for 2019, I’m off to celebrate the fireworks. Happy New Year!

*Laundry Lists is an idea I got from Gerry Weinberg

** SFDIPOT comes from work by James Bach and some others I believe, I tried finding it on the satisfice website but no joy. If someone has a link to the original link, please let me know.

Like this post? Here’s a couple of others you may want to read;

By Anne-Marie Charrett

Anne-Marie Charrett is an internationally recognized expert in software testing and quality engineering.
She keynotes at international conferences on the topic of Quality, Coaching, and Leadership.
Ex-Head of Engineering at Tyro Payments where she transitioned testers to a quality coaching model
Consultant on Quality Engineering, developer of the quality operating model. Invented and rolled out a consulting model for quality engineering.
Consulting across FinTech, Media, Government, Insurance, Banking & Telco Sectors
Creator and Lecturer of Enterprise Software Testing course at UTS Australia. Co-developed a coaching model aiming to transfer testing skill and know-how using the Socratic method.
B.Eng (Hons) Electronic Engineering (I really am a quality engineer!)
Based in Sydney, Australia works – internationally.

4 replies on “2020: The year of dumping old heuristics”

Hi Anne-Marie!

Great post about heuristics.

Here is mine regarding the UX for social share buttons on this blog:

Due to the tooltip feature, this post will have low share percentage.

For Facebook, LinkedIn and Twitter buttons, in order to share this great article, you need to hover over those buttons and then click on the tooltip button.
My expectation was that click will directly open the share button. I even explored the javascript console when nothing happened after I clicked on the button.

It took me some time to figure out this feature. I am very satisfied with following WordPress plugin for social sharing:

https://www.addtoany.com/

Thanks!

Regards, Karlo.

Great post Anne-Marie!

As a long-time fan of your work, I was reading this due to this strange staying-at-home time and I wanted to read something inspiring. Not only inspiring, but you made me go all sleuth for historical information, so thank you for that!

>SFDIPOT comes from work by James Bach and some others I believe, I tried finding it on the satisfice website but no joy. If someone has a link to the original link, please let me know.

The reason you didn’t find it is because the original article called it SFDPO. The ‘T’ was added in 2005/6
http://www.developsense.com/articles/2005-10-ElementalModels.pdf
http://www.developsense.com/articles/2006-05-TimeForNewTestIdeas.pdf.

The reference to the original article can be found from http://satisfice.info/articles.shtml:
“How Do You Spell Testing: a Mnemonic to Jumpstart Testing
First Published: http://www.stickyminds.com as column feature, 5/8/01
Learn SFDPO, “San Francisco Depot”, and use it analyze products to plan testing from five different angles”

Oh yeah, and the ‘I’ was added much later, and the reasoning for that is documented at the end of https://www.developsense.com/blog/2014/07/how-models-change/.

I hope this helps! At least for me it was really entertaining to figure out a good answer to your request.

Best regards,
Jari

Leave a Reply

Your email address will not be published. Required fields are marked *