Conducting Effective Exploratory Testing!

With the interest to know more, for the curiosity to discover, we explore… Roads, Code, Apps.. Exploratory testing is all about discovering the unknown. I have written before on why we should conduct exploratory testing.

Let’s discuss what to bear in mind when conducting effective Exploratory Testing!

There are quite a few misconceptions when it comes to how Exploratory testing is conducted. It’s not the same as exploring the roads. We are not wanderers when we explore the app.. It’s not just “Go and explore”!!

Here are various points to take into account for planning & executing the Exploratory test :

Prerequisites :

• Understand the System Under Test (SUT)- the basic knowledge needed to explore the app as a First time user and then add on more & more functional knowledge to find the edge cases. Analyse the features and then you can come up with ideas to validate the same.
• Go through the slides, spec docs, user guides to know how and what the app should do.
• Use your Mind.. it’s like using your brain and not just the scripts. The brain will come up with the ideas from the earlier experience of curiosity.
• Start at an earlier stage of development - for an understanding of “why”, go through why should you for a QA when you have automated tests

How :

• It is better to start with high-risk areas- where User impact is more and thinking how First time user would use it.
• If the first time the user gets a difficult feeling about handling the app, then it may get challenging to make him use the app more. They may prefer the browser if it’s not the banking or social media app.
• Verify all interactions, calculations, what-ifs. Keep on switching across features.
• Try all possible values and navigation user can think of - even though it may be the rarest of scenarios.
• Pretend to be a different kind of user using it and requiring different functionality every time you use the app/software.
• Switch off the configurations settings just randomly like a user may do while installing other apps or not granting the permissions while installing the apps (default, non-default settings).
• Design & execute based on your curiosity about the application
• Track all your doubts and any defects that you may notice or suspect in the SUT.
• Maintain a checklist of what you want to go through for specific areas. This checklist will get updated as you proceed with the feature, based on your experience of exploring the earlier features and behavior of the app/system.
• It’s always a combination of required validation + Personal perception.
• Get feedback from the exploratory test, learn & design.
• Being creative, keep on Discovering new ideas.

E.g. lock the screen of the app whilst in the middle of using it, clear the cache in the middle of the user journey, submit the empty account number, then right number then the wrong number and then multiple characters in the field. For security vulnerabilities in API - use the new search mock-ups, put the SQL injunction right, just with starting tag. Yes, you can do exploratory testing of API as well!

Keep these things handy when you start so that you come up with ideas..

Remember: You are not “breaking the app” to prove the developers to be incompetent, you are simulating real-world actions that users might perform and then complain about.

When :

• Plan your exploratory testing for a specific time period in a day.

Reason being - To make sure you give time not just to exploratory testing but other daily tasks as well. Some such activities would include, converting exploratory test cases to automation, understanding the defects found, doing manual or automation testing. And if you keep on using your brains to generate ideas for 24 hrs, it’s not going to work that well by the end of the day - you will be fatigued. Instead, if you provide just some specific time of the day, your brain/you function faster and effective.

• By the time you are done with this session (exploratory testing in the planned time-slot), you will have bugs and queries and more ideas to start next exploratory with different functionality.

After the exploratory is done :

• Note down your observations and feedback after every exploratory test. These will come in handy when reviewing behavioral changes in the future, after testing.
• Evaluate defects and categories them. For e.g. - edge case scenarios, crashing ones while exploring, design changes, caching, cosmetic, navigation, buttons, call to action stuff, first time user, user with no data, user with lots of data, user logging in at 2 different channels with the same id, switching off the system etc.
• Get this feedback and add the tests in automation. Repeat.

This will help ensure that the identified defects from exploratory get covered in the automation.

Next step :

You develop the next exploratory test based on your feedback, experience and trials from the previous one. (Give an example here. This is perhaps the most important money point for “why does the client need to keep paying for manual testers? Isn’t a one-time activity enough? Can’t we just repeat the same test plan?”)

Here comes the point of why you should keep on exploring as the product matures-

My earlier articles on Exploratory Testing and on what the QA does before,during, and after automated tests are developed cover this point from many aspects that you can use for internal discussions and your own understanding

See, there is a difference between when you learn to drive a car - you follow all the instructions and combination - no talks, no sightseeing etc etc.. Now contrast this with when you have been driving for 2 years. Everything comes automatically and previous instructions you followed doesn’t really feel the need to use them now. Because they are the base and they come automatically, plus your experience of 2 years adds to the way you enhance your driving. Like a lady who is expecting a child, in the first few months, everything gets handled with due care because we don’t know how it’s gonna work.. But by the time passes, we learn, all these months experience make us grow on how we handle the situation more effectively.

Same goes for testing. “Manual testing / Exploratory testing is not required once you launch the product” - that’s what most organisations think. “Put everything in automation and if automation passes, we are good to go live”, they feel.

But then you end up taking the risks which you had not faced earlier because you had the manual/exploratory tests covering the unknown risks. The manual/exploratory tests covered everything for the SUT and we took it for granted and ended up stopping the practice of manual/exploratory test.

Closing Notes:

Exploratory testing is not just to find the defects. It requires QA to use their curiosity, creativity to search, test the light tested area, more from a user perspective and get the unknown covered in test automation.

But most importantly, they validate the business requirement early, see if the outcomes are being met, highlight gaps and realizations, and provide feedback to business even before going for field trials.

Exploratory always can go hand in hand with automation where we use the feedback uncovered in exploratory to feed the automation. By the time you get expert in exploring, you will feel your product talking to you.. Not to make you scared, but then it becomes your routine.

Keep Exploring!!