Becoming a better Tester!
July 11, 2021
⌚ : 4 min
                                     Click to find relevant articles:

Should we check static UI elements with Automation Tests

Recently, I came across a scenario where all the Xamarin forms for the app had an update. This resulted in issues like static text being cut off, not being visible, changes in colors for a few buttons’ state, etc on different kinds of devices.

Will automation catch such changes?

Before we conclude, let’s pause and reflect - “Is this the purpose of automation?” “Do we write automation to detect such issues?”

Here are some reasons for testing static content on a wide range of device sizes :

Will you say that automation is not smart enough? Or are we failing to automate such checks correctly?

I ask - Is there a need?

My Thoughts :

Can we take care of static text checks at the time of development? I suggest this has to happen in-person between designers and developers whenever there are form factor changes or layout changes. The Designer would be a stakeholder who would review the changes on the relevant form factors and accept the changes. Thus, we could have a specific check before the changes are pushed to the production environment.

Why do we automate? Is it just because we can?:

“We automate because we can” - This is a common interpretation of automated test design. Let’s ponder a little on this interpretation:

After automating the static text check, and even if half of the text is not visible, or cut off, or moved below the viewport, the test case is going to pass as the element has all the content.

But then, that UI element that just cleared the test is in fact just not visible to the eyes of the actual user!

I can check for the corners of the box or if the text is outside of the box, but is that what automation is for? What is the aim of automation?

Automation check of static text should comprise the following:

Automation should stick to - checking that UI has received the data, finding the controls and their working condition and clicks to continue in the workflow or to move to the next workflow. This would help separate a rendering check from a functionality check.

How to verify content (static text)? :

There are tools in the market for checking the sizes, shapes, text, colors etc. - but matching may fail because of external reasons. While you could ram up your skills, you would then be depending upon the correctness of that Visual Check tool, and would need to assess it. One time review as part of story review would remove the need for a visual check tool.

Developers can do this themselves while developing and before pushing a commit for the central build. The designers are the stakeholders who are best positioned to review the form factor changes.

We have Acceptance criteria as part of the user story to determine what checks we need for that content or form factor change and the responsible stakeholder for the same.

A static item won’t change between automation runs on a fixed form factor. There is thus no point wasting automation capacity on static item verification.

Connect / Follow me

I work on everything Quality!