Taiko, useful toy for automation testing

Reading Time: 6 minutes

Every day we are implementing new features/client requirements. On every release, we want those changes to be correct, previous features should still be working… with other words we want a stable application. That is why it’s necessary for the BE and FE to write tests (unit and integration tests).

The best way is to have regression end to end automation tests. But it is not always fun to write them. Sometimes it is complex, takes some time so that we are avoiding writing them. Sometimes if a workload is larger it requires a dedicated team with QA to cover all this work. Furthermore to follow all changes and adapt existing tests etc.

There are a few tools that make all this work easier. Browser robots that record actions on the web pages, some frameworks that offer good and easy ways of writing automation tests, but either they are too difficult to learn and sometimes hard to use or they are not for free.

That is why I chose Taiko, a free and open source browser test automation framework that makes all this work easy to do. A few features that are crucial for writing end to end automated tests in my opinion are:

  • Easy setup
  • Interactive recording
  • Smart selectors
  • Easy integration with Cauge

The best way to present all this is to go through some simple examples. I’ll use http://saucedemo.com/ to write a simple test for adding items in the cart.

I want almost everyone to be able to write tests, it does not have to be a complex procedure to set it up. Taiko is a free open source node.js library and it works with chromium-based browsers. Tests are written in JavaScript or any language that compiles to JavaScript (TypeScript).

This means to start to write a test with Taiko we will need a pre-installed node.js. It is a straightforward setup. (https://nodejs.org/en/download/).

For the given example I used a power shell on windows but you can write it in any terminal application that you are familiar with. The command to install taiko is:

npm install -g taiko

After successful installation of taiko we will run REPL prompt:

npx taiko

Here are two important features:

  • Interactive recorder. It means that taiko will archive all successful commands that we are going to write here
  • And the second one is the use of Taiko’s API. We can list all available APIs with command
.api

or

.api <api>

All these API references are online too: https://docs.taiko.dev/api/reference.

Simple example

Let’s write one basic test for http://saucedemo.com/. By writing following commands in the prompt we will assure that saucedemo login, adding a product to cart and basket works:

await openBrowser();

// opens a new browser, I had chromium and it was open without any other setup because it uses Chrome DevTools Protocol instead of WebDriver…

await goto("saucedemo.com");

// navigates to / opens the web page that we want to test

var password = await text("_", below("Password for")).text();

In this line of code we have a few key commands:

  • var password – we will take the data (password) from and use it to log in
  • text – selector – which identifies an element on the page, it will look for text element with text to match with. In our example, it will be “_”
  • below – proximity selector – it makes relative HTML element search. It will search for elements under bellow “Password for” on the page

var usernames = await text("_", below("usernames")).text();
console.log(usernames);

// as it is js we can use this command too; it will be archived. I used it to check the values, it can be removed from the final script

var username = usernames.split("\n")[1];
console.log(username);
var password = passwords.split("\n")[1];
console.log(password);
await write(username, into(textBox({id: "user-name"})));

After the username and password are read from page, we will log in:

  • write – command that types given text into the given or focused element
  • into  – selector for the element to write text into
  • textBox – selector for a text field for input, selecting it with some attribute. In our case, it will be id, but it can be any attribute too
await write(password, into(textBox({id: "password"})));
await click("LOGIN");

// again smart selector, it automatically looks for and clicks button login

Since there are more products, we want to test a specific one, we will use proximity selector to add a specific product to the cart. If we don’t add “toRightOf” it will click the first component with “ADD TO CART” label on it.

await click("ADD TO CART", toRightOf("$9.99"));
await click("ADD TO CART", toRightOf("$15.99"));

To assure that ADD TO CART functionality works, we will check the basket, if the wanted products are there:

await click(link({class: "shopping_cart_link"}));

Assertions are made with every command, looking for any element. For example command

await click("ADD TO CART", toRightOf("$9.998"));

will throw an error

[FAIL] Error: Element with text $9.998 not found, run `.trace` for more info.

but if we want to make some customer check then we can use any node.js assertions:

assert.strictEqual(await text("9.99").exists(), true);
assert.strictEqual(await text("15.99").exists(), true);
await click("menu");
await click("Logout");

With all these commands we created one basic test scenario. All these commands are already archived and we can write them in js file to execute this test anytime:

.code testAddCart.js

And exit the recording session:

.exit

Running our previous test with:

npx taiko testAddCart.js

Other possibilities

Tests can be grouped and run with test runners. There are three that are supported Gauge, Mocha, Jest. Try it with Cauge, it is an easy straightforward procedure to set it up. Like that, by using Gauge, we can integrate these tests to build pipeline in Jenkins.

Conclusion

The setup is simple, very easy and fast.

The interactive way of writing tests, seeing the result of every command in real-time is very good for learning the library. You don’t have to make a write build run, just write it in REPL and that’s it, you see the results.

But the moment of selecting elements on the page was not so satisfying. Smart selectors are not so smart when there are more of the same elements. You have to go again with XPath or class and have to debug the page and check the code for attributes and values.

iOS Unit Tests – My story

Reading Time: 5 minutes

In my last job interview – almost 2 years ago – I received a question about writing unit tests inside the iOS code. With the confidence of a developer with 8 years of experience in the iOS branch, my answer and my opinion at that moment were that if the developers are high-profiled and write good code writing unit tests is not necessary. Also, my answer continued with a conclusion: if the company has a testing team why do we (iOS developers) take their job? Throwing back on the answer and from today’s perspective, my feelings about the topic are mixed: First of all, I’ve learned more about the importance of the unit tests, and secondly, I’ve continued to work in an environment where we are not obligated to write tests.

Some basics

We should consider our code as pieces of code – called UNITS. The purpose of a unit test is to make validation of our code, and this will allow us to be sure that our code meets the design and fulfil the goal.

In XCode and iOS, the Unit tests are written in a separate target. The most important thing is the XCTest framework. The basic rule is that only the methods that start with the word test will be considered as unit tests by the compiler. Here is an example:

func testFormatForCard() {
    let formatter = DateFormatter()
    formatter.dateFormat = “ddMMyyyy”
    let date = formatter.date(from: “28061978")!
    XCTAssertEqual(date.formatForCard(), “28.06.1978”)
}

Once a test method is written with the proper semantic, the method is associated with rhombus sign on the left side:

Unit Tests with rhombus sign

We can run the tests to see if our code is good or if something is not working. If the test passes, the empty rhombus sign is changed with a green checkmark sign:

Figure 2: Test passed

In case of a failing test we have an assertion and a red sign:

Figure 3: Failed unit test

We can see in image 3 that intentionally the programmer entered the wrong output date 29.06.1978. That’s the way we should think when writing unit tests. First, we have to write the failing state and after that, we should enter the expected correct output. If in the second case the test passes, then we have created a useful unit test for that piece of code (unit). The general idea behind is if we change something in our code, and we made a mistake unintentionally the test should fail and warn us to fix the code.

Unit Test in practice

1. Code Coverage

There is a built-in option inside XCode for checking the code coverage of the tests. The ideal scenario is that 100% of our code is covered by unit tests. But is this really necessary? Will we be better programmers if our full code is supported with tests? Can we spend better the time we spent covering the tiniest pieces of code with proper tests?

In my opinion, writing unit tests is a philosophy and knowing the principles lead us to write qualitative code. That’s, of course, our goal as programmers. Covering the most important part of the code, especially the parts of code that are often changed like networking managers, parsers are a better option than trying to be perfect and writing 100% coverage.

2. Test Driven Development

The popular blog website for programmers Ray Wenderlich emphasizes the FIRST principles as a good way to follow writing unit tests. The basics of these principles are that the test should be fast, autonomous, repeatable, the output should be either “pass“ or “fail”, and ideally, the tests should be written before coding – Test Driven Development TDD.

The recommendation by TDD is writing tests before starting to fix a bug in the code. My opinion on this topic is also a mixing approach. Depending on the time you have, if the deadline is not close on the horizon, you can write tests before coding, or before starting to fix bugs in code, but that’s not always possible, and you won’t make a mistake if you skip this step sometimes.

Conclusion

I can say that writing unit tests is good for every programmer on his way to becoming great. The quality of coding could dramatically improve using unit tests. The philosophy of writing tests and thinking how the code should be structured to allow the tests to pass, will make you write cleaner code, better structured, using more protocols, make reusable classes… As in the strategy games, you shouldn’t always be a slave of the principles – the most important is to fulfill the goals in a quality manner. If you have enough time and not a strict deadline of the project you can make a bigger code coverage, but something around 75% is good enough.

Starting with unit testing using Vue.js 2 and Jest

Reading Time: 4 minutes

As a FrontEnd developer, you may know a lot of FrontEnd technologies and frameworks but in time you need to upgrade yourself as a developer. A good skill to strengthen your knowledge is to learn unit testing.

Since I am working with Vue.js for several years, we are going to see some of the basics for testing Vue components using the Jest JavaScript testing framework.

To start, first, we need a Vue.js 2 project created via the Vue CLI. After that we need to add the Jest framework to the project:

# jest unit testing
vue add @vue/unit-jest

I’ll make a simple component that will increase a number on click of a button:

// testComponent.js
export default {
  template: `
    <div>
      <span class="count">{{ count }}</span>
      <button @click="increase">Increase</button>
    </div>
  `,

  data() {
    return {
      count: 0
    }
  },

  methods: {
    increase () {
      this.count++
    }
  }
}

The way of testing is by mounting the components in isolation, after that comes the mocking the needed inputs like injections, props and user events. In the end, comes the confirmation of the outputs of the rendered results emitted events etc.

After that, the components are returned inside a wrapper. A wrapper is an object that contains a mounted component or a VNode and methods to test them.

Let’s create a wrapper using the mount method:

// jestTest.js

// first we import the mount method
import { mount } from '@/vue/test-utils'
import Calculate from './calculate'

// then we mount (wrap) the component
const wrapper = mount(Calculate)

// this way you can access the Vue instance
const vm = wrapper.vm

// you can inspect the wrapper by logging it into the console
console.log(wrapper)

Next step after we do the wrapping, follows to verify if the rendered HTML output of the component matches the expectations.

import { mount } from '@vue/test-utils'
import Calculate from './calculate'

describe('Calculate', () => {
  // Now mount the component and you have the wrapper
  const wrapper = mount(Calculate)

  it('renders the correct markup', () => {
    expect(wrapper.html()).toContain('<span class="calculate">0</span>')
  })

  // it's also easy to check for the existence of elements
  it('has a button', () => {
    expect(wrapper.contains('button')).toBe(true)
  })
})

Then run the tests with npm test and see them pass.

The code in testComponent.js should increment the number on button click so next step is to simulate the user interaction. For this, we need the wrapper’s method wrapper.find() to get the wrapper for the button and then simulate the click event by calling the method trigger().

it('simulation of button click should increment the calculate by 2', () => {
  expect(wrapper.vm.calculate).toBe(0)
  const button = wrapper.find('button')
  button.trigger('click')
  button.trigger('click')
  expect(wrapper.vm.calculate).toBe(2)
})

For asynchronous updates, we use the Vue.nextTick()(need to receive a function as a parameter) method, which comes from Vue. With this method, we are waiting for the DOM update and after that, we execute the code (the code in the function parameter).

// this will not be caught

it('time out', done => {
  Vue.nextTick(() => {
    expect(true).toBe(false)
    done()
  })
})


// the three following tests will work as expected 
// (1)

it('catch the error using done method', done => {
  Vue.config.errorHandler = done
  Vue.nextTick(() => {
    expect(true).toBe(false)
    done()
  })
})

// (2)
it('catch the error using a promise', () => {
  return Vue.nextTick()
    .then(function() {
      expect(true).toBe(false)
    })
})

it('catch the error using async/await', async () => {
  await Vue.nextTick()
  expect(true).toBe(false)
})

Using nextTick can be tricky for the errors because the errors thrown inside it might not be caught by the test runner. That is happening as a consequence of using promises internally. To fix this we can set the done callback as a Vue’s global error handler (1) or we can use the nextTick method without parameter and return it as a promise (2) like we did earlier.

This article is a guide on how to set up the environment and start writing unit tests using Jest. For more information about testing with Vue and using Jest, you can visit the official site for Vue test utils.

Testing asynchronous code in a concise and easy to read manner

Reading Time: 7 minutes

We live in a fast-paced world where a standard project delivery strategy is agile or it is a direction which people tend to follow. If you have been part of an agile software delivery practice then somewhere in your coding career you have met with some form of tests. Whether they might be unit or integration ( system ) or some form of E2E test.

You might be familiar with the testing pyramid and with the benefits and scopes of the different types of tests presented in the pyramid.

Let’s take a quick look at the pyramid:

Unit

As shown on the image above tests that we write are grouped into layers from which the pyramid is built. The foundation layer which is the biggest. It shows us their quantity. Meaning we need more of them on our application. They are also called Unit Tests because of the scope which they are testing. A small unit e.g. an if clause.

Integration/System

The tests belonging to the middle layer are called Integration tests and their purpose is to test integration between one or more elements inside an application and in quantitative representation we need fewer tests of this type than Unit tests.

UI/E2E

The last layer is the smallest one meaning that the quantity of those tests should be the smallest. Those types of tests are also called UI or E2E tests. Here a test has the biggest scope meaning that it is checking more interconnected parts of your application i.e whole register scenario from UI perspective.

As we go from the bottom to the top costs for maintenance are increasing, respectively their speed is decreasing. Confidence is also a crucial part. If a test higher in the pyramid passes we are more confident that our application works or some part of it at least.

Our focus is on the middle layer. So-called Integration tests lay there. As we mentioned above those are the tests that check the interconnection between one or more modules inside an application e.g tests which check that a user can be registered by pinging an endpoint. The scope of this test is to prepare data, send a request to the corresponding endpoint and also check whether the user has been successfully created in the underlying datastore. Testing integration between controller and repository layer, therefore, their name “An integration test”.
In my opinion, I think that tests are a must-have for every application.

Therefore we are writing integration tests for asynchronous code.

With multi-threaded data processing systems and increased popularity of reactive programming in Java, we are puzzled with writing proficient tests for asynchronous code.
Writing high-value tests is hard, but writing high-value tests for asynchronous code is harder.

Problem

Let’s take a look at this example where we have a small system that exposes several endpoints for updating a person. We have created several tests each is updating a person with different names. When a test is running it tries to update a person by sending a request via an endpoint. The system receives the request and returns ok status. In the meantime, it spans a different thread for the actual person update. On the side of the tests, we don’t know how much time does it gonna take for the update to happen so the naive approach is to wait for a specific time after which we are going to verify whether the actual update has happened.

We have several tests which ping a different endpoint. The endpoints are differing in the wait time that would be needed to process each request
updatePersonWith1SecondDelay
updatePersonWith2SecondDelay
updatePersonWith3SecondDelay
updatePersonWithDelayFrom1To5Seconds

In order for our tests to pass, I used the naive approach by adding a function waitForCompetition() which is nothing else than some sleep of the test thread. Thread.sleep() in Java.

Example

The first execution of tests with a timeout of 1 second. The total execution is 4 seconds but not all tests have passed.

The second execution of tests with a timeout of 3 seconds. The total execution is 12 seconds but not all tests have passed.

Third execution of tests with a timeout of 5 seconds. The total execution is 20 seconds where all tests have passed.

But in order for all the tests to pass, we would need a max of 5-second sleep wait which is executed after each test. This way we are guaranteeing that every test will pass. However, we add an unnecessary wait of 4 seconds for the first test and respectively add wait time for other tests. This results increased execution time, hence optimum wait time is not guaranteed.

Solution

As stated in the official documentation Awaitility is a small java library for synchronizing asynchronous operation. Which helps expressing expectations in a concise and easy to read manner. Which is a smart option for checking the outcome of some async operation.
It’s fairly easy to incorporate this library into your codebase.

You just need to add the library into pom.xml:

		<dependency>
			<groupId>org.awaitility</groupId>
			<artifactId>awaitility</artifactId>
			<version>3.0.0</version>
			<scope>test</scope>
		</dependency>

And add the import in your test:
import static org.awaitility.Awaitility.await;

Let’s take a look at an example before using this library:

@Test
    public void testDelay1Second() throws Exception {
        Person person = new Person();
        person.setName("Yara");
        person.setAddress("New York");
        person.setAge("23");
        personRepository.save(person);

        ObjectMapper mapper = new ObjectMapper();

        person.setName("Daenerys");

        this.mockMvc.perform(put("/api/endpoint1/" + person.getId())
                .contentType(APPLICATION_JSON)
                .content(mapper.writeValueAsBytes(person)))
                .andExpect(status().isOk())
                .andExpect(content().string(containsString("Request received")));

        waitForCompletion();
        assertThat(personRepository.findById(person.getId()).get().getName())
                .isEqualTo("Daenerys");
    }

An example with Awaitility:

@Test
    public void testDelay1Second() throws Exception {
        Person person = new Person();
        person.setName("Yara");
        person.setAddress("New York");
        person.setAge("23");
        personRepository.save(person);

        ObjectMapper mapper = new ObjectMapper();

        person.setName("Daenerys");

        this.mockMvc.perform(put("/api/endpoint1/" + person.getId())
                .contentType(APPLICATION_JSON)
                .content(mapper.writeValueAsBytes(person)))
                .andExpect(status().isOk())
                .andExpect(content().string(containsString("Request received")));

        await().atMost(Duration.FIVE_SECONDS).untilAsserted(() -> assertThat(personRepository.findById(person.getId()).get().getName())
                .isEqualTo("Daenerys"));
    }

Example of the executed test suite with the library:

As we can see the execution time is greatly reduced from 20 seconds for all tests to pass in just under 10 seconds.
As you can spot the function waitForCompletition() is removed and a new wait is introduced from the library as await().atMost(Duration.FIVE_SECONDS).untilAsserted()

You can also configure the library using static methods from the Awaitility class:
Awaitility.setDefaultPollInterval(10, TimeUnit.MILLISECONDS);
Awaitility.setDefaultPollDelay(Duration.ZERO);
Awaitility.setDefaultTimeout(Duration.ONE_MINUTE);

Conclusion

In this article, we have taken a look at how to improve tests when dealing with asynchronous code using an interesting library. I hope this post helps benefit you and adds to your knowledge. You can find a working example with all of the tests with and without the Awaitility library on this repository.
Also, you can find more about the library here.