integration tests

Headless browser tests are often slow and flaky. When starting a project in a new or unfamiliar framework, having seemingly straight-forward integration tests in a (headless) browser feels nice and re-assuring. The confidence grows with each written test until some begin to fail without a notable change in the code. As a result, more and more time is lost diving into the errors that come around, and the gained confidence is gone.

It’s easier to just drop headless browser testing at a certain point, and that is what we did. What was left were the interface tests, view tests (including JS), and controller tests. This worked fine except the missing integrations between the forms and controllers. Since they were tested separately, missing an input in a form, the right action or method could still result in bugs.

TestDispatch

For this, I wrote the library TestDispatch. This library currently provides three functions:

  • submit_form/3 - Submit the form based on the attributes and inputs of the form
  • click_link/3 - Triggers the link.
  • follow_redirect/2 - Does a get request to the page that the controller redirects to.

They all take the conn as the first argument and make sure we can write integration tests.

With submit_form/3, we can act as if we fill the form and submit it. The second parameter is a map that takes keys matching the input fields. When the keys match the inputs, the default values are overridden.

The form can be found with either TestSelector or an atom matching the entity of the changeset (i.e., a User changeset needs :user as an entity) as the third argument.

Using this in a test will give the following code.

0
1
2
3
4
5
6
7
8
9
10
11
12

test "create a course", %{conn: conn} do
  params = %{title: "Learn TestDispatch"}

  assert conn
         |> get(Routes.courses_path(conn, :index))
         |> click_link(CoursesView.test_selector("new-course-link"))
         |> submit_form(params, :course)
         |> follow_redirect()
         |> html_response(200)
         |> find_test_selector(CoursesView.test_selector("title"))
         |> Floki.text() == "Learn TestDispatch"
end

In this example, we go to the courses index route, “click” on the link labeled with new-course-link. On the :new page, we select the form we want with :course and enter the title “Learn TestDispatch”. This gets posted to our endpoint, and after creation, it will redirect us to the show page of the course. Where we verify that the title matches our input.

This won’t take the browser validation into account, and changes done JavaScript are also not included in these tests. It’s a tradeoff where we lose some cases but gain speed in executing the tests and have to dive less into headless browser errors, while the confidence in our test suite returns.