TL;DR: I'm not a huge fan of extensive unit tests for rendered HTML. Despite that, I've put a fair amount of work into some test support code for Phoenix views. Mostly, that was because I thought it would be a good learning experience for me, a novice in Phoenix and Elixir. But I came up with some testing conventions and code that might be useful to others. They might also serve as a model for testing other complex data structures.

Why don't I like automated tests for views?

  1. Nowadays, it's usually very quick to visually check a change.

  2. You should visually check a change, especially because you may have messed up usability in ways that aren't obvious from an automated check.

  3. Given manual testing, the unit tests will likely have a lower return on investment than is usual for automated testing.

  4. Moreover, because most view code is "close to the surface" of the app, there's less chance that bugs will be of the costly "I changed something here and it broke that thing over there" sort. See my 1998(!) paper "When should a test be automated?" for more on these last two points.

  5. View tests are kind of grotty, what with having to grovel through HTML. That means the reduced value comes with increased cost.

I do like two types of view tests. I like tests of helper code (which, in Phoenix, is Elixir that lives in View modules rather than HTMLish text that lives in templates). Because I'm fairly allergic to having conditional logic in templates, that means I write tests that have assertions like "only admins can see this set of buttons".

I also like tests of page structure, particularly of outgoing links. Because of some long-ago conversations with Jeff Patton, I favor visualizing apps as consisting of "places" that the user moves between. It's easy for all the detail of a UI to obscure that, leading to hard-to-navigate programs - especially as page design changes. I have a hope that abstracting the "doors" out of a given place/page will lead to clearer thinking about overall app structure.

(I think the notion of "places" comes from Constantine and Lockwood, but am no longer sure.)

Because I'm fairly obsessive about test readability, wanting to talk about tests in terms of structures has led me to this:

Here are things I find interesting about these tests:

  1. With larger structures (of many sorts, not just HTML), you often want to make N assertions about a particular value. Mentioning that value each time is kind of annoying, possibly obscures that a group of assertions is all about the same thing, and probably makes each assertion just that much harder to quickly grasp. (I suspect readability may be more important for tests than for product code.)

    Some libraries, like ShouldI, make the target of assertions implicit. I don't have any particularly strong objection to that, but it occurred to me that pipelining with |> is so idiomatic in Elixir that I should just use that. All it takes is that each checking function return its first argument when it doesn't raise an AssertionError.

    So that's what I do. I use assert-style tests for single checks but pipelining through functions for multiple checks. Since the pipelined checks might raise an ExUnit.Assertions.AssertionError, I follow the Elixir convention of ending their names with !. I find that a pleasing symmetry: you can imagine every predicate like even? having a corresponding checker, like even!.

    Note: the matches! checker is a macro, so it produces the same (very nice) output as assert x =~ y.

  2. I'm emphasizing these function "checkers", rather than various types of equality and related-to-equality checks, because my use of Midje (a Clojure framework) convinced me comparison of actual to expected values is kind of a historical accident. It's an often-useful special case of applying predicates to all or part of the actual value. (My clojure structure-checking library is all about that.)

    If that's true, I see particular applications developing their own custom set of checker functions or macros that make it simple to talk about what's important in that application's domain or implementation. That should both reduce the cost of tests and also make tests more readable and so less likely to "decay". (I don't know if it's still active, but midje-cascalog was a nice example of that.)

  3. But you wouldn't just have groups of checkers for particular applications. You can also have them for particular application frameworks. Like Phoenix. I earlier showed some tests. The HTML being tested is created by bog-standard eex:

    Notice how nicely Phoenix elides some annoying facts about HTML and REST:

    1. The first link produces an <a> (styled as a button), the second a <form>. Because that detail almost never matters to you, the programmer, it's hidden.

    2. In keeping with REST, the deletion link is written with method: :delete. Because browsers, the real form has to use a method='post' and the _method hack to fake delete.

    Great! But just as Phoenix link lets the programmer-as-coder ignore incidental complexity, there should be specialized checkers to let the programmer-as-tester do the same. That's the purpose of the last five checkers in the original code snippet, repeated again here:

  1. Checkers like the one on line 8 show something else. Given that I'm doing REST here, the "doors" outward from this page lead to resources. I'd rather the tests talk about models/modules and structs than about paths, since the latter are less fundamental. That's why the checkers convert values like OldProcedure or an %OldProcedure{} struct into paths.

(Note: because not everything is a resource, not all paths can be replaced with models. For that reason, you can also supply the names of Router.Helper functions, as in this:

  1. One final virtual of checkers is that they're composable. The three lines of code I just showed you are the body of a function, logout_allowed!, which is itself used in pipelines like this:

Although these are early days, this approach seems promising and congruent with Elixir and Phoenix style. Would love comments. I'd also be happy to see my code tried out by other people, maybe eventually make it into a Hex package. I'd also love to hear that someone else has already improved on, or superseded, this approach.

All of this is from my fairly young eecrit repo. I'll link to today's version of the checker source and tests.

Warning: a lot of the repo code is kind of a mess. As I say, I'm very much still learning.