Elementium: Browser testing done right

Browser testing is something that most large web applications have to go through. Most small projects start out with manual testing (i.e. you deploy and then click around the app to make sure nothing is broken) and then progress to automated testing that codifies these tests and automatically runs them on each commit (or you can start them manually before each commit/merge, if there is no continuous integration server setup). One of the major players in providing the framework necessary to perform automated testing is Selenium. More specifically, in the world of python, this is the Selenium python bindings. Note, this entire post, and elementium, is a python library. As such, if you are using a different language, elementium is not for you. Similarly, when I refer to "Selenium" I am referring specifically to the Selenium python bindings.

Everyone's reaction to Selenium is the same. First, it's a sense of awe for how it can navigate around a website, enter text, and click on buttons. This is soon followed by frustration when finding out that unlike most code that has been written to develop the application, testing in browsers is particularly fickle; widgets don't load in the same number of seconds in different runs, and elements become stale when the DOM updates. This leads to a state of despair and generally results in brittle testing code that is littered with time.sleep() calls and with ugly retry logic. This was exactly my progression through the journey of automated browser testing. Fortunately, it didn't end there.

Before I continue, let me be very clear in saying that I am eternally grateful to the folks that created and maintain Selenium - I could not do any of the testing that I do without it. It is a fantastic library that does a fantastic job. It just happens to have been created in an era when dynamic pages littered with AJAX calls was not the norm. Any of this work or potential "criticisms" are meant to be taken with this in mind -- without Selenium, we wouldn't even be this far.

If one takes a look at what people try to accomplish with automated browser testing it is:

• Select browser elements, and perform actions based on selected elements
• Assert to insist that particular elements exist

That's about 90% (if not more) of the work. Unfortunately, the base Selenium library does a great job at this if the page is not changing. I.e. as long as you don't have a dynamically loading page, that has widgets appear and get populated asynchronously, the core Selenium driver will serve you well. However, if you have such a dynamic page (or a web application that is completely written in JavaScript using something like Backbone.js), you will (or have already) run into two major obstacles:

1. Waiting for an element to appear on the page
2. Handling StaleElementExceptions

The current state of the art solution to this problem (as proposed by the hosted testing service SauceLabs) is to use something called "spin" methods. For example, if you need to assert that a certain element exists on a page, you should use a spin_assertthat spins and waits until the element appears. Here is a modified version of what they present (but the basic idea is exactly the same):

def spin_assert_equal(element, assertion):
    for i in xrange(60):
        try:
            # Re-get the element from the page via the lambda
            # and assert they are equal
            assert element() = assertion
            return
        except Exception, e:
            pass
        sleep(1)
    # If we get here, give it one more try, or make it raise an
    # AssertionError
    assert element() = assertion

# Create a lambda that finds the element
element = lambda: selenium.find_element_by_id('foo').text

# Try the assertion
spin_assert(element, 'FOO')

For full details on their method, check out their post here.

Does the above method work? Of course! But let me ask you this. Which of the following do you think is easier to read, understand, and maintain?

# Option 1: Pure selenium
element = lambda: selenium.find_element_by_id('foo').text
spin_assert(element, 'FOO')

# Option 2: Elementium
elements.find('#foo').insist(lambda e: e.text() == 'FOO')

I'm hoping that you say Option 2. Taking cues from jQuery, Elementium allows you to chain commands and handles all of the automatic retrying for you. For example, you could do something like this if you really wanted to:

elements.\
    find('.foo').\
    filter(lambda e: e.text().startswith('a')).\
    until(lambda e: len(e) == 2).\
    foreach(lamba e: e.click())

This here will find all elements with the CSS class 'foo', filter it down to the elements that have text starting with 'a', insist that there are exactly 2 of them, and then click on them. Yes, all of that in 1 line of code. Oh, and did I mention that this handles all of the retry logic automatically for you? (Btw, you can use the click() method on a list of elements as well elements.find('.foo').click())

How about if you want to wait until there are exactly 3 elements of this type on the page (because, for example, you have made some AJAX calls that creates three notifications)

elements.find('.notification').until(lambda e: len(e) == 3)

That's it. This will retry (for 20 seconds by default) and wait until there are three elements with the CSS class 'nofitication'.

"How does it do all this magic," you ask. I won't go into too much detail here, but under the hood, each selector (e.g. '.foo' or '#foo') is stored as a callback function (similar to the lambda: selenium.find_element_by_id('foo') of the first example. This way, when any of the calls to any of the methods of an element has an expected error (StaleElementException, etc.) it will recall this function. If you perform chaining, this will actually propagate that refresh (called update()) up the entire chain to ensure that all parts of the call are valid. Cool! Ok, and now to a "full" example that shows you exactly how to set everything up so that you can start using this today.

from selenium import webdriver
from elementium.drivers.se import SeElements

# Initialize the elements wrapper 
elements = SeElements(webdriver.Firefox())

# Do cool stuff
elements.find('#foo')

It's as simple as that!

Although this library is still under development, you can get this library and make use of it now by getting it from this public GitHub repo: https://github.com/actmd/elementium. There you will find more usage examples, the code, etc. We have been using it at ACT.md for the past 3 months and it has reduced our testing code, made it more stable, and made it much more legible. I'd say that's a win, win, win situation!

Don't hesitate to let me know if you have any questions, suggestions, etc. Happy testing.