I am new to doing automated testing of web sites and web apps using Selenium. Perhaps that’s why I was surprised that a simple test that worked with the Selenium Chrome Web Driver was failing with the Firefox Web Driver.

The test

I’ve been working through a book on Flask: Flask Web Development, by Miguel Grinberg. I’ve made it through Chapter 15, which was all about automated testing of a web site or web app, including testing using Selenium.

The tests use the Python unittest library. In the test class setup method, this code establishes a connection to a remote Chrome WebDriver.

url = 'http://crdriver:4444/wd/hub'
capabilities = webdriver.DesiredCapabilities.CHROME.copy()
cls.client = webdriver.Remote(
    command_executor=url,
    desired_capabilities=capabilities)

The test method looks like this:

def test_admin_home_page(self):
    # navigate to home page
    #
    self.client.get('https://tester.example.dev/')
    self.assertTrue(re.search(
        'Hello,\s+Stranger!', self.client.page_source))

    # navigate to login page
    #
    self.client.find_element_by_link_text('Log In').click()
    self.assertTrue('<h1>Login</h1>' in self.client.page_source)

    # login
    #
    self.client.find_element_by_name('email').\
        send_keys('john@example.com')
    self.client.find_element_by_name('password').send_keys('cat')
    self.client.find_element_by_name('submit').click()
    self.assertTrue(re.search('Hello,\s+john!', self.client.page_source))

    # navigate to the user's profile page
    #
    self.client.find_element_by_link_text('Profile').click()
    self.assertTrue('<h1>john</h1>' in self.client.page_source)

That test performs these steps:

  1. Load the home page
  2. Check that it contains content we expect.
  3. Find the Log In link and click it
  4. In the login page, find and fill in the authentication form fields
  5. Find the submit link and click it
  6. In the user page, check that it contains content we expect
  7. Find the Profile link and click it
  8. In the profile page, check that it contains content we expect

I’m not suggesting that is a well-written unit test. The main point is to demonstrate some Selenium capabilities with respect to automated testing.

The problem

That test worked just fine. And it seemed to work every time. But when I executed the same test code with the Firefox Web Driver, the test would fail. Worse, in repeated test runs it would fail at different places.

Eventually, I had the insight that the failures could all be attributed to a test step being performed before the browser and web site had completed their response to a click(). That is, the click() methods were executing the click action, but were not blocking until the response of the system to that action had completed. So the next line of code was often being executed before any new page had been loaded.

Some searches led me to a very helpful post by Harry Percival (author of Test-Driven Development with Python): “How to get Selenium to wait for page load after a click.”

That post confirms that click() doesn’t block and so the next test instruction can be executed before the response to the click() is complete. Furthermore, the post explains why Selenium, in fact, cannot block—Selenium doesn’t know what the response should be! It could be a page load, it could be an XHR call, it could be some script that changes the style of a button.

Therefore, it is up to the test author to change the test so that it detects the completion of the response associated with a given click() and waits for it.

In Python, a good way to do this is with a context manager.

A “wait for page load” context manager

A Python context manager is a class that contains __enter__ and __exit__ methods. When constructed using a with statement, the enter method will be invoked before any of the with block statements, and the exit method will be invoked if the with block statements throw or if they complete normally.

Harry Percival gives an example of a context manager for a page load in his blog post. I’ve taken that as a starting point and ended up with this class:

class wait_for_page_load(object):

    def __init__(self, browser):
        self.browser = browser

    def __enter__(self):
        self.old_page = self.browser.find_element_by_tag_name('html')

    def __exit__(self, *_):
        self.wait_for(self.page_has_loaded)

    def wait_for(self, condition_function):
        import time

        start_time = time.time()
        while time.time() < start_time + 3:
            if condition_function():
                return True
            else:
                time.sleep(0.1)
        raise Exception(
            'Timeout waiting for {}'.format(condition_function.__name__)
        )

    def page_has_loaded(self):
        new_page = self.browser.find_element_by_tag_name('html')
        return new_page.id != self.old_page.id

The idea is that:

  • the __enter__ method will capture a reference to the current page, then
  • the with block statements will execute something that causes a new page to load, and then
  • the __exit__ method will spin wait until a new page has been loaded

This isn’t the most robust code. For example, it doesn’t do something intelligent if the with block statements throw, or if the __exit__ method throws. And it has a hard coded wait timeout of 3 seconds.

But it is a pretty good start. And it worked (see “The Solution”, below).

Note: there may be a better way to do this, especially in Selenium 3.0. If I find one then I’ll update this post to reference that.

The solution

Using the wait_for_page_load context manager around each click() made the test robust for Firefox. Now the Firefox tests work every time.

from selenium.webdriver.support.ui import WebDriverWait # available since 2.4.0

def test_admin_home_page(self):
    wait = WebDriverWait(self.client, 2)

    # navigate to home page
    #
    self.client.get('https://tester.example.dev/')
    self.assertTrue(re.search(
        'Hello,\s+Stranger!', self.client.page_source))

    # navigate to login page
    #
    with wait_for_page_load(self.client):
        self.client.find_element_by_link_text('Log In').click()
    self.assertTrue('<h1>Login</h1>' in self.client.page_source)

    # login
    #
    self.client.find_element_by_name('email').\
        send_keys('john@example.com')
    self.client.find_element_by_name('password').send_keys('cat')
    with wait_for_page_load(self.client):
        self.client.find_element_by_name('submit').click()
    self.assertTrue(re.search('Hello,\s+john!', self.client.page_source))

    # navigate to the user's profile page
    #
    with wait_for_page_load(self.client):
        self.client.find_element_by_link_text('Profile').click()
    self.assertTrue('<h1>john</h1>' in self.client.page_source)

I moved the updated test code to the Chrome test fixture and it also works every time. So no special case, browser-specific, code required. Nice!

Addendum: Firefox test setup

Above, I presented the relevant Chrome test setup code. For the curious, here is the test setup code for a Firefox Web Driver (including specification of a custom CA to authenticate test server certificates, as discussed in a previous post):

url = 'http://ffdriver:4444/wd/hub'
capabilities = webdriver.DesiredCapabilities.FIREFOX.copy()

home_directory = os.environ['HOME']
profile_directory = os.path.join(
    home_directory, 'ff-profile-with-cert')
os.mkdir(profile_directory)
shutil.copy('firefox_trusted_certs/cert8.db', profile_directory)
profile = FirefoxProfile(profile_directory)

cls.client = webdriver.Remote(
    browser_profile=profile,
    command_executor=url,
    desired_capabilities=capabilities)