Selenium and TLS

(Updated 2017-03-04 to add a table of contents and the preface paragraph.)

If you are going to be running tests by driving web browsers, you should be doing so with HTTPS endpoints—without bypassing TLS authentication. There is not a lot of guidance for configuration a Selenium-based test environment to use HTTPS properly; this post helps close that gap.

• Automated testing of web sites
• The issue
• The solution
• The example
• Conclusion

Automated testing of web sites

The Selenium project¹ provides an excellent way of testing a web site or web app. You can stand up a web site, instantiate one or more browser clients, and then run a suite of automated tests that drive the browser clients to interact with the web site and funnel responses back to the automated tests for review and further direction. You can even use Selenium to do performance or stress testing of a website by directing a large number of clients to the test web site.

All of that works very well—as long as all the endpoints being tested are HTTP endpoints (no TLS required).

The issue

Given that security and privacy concerns require that a production implementation of a web site should be using TLS, and given that the use of an HTTP/2 server and the use of CSP and HSTS headers are emerging best practices, it seems important that the automated testing of the web site also use TLS connections. Aside from the clear benefit of minimizing differences between production environments and test environments, issues related to HTTPS could be uncovered by automated testing—issues that might go undiscovered if HTTP was used in the test environment (instead of HTTPS) or if certificate validation errors were routinely ignored in the test environment.

As of this writing, the Selenium documentation and most examples assume an insecure HTTP connection will be made. There is very little guidance on testing connections made using TLS (that is, HTTPS connections). The bulk of the guidance, and the bulk of online Q&A on the topic, suggests configuring the browsers to ignore certificate errors (something Selenium directly supports in the WebDriver API).

This post and example attempts to address that gap by describing and demonstrating an automation-friendly mechanism for using HTTPS in the test environment with uncompromised certificate validation.

The solution

In a test environment you do not have available the certificates that exist in the product site, regardless of whether the production site is public facing or is an internal site. However, you can provide privately generated and signed certificates to the testing framework such that all test endpoints can provide server domain validation certificates, and such that all test clients can validate those server certificates when making a TLS connection.

The general idea is this:

• Create a root CA certificate to be used in the test environment
• For each host (machine, VM, container) used to stand up HTTPS endpoints in the test environment, create a DV certificate (domain validation certifcate), signed by that root CA
• Use the root CA on the client side in automated tests to validate the certs of the HTTPS endpoints being tested

For some more discussion of that point, and how to implement it, see my earlier post: “Creating a private CA for non-public HTTPS endpoints”.)

That approach makes it possible to keep the client configuration simple and stable, and allows the automatic generation of server certificates on the fly to match server FQDNs that may change from test to test or between one test suite execution and the next.

For Selenium-based tests in particular, it is possible to provision the Selenium nodes with the private root CA certificates needed to validate the server certificates used in the test environment. Different browsers have different mechanisms for referencing the root CA certificates, but that is easily addressed through automation.

The Firefox browser’s use of a per-profile internal trusted root CA certificate store² does introduce one manual step—adding the test root CA to the internal certificate store. But that step only needs to be taken when a root CA is added or retired, something that happens perhaps once a year, and the updated certificate store (the cert8.db file) becomes input for all automated testing until the next time the test environment root CA certificates need to be updated.

The example

I’ve put together a git repo that demonstrates the solution described above. It shows how to use Chrome and Firefox Selenium nodes to conduct automated tests of a website that exclusively uses HTTPS endpoints.

The example³ has comprehensive documentation (via README.md files). You can probably just fetch the repo and work through these README’s:

• README
• helloworld/README
• helloworld-tls/README

Conclusion

There are many good reasons to use valid certificates and HTTPS endpoints in automated test suite, and there is risk to ignoring certificate validation errors in the test environment.

I’ve provided a good example as a guide to how secure endpoints can be tested using Selenium and without compromising on TLS integrity.