A CLI version of Google Authenticator—A Unit Testing Success Story

The short story is that I was successful, back in 2013, at creating a Python CLI HOTP/TOPT client. And, three years later, I’ve just published that tool so you can use it too. It’s named authenticator.

I started using Two-Factor Authentication on my AWS and Google accounts back in early 2013. To generate the time-sensitive authentication codes, I ran Google Authenticator on my iPhone. But I was worried that I’d lose access to my accounts if my Phone was lost or damaged. To mitigate that risk I wrote a Python command line version of Google Authenticator, my own clean room implementation of a HOTP and TOTP client. I stored the account secrets in both Google Authenticator and in my bespoke CLI tool on my OS X laptop. That way I had a backup authentication code generator in case I couldn’t run Google Authenticator.

My original implementation, although serving my immediate need, was too crufty to publish. But I’m happy to report that it had a large set of unit tests in place from day one. Those unit tests allowed me to confidently perform significant changes over the years as I intermittently refactored the code into something I could publish and support. I want to share the story of that refactor, to give you some sense of how important those unit tests were, how they paid off the initial investment time and time again.

• Where is authenticator?
• History of refactoring
• The last bug

Where is authenticator?

authenticator is available as a Python 3 package and can be installed in any Python 3.5 (or later) environment using “pip install authenticator”.¹ You can see the README documentation at the PyPI site. The README gives you an introduction into how the tool is used and what it can do. There is plenty of help available from the CLI (start with “authenticator --help”).

The source code is available on GitHub. However, that repo won’t show you the history of all the refactorings I did. The public repo commit history starts with authenticator 1.1.1, the first version I published.

History of refactoring

The first implementation, although it served my needs very well, had some issues that prevented me from sharing it with anyone else – and it fell short of my goal of running on any general purpose computer that could run Python.

That was my first non-trivial Python application. I was still in the process of learning the Python language and the core library. I was taking what I’d learned from years of C++ and C# programming and applying it to Python. To put it succinctly, I was writing C# code using Python.

I did write unit tests as I wrote the code, using Python’s unittest and unittest.mock frameworks. I had been using NUnit and MS Test in my C# work, and saw tremendous benefits from using a test automation framework and pervasive unit testing. There was no way I was going to write new code without doing unit testing. I ended up with over 170 unit tests (currently at 177 I think).

But, once I got it working, I was done. It was good enough to act as a backup for Google Authenticator and so I moved on to the next thing.

Eventually I learned a lot more about Python, and read Python code in other projects, and doggedly read through some O’Reilly books on Python. It became clear that my original project, authenticator was not implemented in a pythonic way. For example, instead of following the PEP8 style guidance, I was camelCasing everything. And I wasn’t using in-source documentation properly.

I had also relied on an external library ‘cryptlib’ for some of the cryptographic functions. There is a package called pycrypto that wraps cryptlib. But starting with OS X 10.10 (Yosemite), pycrypto was not installable–the install failed with compilation errors trying to build cryptlib. The issue was reported but not resolved and remained an issue for OS X 10.11 (El Capitan). I really wanted to move to El Capitan but was stuck back on OS X 10.9 (Mavericks) because that’s the last version that could install pycrtypo on a Mac.

And, finally, I wanted to publish authenticator on PyPI to make it easily available to any system I might be using (a Windows machine at work, a bootable Linux USB, etc.). So I needed to make sure it worked on Windows and Linux, and I needed to standardize it a bit so it complied with PiPY packaging guidelines.

So, the gaps were:

• Bad coding style and non-idiomatic code
• Using broken external cryptographic library
• Not tested on Windows and Linux platforms
• Not cleanly packaged

Coding style—PEP8 and PEP257 guidance

To resolve the style issues, and just poor code, I using some lint packages available for Python. Specifically:

• flake8
• pep8-naming
• pep257
• mccabe

The first three check that the code complies with the PEP8 and PEP257 style guides. Then last one checks for overly complex code.

Needless to say, the lint script reported a huge number, thousands, of issues in both the application code and the unit test code.

I just powered through it. I changed a class at a time, just as a way of partitioning the work. After each class was refactored for style and complexity improvements I ran the full set of unit tests. Any issues found were fixed. Then I moved on to another class; refactored and tested. Repeat until done.

Periodically I’d also rerun the lint script, just to get a sense of my progress, but it wasn’t really practical to start addressing individual lint issues until I’d worked through all the classes addressing the most obvious style and complexity issues.

It took several days of 6 to 8 hours of focused effort. At then end of it I could still read and process the encrypted accounts file that I had been using and maintaining with the original krufty version of the code.

Without that comprehensive suite of 170 unit tests there is no way I could have performed that massive refactoring and still ended up with working software that had no added defects.

Replacing a broken dependency

Even after the style-compliance refactoring, the application still depended on pycrypto and so I was still locked in to an older version of OS X. I sat on that issue for 8 or 9 months until I noticed that Python 3.5 had some of the hashing functions I had depended on pycrypto to provide. So I decided to replace pycrypto with the built-in hashing functions from the Python 3.5 Library; whatever else I needed could be obtained from the cryptography package.

This was a much more modest refactoring effort. Only a few methods and classes needed to be touched. The unit tests demonstrated when and where I got it wrong, and I just addressed each broken test until I had no more failing tests and I was fully confident that I was using the hash functions correctly.

The refactored code continued to correctly process my original encrypted account file. Nice.

Within days I had upgraded my Mac to OS X 11.11.5 (El Capitan). That felt really good … to be free of being locked in to the old Mac operating system.

Packaging and publishing

It felt so good that I decided it was time to share this tool with the world. But to do that I needed clean it up and package it properly.

First step was to reorganized my project source tree. I split things up into three main sub-trees: src (for the authenticator package source code), tests (for the unit tests), and dev (for all the scripts needed to configure the development environment, run tests, build the distribution, etc.). I reorganized everything, ran the linter and ran the unit tests—all was OK.

Next step was to rename the tool. My original name was hotp. But that wasn’t correct (because it is also a TOTP client) and wasn’t very “findable” for someone searching for something like Google Authenticator. So, after checking PyPI for conflicts, I changed the name to ‘authenticator’. That required refactoring some code, tests, and scripts. Once done the linter and unit tests were used to discover and repair any issues.

I then turned to the packaging. That required no refactoring, but it did require (in my opinion) testing on various other platforms. So I copied the code to my Windows sytem and created Powershell versions of all my development scripts. Then I ran the linter and unit tests. No errors. Great!

Confident I had portable code I created the package distribution and uploaded it to the TestPyPI server. I decided to test the distribution on a Linux box. I don’t actually have a Linux system. But I do run Docker from my Mac and so I spun up a Docker container from the Python 3.5 official image and installed authenticator on it and tried a few things.

It didn’t work, not completely.

The last bug

Everything worked fine on OS X and Windows 10. Not so much on Ubuntu or Debian. The install was fine. That looked like this:

docker run --rm -t -i python:3.5.1 /bin/bash
root@c8b205fb2ef6:/# pip list
pip (7.1.2)
setuptools (18.2)
You are using pip version 7.1.2, however version 8.1.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
root@c8b205fb2ef6:/# pip install --upgrade pip
...
Successfully installed pip-8.1.2
root@c8b205fb2ef6:/# pip install --upgrade setuptools
...
Successfully installed setuptools-22.0.5
root@c8b205fb2ef6:/# pip install authenticator
...
Successfully installed authenticator-1.1.1 cffi-1.6.0 cryptography-1.4 idna-2.1 iso8601-0.1.11 pyasn1-0.1.9 pycparser-2.14 six-1.10.0

Then, I just did the example from the README.

root@c8b205fb2ef6:/# authenticator --version
authenticator version 1.1.1
root@c8b205fb2ef6:/# authenticator add Google:example@gmail.com
No data file was found. Do you want to create your data file? (yes|no) [yes]:
Enter passphrase:
Confirm passphrase:
Enter shared secret: xj6p kokw ipvk usc6 bveu sz3b csir xhbu
OK
root@c8b205fb2ef6:/# authenticator list
Enter passphrase:
Traceback (most recent call last):
  File "/usr/local/bin/authenticator", line 11, in <module>
    sys.exit(authenticator_command())
  File "/usr/local/lib/python3.5/site-packages/authenticator/cli.py", line 1241, in authenticator_command
    m.execute()
  File "/usr/local/lib/python3.5/site-packages/authenticator/cli.py", line 1227, in execute
    self._execute_subcmd()
  File "/usr/local/lib/python3.5/site-packages/authenticator/cli.py", line 1181, in _execute_subcmd
    self._execute_list()
  File "/usr/local/lib/python3.5/site-packages/authenticator/cli.py", line 1152, in _execute_list
    self._list_client_data(self.args.clientIdPattern)
  File "/usr/local/lib/python3.5/site-packages/authenticator/cli.py", line 640, in _list_client_data
    cds = self.__cf.load(self.__data_file)
  File "/usr/local/lib/python3.5/site-packages/authenticator/data.py", line 720, in load
    cds = json.loads(plain_text, cls=ClientDataDecoder)
  File "/usr/local/lib/python3.5/json/__init__.py", line 332, in loads
    return cls(**kw).decode(s)
  File "/usr/local/lib/python3.5/site-packages/authenticator/data.py", line 94, in decode
    o = self._decoder.decode(s)
  File "/usr/local/lib/python3.5/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.5/json/decoder.py", line 355, in raw_decode
    obj, end = self.scan_once(s, idx)
  File "/usr/local/lib/python3.5/site-packages/authenticator/data.py", line 78, in _object_decode
    cd = ClientData(**d)
  File "/usr/local/lib/python3.5/site-packages/authenticator/data.py", line 334, in __init__
    self._init_last_count_update_time(kw_args)
  File "/usr/local/lib/python3.5/site-packages/authenticator/data.py", line 237, in _init_last_count_update_time
    t = iso8601.parse_date(v)
  File "/usr/local/lib/python3.5/site-packages/iso8601/iso8601.py", line 190, in parse_date
    raise ParseError("Unable to parse date string %r" % datestring)
iso8601.iso8601.ParseError: Unable to parse date string '10101T000000+0000'
root@c8b205fb2ef6:/#

Oops.

The key bit being:

iso8601.iso8601.ParseError: Unable to parse date string '10101T000000+0000'

So I created an Ubuntu development Docker container in which I could run the unit tests.²

I found and reran the simplest test of the many failing tests. Pretty obvious now, where I should look.

(venv35) root@83d77aea42aa:~/trash/authenticator# ./dev/runonetest.sh
F
======================================================================
FAIL: test_string (tests.test_ClientData.CoreClientDataTests)
Test for __string__().
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/root/trash/authenticator/tests/test_ClientData.py", line 704, in test_string
    self.assertEqual(expected, s)
AssertionError: 'clie[139 chars]ime: 00010101T000000+0000\nperiod: 30\npasswor[31 chars]""""' != 'clie[139 chars]ime: 10101T000000+0000\nperiod: 30\npassword_l[28 chars]""""'
  client_id: 'What.Ever.Dude'
  shared_secret: 'GEZDGNBVGY3TQOJQGEZDGNBVGY3TQOJQ'
  counter_from_time: True
  last_count: 0
- last_count_update_time: 00010101T000000+0000
?                         ---
+ last_count_update_time: 10101T000000+0000
  period: 30
  password_length: 6
  tags: []
  note: """"""

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (failures=1)

The problem is associated with the datetime.strftime() method. It is implemented in inconsistent ways on OS X, Windows, and Linux. On OS X and Windows, the 4-digit year is padded with zeros; so we get this:

python
Python 3.5.1 (default, May 21 2016, 13:01:29)
[GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from datetime import datetime
>>> isofmt = "%Y%m%dT%H%M%S%z"
>>> d = datetime(1, 1, 1, 0, 0, 0, 0).strftime(isofmt)
>>> d
'00010101T000000'
>>>

But on Ubuntu, or Debian, the 4-digit year is not padded; so we get a different result:

(venv35) root@0260c764f9a1:~/trash/authenticator# python
Python 3.5.1 (default, Dec 18 2015, 00:00:00)
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import datetime from datetime
  File "<stdin>", line 1
    import datetime from datetime
                       ^
SyntaxError: invalid syntax
>>> from datetime import datetime
>>> isofmt = "%Y%m%dT%H%M%S%z"
>>> d = datetime(1, 1, 1, 0, 0, 0, 0).strftime(isofmt)
>>> d
'10101T000000'
>>>

To me this seems to be a bug in the Linux distribution of Python 3.5.1. The documentation clearly says:

The strptime() method can parse years in the full [1, 9999] range, but years < 1000 must be zero-filled to 4-digit width.

That applies to strftime() as well, since srtptime() should parse what strftime() produces. There is a bug report, issue #13305, that discusses the inconsistency.

It doesn’t look like something that will be resolved soon, and I don’t want to count on everyone having patched their Python framework (should a fix be issued), so I worked around it in my code. If you are interested in the gory details of that, see this checkin. The fixed version of authenticator is 1.1.2.

I then double-checked the implementation by running all the unit tests on OS X, Windows, and Ubuntu. All was OK, so I published authenticator to the primary PyPI site.³

Another win for unit testing. It is hard to imagine writing software without also writing corresponding tests. Not software that needs to be reliable or needs to last any length of time.