This is a Work In Progress (WIP) post. I explain what I’m working on that is related to automating software development, or related to software development in general.

First, that performance issue …

In the last post I noted that I saw the Terminal process on my Mac jump up to 100% CPU during a docker build while the sinatra gem was being installed. After finding the time for a closer investigation, I find that it is not Terminal that was using all that CPU but rather the VBoxHeadless process (i.e., the guest VM process).

I use an application called iStat Menus to provide system and device status information in the OS X menu bar. The CPU utilization menu widget will list the top 5 processes consuming CPU and it was report Terminal at 100%. But that was somehow inaccurate. When I use the OS X Activity Monitor (something like Windows Task Manager) I see the real metrics, which does show the guest VM consuming all that CPU, not my Terminal session.

So, there is no performance problem. At least not the one I had identified.

My Docker recipe repository

I’m keeping a record of what I do as I’m learning to use containers. It’s in a GitHub repository JeNeSuisPasDave/docker-recipes. Currently the t-dev/llvmrecipe branch is where active development is going on.

That branch name reflects my initial idea to put the LLVM tool chain on a container and use that container for experimenting with LLVM and clang. It quickly became evident that was too ambitious and I needed to start smaller, and so I did, trying many simple scenarios.

I’m not going to go into every recipe. But the example from the last post is there, in recipes/Docker Demo - simple web app. And the second example from the Lees-Miller gentle introduction to containers and Docker is also there, in recipes/Docker Demo - latex runner.

Need for local package sources

While going through these examples and trying other experiments, I noticed a couple things:

  1. The network and remote packages sources are getting a workout.

    Creating well-formed containers typically requires attempting many variations of the Docker file. Running docker build a lot will often cause the same packages to be repeatedly downloaded from remote package sources.

  2. There’s some awkwardness when working with Docker on Windows. For example:

    • The Docker container is Linux and expects Unix line endings (lf). But on Windows the local files that might get COPY-ed into the image may have Windows line endings (crlf), and that can cause breakage within the container.
    • Local files COPY-ed into the image will have “-rwxr-xr-x” permissions in the container.

I also ran into a problem where some Ruby gems could not be fetched from a remote package source. This was almost certainly a problem with the network at my office (where I’m using Windows 10), but does cause me to be concerned about relying on remote package sources.

That issue and issue #1 above led me to decide to run my own local package sources for Ruby Gems and Node packages (and probably Python packages). By running local package sources I can avoid over-utilization of the Internet connection. I can also be much more certain of the availability of the package sources. With local package sources I think building and using container images will be much more reliable.

Unix line endings

Issue #2 is a problem only when running the Docker client on a Windows system. If your Dockerfile COPYs files from your local system into the Docker image, you’ll get a warning like this (on a Windows system):

SECURITY WARNING: You are building a Docker image from Windows against a non-Windows Docker host. All files and directories added to build context will have ‘-rwxr-xr-x’ permissions. It is recommended to double check and reset permissions for sensitive files and directories.

I handled this in the docker-recipes repository by configuring a .gitattributes file in the project to force all files to use Unix line endings. And I also created a bash script to reset the file permissions to the expected value; that script is RUN after the files are COPYed.

I use Sublime Text 3, so I also configured the ST3 project file to enforce unix line endings.

Next time

In the next post I hope to demonstrate a Ruby gem package source server running in a container.