relaxdiego (Mark Maglana's Technical Blog)

Auto-testing code snippets in my blog

Feb 16, 2022
Est. read: 4 minutes

I finally got around to auto-testing the code snippets in my blog posts. As of today, the process of writing a blog post with code snippets goes like this:

Step 1: Run the background services

First I’ll run Foreman locally while in my blog’s dir. Foreman starts up three processes in the background: a Jekyll server, my résumé autorenderer, and a snippets autotester (the topic of this post). Each subprocess’ stdout and stderr are printed by Foreman, with each line properly prefixed such that it says which process printed out which line:

21:01:04 blog.1     | started with pid 88969
21:01:04 resume.1   | started with pid 88970
21:01:04 snippets.1 | started with pid 88971
21:01:04 resume.1   | make: Nothing to be done for 'resume'.
21:01:05 blog.1     | Configuration file: /Users/mark/src/
21:01:05 blog.1     |             Source: /Users/mark/src/
21:01:05 blog.1     |        Destination: /Users/mark/src/
21:01:05 blog.1     |  Incremental build: disabled. Enable with --incremental
21:01:05 blog.1     |       Generating...
21:01:06 blog.1     |                     done in 1.51 seconds.
21:01:06 blog.1     |  Auto-regeneration: enabled for '/Users/mark/src/'
21:01:07 blog.1     |     Server address:
21:01:07 blog.1     |   Server running... press ctrl-c to stop.

Step 2: Write a blog post

I then create a new file under my _posts directory, writing content in the usual way except for the code snippet. For that part, I’m going to write an include directive. For example, if I wanted to include a Python snippet in this post, I’d write the following:

{% highlight python linenos %}
{% include code-snippets/autotest-post/ %}
{% endhighlight %}

That tells Jekyll to include the contents of _includes/code-snippets/autotest-post/ inside this blog post when it renders it in HTML form. If I save the blog post now, the blog.1 subprocess would fail since the code snippet file I want to include doesn’t exist yet. So I’m going to hold off on saving it for now.

Step 3: Create the test for the code snippet

Just because we’re writing code snippets doesn’t mean we abandon our TDD principles. We’re not savages! So then I’m going to write the test for the snippet first and, for this example, it’ll just be:

# _includes/code-snippets/autottest-post/

from s01_hello_world import hello_world

assert hello_world() == "Hello, world!"

When I save this file now, we’re going to immediately see Foreman spew a test failure in the terminal:

22:03:45 snippets.1 | Events: OwnerModified Created AttributeModified IsFile Updated
22:03:45 snippets.1 | Snippet: /\_includes/code-snippets/autotest-post/
22:03:45 snippets.1 | Type: Python
22:03:45 snippets.1 | Tester: autotest-post/
22:03:45 snippets.1 | pwd: /Users/mark/src/
22:03:45 snippets.1 | + python /Users/mark/src/\_includes/code-snippets/autotest-post/
22:03:45 snippets.1 | Hello world!
22:03:45 snippets.1 | Traceback (most recent call last):
22:03:45 snippets.1 |   File "/Users/mark/src/\_includes/code-snippets/autotest-post/", line 1, in module
22:03:45 snippets.1 |     from s01_hello_world import hello_world
22:03:45 snippets.1 | ImportError: cannot import name 'hello_world' from 's01_hello_world' (/Users/mark/src/\_includes/code-snippets/autotest-post/
22:03:45 snippets.1 | FAILED

The key part of the error there is this: ImportError: cannot import name 'hello_world' from 's01_hello_world' which makes sense because we’ve written the tests but we haven’t written the actual code that we’re testing. TDD FTW, baby!

Step 4: Create the snippet

Nothing else to do, right? I’m going to write the snippet file as follows:

# _includes/code-snippets/autottest-post/

def hello_world():
    return "Hello, world!"

When we save that, we see this output from Foreman:

22:14:46 snippets.1 | Events: IsFile Renamed
22:14:46 snippets.1 | Snippet: /\_includes/code-snippets/autotest-post/
22:14:46 snippets.1 | Type: Python
22:14:46 snippets.1 | Tester: autotest-post/
22:14:46 snippets.1 | pwd: /Users/mark/src/
22:14:46 snippets.1 | + python /Users/mark/src/\_includes/code-snippets/autotest-post/
22:14:46 snippets.1 | PASSED


At this point, when I save this blog post’s .md file, Jekyll will happily include the snippet file’s contents into the rendered HTML, neither knowing nor caring that the file was fully tested by this awesome setup. And that’s totally OK because I want to keeps things decoupled this way.

Step 5: Run all tests as a final sweep

Just like the usual TDD workflow, I run all the snippet tests as a final step before pushing my changes:

for snippettest in _includes/code-snippets/**/*_test.*; do script/test-snippet "$snippettest"; done

Right now script/test-snippet only supports Python but nothing’s stopping it from supporting other languages down the line. It only takes a bit of my time and energy.

Talk is cheap. Show me the code!

Amen to that! Alright, if you want to see how this magic is done, you should start with my Procfile since that’s what Foreman reads when it starts up. Check out the command that gets executed by the snippets entry and then just fall into the rabbit hole from there!

Enjoy. :-)

Oh and, of course, all code snippets in this blog post are available on Github!