Co-Written by Andreas Hubert
This is part 3 of a series of articles. For other parts, see the introductory article.
Let’s Implement fetch()
First, let’s write the test:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
@patch('ansible.modules.cloud.somebodyscomputer.firstmod.open_url')
def test__fetch__happy_path(self, open_url):
# Setup
url = "https://www.google.com"
# mock the return value of open_url
stream = open_url.return_value
stream.read.return_value = "<html><head></head><body>Hello</body></html>"
stream.getcode.return_value = 200
open_url.return_value = stream
# Exercise
data = firstmod.fetch(url)
# Verify
self.assertEqual(stream.read.return_value, data)
self.assertEqual(1, open_url.call_count)
expected = call(url)
self.assertEqual(expected, open_url.call_args)
- Line 1 - We patch an Ansible function which we expect
fetch()to call - Line 4 - The url that we will pass on to
fetch(). We’re assigning it to a variable here because we want to verify if it gets passed on toopen_url() - Lines 7 to 10 - We mock the IO object that
open_urlreturns tofetch() - Lines 16 to 21 - We verify if
fetch()returned the correct data and that it called the underlyingopen_url()correctly.
Run the test and see it fail. Let’s now write the code that makes it pass.
First, add this near the top of your firstmod.py right after the first
import line:
1
from ansible.module_utils.urls import open_url
Then, modify your fetch() method to look like this:
1
2
3
4
5
6
def fetch(url):
try:
stream = open_url(url)
return stream.read()
except URLError:
raise FetchError("Data could not be fetched")
Notice that we’re catching a URLError here. Import that class as follows:
1
from urllib2 import URLError
Notice also that we’re raising a custom error class here called FetchError. This is
so that we don’t have to write an except Exception catchall in save_data()
which is poor error handling. So let’s add the following class to the file. I typically
write this near the top, just after the imports.
1
2
class FetchError(Exception):
pass
Run the test again and see it pass.
Let’s Implement write()
Add the following test to test_firstmod.py:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
def test__write__happy_path(self):
# Setup
data = "Somedata here"
dest = "/path/to/file.txt"
# Exercise
o_open = "ansible.modules.cloud.somebodyscomputer.firstmod.open"
m_open = mock_open()
with patch(o_open, m_open, create=True):
firstmod.write(data, dest)
# Verify
expected = call(dest, "w")
self.assertEqual(expected, m_open.mock_calls[0])
expected = call().write(data)
self.assertEqual(expected, m_open.mock_calls[2])
- Lines 7 to 9 - We expect the test subject to use the builtin
openmethod when it writes the file. This is the de facto way of mocking it. - Line 13 to 14 - We check that the test subject opened the destination file for writing.
- Line 16 to 17 - We check that the test subject actually wrote to the destination file.
You know the drill. Run to fail. Now write the code to pass it:
1
2
3
4
5
6
def write(data, dest):
try:
with open(dest, "w") as dest:
dest.write(data)
except IOError:
raise WriteError("Data could not be written")
Like fetch(), this method also throws a custom exception. Add this to your
code:
1
2
class WriteError(Exception):
pass
Run the test again to see it pass.
Almost There!
Let’s run the linter against our module:
$ test/sanity/validate-modules/validate-modules <path to module dir>
That should list a few errors about documentation, examples, metadata, and the GPLv3 license header. Let’s fix that by modifying the top part to look like this:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
#!/usr/bin/python
# This file is part of Ansible
#
# Ansible is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# Ansible is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Ansible. If not, see <http://www.gnu.org/licenses/>.
# Make coding more python3-ish
from __future__ import (absolute_import, division)
__metaclass__ = type
ANSIBLE_METADATA = {'metadata_version': '1.0',
'status': ['preview'],
'supported_by': 'community'}
DOCUMENTATION = '''
---
module: firstmod
short_description: Downloads stuff from the interwebs
description:
- Downloads stuff
- Saves said stuff
version_added: "2.2"
options:
url:
description:
- The location of the stuff to download
required: false
default: null
dest:
description:
- Where to save the stuff
required: false
default: /tmp/firstmod
author:
- "Your Name Here (@yourgithubusernamehere)"
'''
RETURN = '''
msg:
description: Just returns a friendly message
returned: always
type: string
sample: Hi there!
'''
EXAMPLES = '''
# Download then save to your home dir
- firstmod:
url: https://www.relaxdiego.com
dest: ~/relaxdiego.com.txt
'''
from urllib2 import URLError
from ansible.module_utils.basic import AnsibleModule
from ansible.module_utils.urls import open_ur
For an explanation of these strings, see Ansible’s Developing Modules Page.
Run the linter again. BOOM!
Test The Module Manually With Arguments This Time
We can write a playbook that uses our module now if we want to
but I’ll leave you to do that on your own time. For now, let’s
run it using the test-module script that comes with the
ansible repo. Run the following:
$ <path to ansible repo>/hacking/test-module -m <path to first module dir>/firstmod.py -a "url=https://www.google.com dest=/tmp/ansibletest.txt"
Next, run the following to see what it downloaded:
$ cat /tmp/ansibletest.txt
Run the Sanity Tests
It’s a good idea to run the sanity tests every now and then so that you’ll know ahead of time if you’ve introduced some non-compliant code into the source. From your ansible repo run:
$ INSTALL_DEPS=1 TOXENV=py27 ansible-test sanity
If the test fails, you may have introduced some erroneous code. Check the error messages and fix as needed. If you’re sure it’s not your fault, check if you’re working on top of an old upstream commit. If that’s that case, rebase your changes to the latest from upstream and try again.
BONUS: Get some code coverage!
Writing unit tests is great but blindly writing tests is not enough. So let’s see how much of your module’s code is covered. For this, we’ll use Ned Batchelder’s awesome coverage library. To get started, let’s install it:
$ pip install coverage
Next, we’ll use it with nose as follows:
$ nosetests -v --with-coverage --cover-erase --cover-html \
--cover-package='ansible.modules.cloud.somebodyscomputer' \
--cover-html-dir=/tmp/coverage -w test/units/modules/cloud/somebodyscomputer/
By running this single command, you’ll get two things. First is a quick
test coverage summary via the terminal. Second is an HTML format of the
same coverage report with information on which lines of your code has
been executed by your tests. A green line means it’s been executed
(and therefore tested) while red means it was not. Go ahead and open
/tmp/coverage/index.html in your browser and be enlightened!
IMPORTANT: A fully covered module doesn’t automatically mean it’s bug free. But the coverage report is a great way to find out which parts of your code needs some testing TLC.
Holy Smokes!
I hope you’re as pumped as I am for getting this far. You do realize that, in just 3 articles, you went from zero to writing a fully tested Ansible module. That’s quite an achivement so grab a beer (or whatever is your cup of…ummm…tea) and celebrate your awesomeness!
When you’re ready, head over to part 4 where we’ll learn how to submit our code upstream. Alternatively, you can go back to the the introduction if you want to jump ahead to other parts.