Tuesday, May 10, 2016

Testing Puppet Code: Bundle, Rake and Gems. Oh My! Part 3

This is the third post in a series on testing Puppet code.  The first two posts explored why we should test code and how to use Puppet's 'puppet parser validate' and rodjek's 'puppet-lint' commands.  In this post we'll set up the Ruby framework needed to run 'puppet parser validate' and 'puppet-lint' via the Ruby gem.  The Ruby framework will also allow us to add unit tests with rspec.

A collection of Puppet code, tests, and associated templates and files is called a Puppet Module.  All of those files should be in a single directory named 'your_puppet_module'.  Your Puppet manifests should be in a directory named 'manifests'.  See the Puppet documentation for more information on Puppet module layout.  Although not required, if your Puppet code is not currently in a module format you should try to get it into a module format now.

Example:
- your_puppet_module
|
| - manifests
  | - init.pp
  | - another_manifest.pp

Before following the examples please install ruby and bundler on your system.

The Gemfile and Rakefile


We'll use Ruby's 'rake' command to run puppet-lint.  'rake' is a Ruby tool used to run repetitive tasks, such as tests.  'rake' uses a special file named 'Rakefile' where we configure the tasks we want to run.

The 'Rakefile' can require other Ruby gems to run the tasks we write.  We will use the Ruby dependency manager bundler to manage the Ruby gems our 'Rakefile' needs.  'bundle' uses a configuration file named a 'Gemfile'.  The 'Gemfile' lists the Ruby gems and version.

Our sample Puppet module has a 'Rakefile', 'Gemfile', and a manifests directory with a sample manifest.  The 'Gemfile' contains our two dependencies: 'puppet-lint' and 'rake'.  The 'Rakefile' requires the 'puppet-lint' gem.

Clone the sample repository and look at the 'Rakefile' and 'Gemfile'.
git clone https://github.com/mmarseglia/testing_puppet.git

Running puppet-lint


According to the 'puppet-lint' documentation all we have to do is run 'rake lint' after the module is installed.

Clone the sample repository.
git clone https://github.com/mmarseglia/testing_puppet.git
Enter the directory and checkout example1.
cd testing_puppet
git checkout example1 
Use bundler to install required Ruby gems in a directory local to our project,
bundle install --path vendor/bundle
Run the puppet-lint task on our Puppet code. We prefix the 'rake' command with 'bundle exec'.  This runs 'rake' in context of the gems we installed.
bundle exec rake lint

Congratulations!  We just ran puppet-lint as a rake task and it checked all the Puppet manifests in our project.  puppet-lint found some problems but those files aren't part of our project.  We'll need to configure puppet-lint to ignore that directory.

Checkout the tag named 'example2'.
git checkout example2
In example2 I've made two modifications to the Rakefile:
  1. Clear any previous definition of the rake task named 'lint'.
  2. Configure puppet-lint to ignore vendor directory

Run the rake task again and puppet-lint is no longer checking files in the vendor directory.


puppet-lint can be configured entirely in the 'Rakefile'.  You can still use puppet-lint's configuration file named .puppet-lint.rc when puppet-lint is called as a rake task. 

Running puppet syntax checks


Adding Puppet syntax checks to the 'Rakefile' is just as easy.  Let's look at example 3 to see the changes.

Checkout the tag named 'example3'.
git checkout example3
We need two new Ruby gems: puppet-syntax and puppet.  The puppet-syntax gem has a rake task named 'syntax' that will check our manifests and depends on the puppet gem.  The Ruby gems are added to our Gemfile.


We require the puppet-syntax gem in the 'Rakefile' to be able to run the 'syntax' task and, like puppet-lint, we'll configure it to ignore the 'vendor' directory.



Run the rake command to check our manifests for any syntax errors.  No errors!
bundle exec rake syntax 

What if we had errors in our code?  What would that look like?  I've inserted an error into example 4 so we can see what happens.

Checkout the tag named 'example4'.
git checkout example4
 Run the 'rake' command to check our manifests for any syntax errors.
bundle exec rake syntax

Puppet could not parse our manifest, init.pp, due to a missing curly brace at the end of the file.  The syntax check will tell you the line number, the file and error message.  The error message is the same if we run the 'puppet parser validate' command.


The puppet-syntax rake task will also check templates and hiera YAML files if present.  You can read the full documentation for puppet-syntax on GitHub.

Conclusion


In the previous blog posts I demonstrated running 'puppet-lint' and 'puppet parser validate' on the command line.  Many shell scripts take advantage of those command line utilities and I relied heavily on scripts when I started.  The command line utilities may have a place in your environment but advancing to the next level of testing with rspec and beaker requires laying a foundation.

We started with our Puppet code and gathering our manifests, templates and file into something more resembling a Puppet module.  Then we install ruby and bundler in our development environment.  After bundler is installed we add the 'Gemfile' to our project to manage our Ruby gem dependencies.  Lastly we create a 'Rakefile' that defines the rake tasks we will repeatedly run to lint and syntax check our Puppet code.  Now we have a project that can be easily added to a continuous integration system and can start building a software development pipeline for our Puppet code.