Often times when servicing operations departments and getting them used to the idea of using development tools and methodologies to define and implement infrastructure, I get asked the inevitable question:
“Since we’re now using these development methodologies, how do we test locally or iterate, or run unit tests, etc. without affecting our infrastructure?”
This question is bound to come up… it’s unavoidable. We’ve brought Operations folks (sysadmins, engineers, architects, etc.) into the world of the developer with concepts such as revision control, unit testing, lint style checking, and the like. They know from experience that while they have a development environment out there to do development work in, it might as well be production.
Why? Simply, the organization has a number of highly paid individuals that will be sitting on their hands if you make a system unusable for any length of time, and generally speaking, management doesn’t tend to stand for that. Developers have deadlines and sprints; test scenarios that all play into the latest story… you name it. These systems cannot see any amount of unpredictability, and it’s your responsibility as Operations to ensure they continue to, well, operate, so how do we test without affecting our organizational environments?
Vagrant, as their website says:
“…provides easy to configure, reproducible, and portable work environments built on top of industry-standard technology and controlled by a single consistent workflow to help maximize the productivity and flexibility of you and your team.”
What does this mean for you?
In short, you can build a representative environment on your own desktop or laptop that mirrors your production environments for development, deployment testing, workflow organization, and fully vet new changes before they ever see your site “live” environments. Further, they’re easily built, destroyed, and replicated as often as you need with completely free tools easily obtained for your platform.
For instance, I use a MacBook Pro as my primary work computer. On here I have this infrastructure installed with a single environment, which is “mine” (my pet projects, company tools, etc. live here), but I can use the precise same configuration to stand up another environment just like it in less than 5 minutes.
“What sort of environment” you might ask? I prototype a common infrastructure, so when I stand up this new environment for development and testing, I have a Git server, a Puppet Master, and three Puppet clients designated as DEV, TEST, and PROD environments. In this way, when I build conditional Puppet logic destined only for DEV instances, I can test my deployments and ensure it works as was intended before ever committing the code into my company’s revision control servers and designated to be tagged for deployment.
This is highly beneficial to me, as I go from customer to customer and need to stand up “clean” environments for them on a continual basis. You, the Operations person, may instead need to stand up “clean” environments every time you test a new feature, build a new tool, or start a new tag. Using Vagrant can smooth this process for you.
Tools You’ll Need
You’ll need to download a few things to get started. First, you’ll need Vagrant itself. Also, you will need Virtualbox or VMware as your virtualization platform. (I use Virtualbox, so the examples will cover its use), and you’ll need plenty of memory and disk space to hold all the VMs and support files.
Getting Set up
First, we’re going to run a test case to show you the Vagrant mechanism, then we’re going to beef it up to have a full Puppet development environment.
Install the products for your specific platform, and make sure you’ve had no errors. Once they’re in, you’re ready to get started with your first Vagrant instance. For starters, we’re just going to show you the power of Vagrant.
Create yourself a workspace (I use ~/Projects on my system) but you can create a space for yourself anywhere… Since MacOS is a UNIX operating system, I place all my projects in my home directory under “Projects”. Change to that directory and create a new subdirectory to do our work in. I called mine “testcase”.
Change to the “testcase” directory and initialize it for Vagrant’s use, telling Vagrant what sort of VM you want to run (more on that later):
MacBook-Pro:testcase jsheets$ vagrant init hashicorp/precise64
(Note: If you are running a 32-bit machine, replace the “64” with “32”)
This creates a configuration file and work directory in the current location that provides the infrastructure for standing up your new box. Once you’ve created this configuration, Vagrant has everything it needs to stand this machine up, so let’s do just that:
MacBook-Pro:testcase jsheets$ vagrant up
Since this is your first run at using Vagrant, it will download the system image you need (hashicorp/precise 64) and will start the VM. Once all this is complete, you can simply SSH to the VM by typing:
MacBook-Pro:testcase jsheets$ vagrant ssh default
Vagrant has already pre-populated SSH keys for you, done the key exchange, and automatically given you sudo in the new box! As quickly as that, you’ve a new box to do whatever you like.
As easily as you stood the box up, you can also remove it:
MacBook-Pro:testcase jsheets$ vagrant destroy
default: Are you sure you want to destroy the ‘default’ VM? [y/N] y
==> default: Forcing shutdown of VM…
==> default: Destroying VM and associated drives…
At this point, the VM is no longer on your system! Best yet, to do it all again, all you need is to run the “vagrant up” command again, and you get a fresh VM.
If you’re like me, this all seems great, but your head may be spinning and asking questions like can you customize the startup, can you start more than one system at a time, and the like. The answer is a resounding YES! Not only can you automate the provisioning of such a host, but you can also automate the configuration of post-provisioning elements in a number of ways.
For instance, I have an environment that stands up Satellite and configures a couple clients along with it for testing. I have one that builds a Puppet 3.x master, 3 agent nodes, and a Git server. I have yet another that automatically creates me a docker host, an openstack host, and even an Open Source Katello instance. The possibilities are endless, and with a bit of bash and customization of the Vagrantfile, you can model any configuration you may have.
Building your Puppet Instance
Vagrant supports modules, plugins, and the concept of “boxes”, and can auto-provision your environment. An even more convenient feature is you can create and share Vagrantfile configurations with your teammates & colleagues without fear of sharing private company data, while empowering them with the same great tools.
As such, I will direct you to a Vagrantfile that configures the above mentioned infrastructure you can find here: http://questy.org/stuff/Files/Vagrantfile. Place this file in your newly created empty directory (just as above) to hold your test Puppet environment. Next, you will also need two plugins that are referenced inside the Vagrantfile—one for provisioning Puppet and one for provisioning multiple hosts. To install the plugins, do the following:
MacBook-Pro:vagrant-pelab jsheets$ vagrant plugin install vagrant-hosts
MacBook-Pro:vagrant-pelab jsheets$ vagrant plugin install vagrant-pe_build
This will prepare Vagrant to handle the new syntax in this Vagrantfile. Finally, as before, run your “vagrant up” command and watch the fun!
Vagrant will download the appropriate OS images if you don’t have them, download the appropriate PE image if you do not have it, stand up all the VMs specified in the Vagrantfile, install and configure the PE Master and clients, the Git server, and make sure all the appropriate packages are installed.
One added bonus is it’s added port forwarding for you to be able to pull up the PE Console on your local machine at https://localhost:8443 and exchanged all the necessary agent info to have a functioning system.
I’m sure you can imagine a thousand different uses for the system just from this point forward. Share that Vagrantfile with new employees, and they immediately have their own test environment. You can also use the “bootstrap.sh” mechanism and do a lot of post-configuration once the VMs are built to customize the environment to your liking.
The best part is once you’ve completed an iteration, eliminating or “resetting” your environment is as easy as “vagrant destroy” and then “vagrant up” to start all over. You have many other options to you which can be found by running “vagrant –h” to view the command reference.
I hope Vagrant shows promise for your use case. It is a simple and easy way to stand up a testing environment, fully integrated and ready to go for your Puppet testing and development, and to reset those environments again and again without having to work with a full hypervisor, request resources from another group, or re-provision full machines after every deployment.