--- title: Packer Image Pipelines description: Introduction Packer is the industry standard tool for managing machine image artifacts. It is also essentially the “only game in town”. It supports a variety of platforms and provisioners for sophisticated management of various sources and builds using HCL2, and… image: https://shadow-soft.com/hubfs/Imported_Blog_Media/Hashi-Shadow-Soft-3.png --- [Skip to content](https://shadow-soft.com/content/packer-image-pipelines#main-content) This is a search field with an auto-suggest feature attached. - There are no suggestions because the search field is empty. [![](https://shadow-soft.com/hs-fs/hubfs/Shadow-Soft-Logo-1.png?width=1878&height=431&name=Shadow-Soft-Logo-1.png)](https://shadow-soft.com) - What We Do Expertise - [Modern Infrastructure](https://shadow-soft.com/expertise/modern-infrastructure) - [Automation](https://shadow-soft.com/expertise/automation) - [Observability & Monitoring](https://shadow-soft.com/expertise/observability-monitoring) - [Cloud Engineering](https://shadow-soft.com/expertise/cloud-engineering) - [Security & Compliance](https://shadow-soft.com/expertise/security-compliance) - [AI Infrastructure](https://shadow-soft.com/expertise/ai-infrastructure) Technologies - [Red Hat OpenShift](https://shadow-soft.com/expertise/openshift) - [Kubernetes](https://shadow-soft.com/expertise/kubernetes) - [Red Hat Ansible](https://shadow-soft.com/expertise/ansible) - [HashiCorp Terraform](https://shadow-soft.com/hashicorp-terraform-consulting) - [Dynatrace](https://shadow-soft.com/expertise/dynatrace) - [Icinga](https://shadow-soft.com/icinga) Solutions [**VMware to Red Hat Migration**](https://shadow-soft.com/solutions/vmware-migration-program) Need a strategic path forward as VMware licensing costs rise? [**IT Operations Automation**](https://shadow-soft.com/solutions/it-operations-automation-program) Transform IT operations into a resilient, cost-efficient foundation [**Unified Observability**](https://shadow-soft.com/solutions/unified-observability-program) Stop firefighting with 6+ monitoring tools that don't talk to each other. - Who We Serve - [Enterprises](https://shadow-soft.com/solutions/enterprises) - [Software Companies](https://shadow-soft.com/solutions/software-vendors) - [IT Solution Providers](https://shadow-soft.com/solutions/it-solution-providers) - Resources - [Client Stories](https://shadow-soft.com/client-stories) - [Insights](https://shadow-soft.com/content-hub) - [Events](https://shadow-soft.com/events) - [Platform Pulse Newsletter](https://shadow-soft.com/platform-pulse) - [Academy](https://shadow-soft.com/academy-home) - About - [About Shadow-Soft](https://shadow-soft.com/about) - [Partners](https://shadow-soft.com/partners) - [Careers](https://shadow-soft.com/careers) - [Newsroom](https://shadow-soft.com/newsroom) - [Contact](https://shadow-soft.com/contact) [Let's Talk](https://shadow-soft.com/contact) ![Search icon](https://5018647.fs1.hubspotusercontent-na1.net/hubfs/5018647/Design/Icons/Font%20Awesome/search.svg) [Let's Talk](https://shadow-soft.com/contact) [All posts](https://shadow-soft.com/content/all) July 5, 2022 # Packer Image Pipelines   Shadow-Soft Team  ·  4 minute read ## Introduction Packer is the industry standard tool for managing machine image artifacts. It is also essentially the “only game in town”. It supports a variety of platforms and provisioners for sophisticated management of various sources and builds using HCL2 and a workflow comparative to Terraform’s. It enables a straightforward and simple process to manage an entire image hierarchy for various platforms, systems, and services. In this article, we will explore pipeline examples for continuous integration of Packer templates and configs and automated builds and publications of image artifacts. We will demonstrate how to achieve this with four sufficiently different yet common CI pipeline tools: Jenkins, Circle, Concourse, and Travis. ## General Design In general, the design for a Packer template pipeline adheres to the following design: - install dependencies (if not using custom build agent) - initialization - validation - format check - build image artifact (if release branch and/or scheduled) This is somewhat similar to the Terraform config pipeline, with the final step of infrastructure management only applicable towards root config modules and the dependency installation being an even rarer requirement. ## Jenkins We can construct a basic Jenkins Pipeline for this example using the [Jenkins Pipeline library for Packer](https://github.com/mschuchard/jenkins-devops-libs/blob/master/docs/Packer.md). The Packer configs and templates are assumed independent of platform and provisioner, and therefore this pipeline does not include credentials retrieval or a custom agent with additional tools. These would both be normal for a standard production pipeline. The pipeline is a basic retrieval, initialization, validation, and artifact build. Without respect to branching, this pipeline executes sequentially due to no potential computational optimizations. `// using GitHub Branch Source plugin` `@Library('github.com/mschuchard/jenkins-devops-libs@2.0.1')_` `pipeline {` `  agent { docker { image 'hashicorp/packer:1.7.10' } }` `  parameters {` `    string(name: 'SCM_URL', description: 'The URL (HTTPS or SSH URI) to the source repository containing the Packer templates and configs (should also contain provisioning and validation support code for the artifact).')` `  }` `  stages {` `    stage('Initialize Packer Templates and Configs') {` `      steps {` `        checkout([` `          $class:            'GitSCM',` `          userRemoteConfigs: [[url: params.SCM_URL]]` `        ])` `        script {` `          packer.init(` `            dir:     '.',` `            upgrade: true` `          )` `        }` `      }` `    }` `    stage('Packer Templates and Configs Validation') {` `      steps {` `        script {` `          // remember template param also targets directories` `          packer.validate(template: '.')` `          packer.fmt(` `            check:    true,` `            diff:     true,` `            template: '.'` `          )` `        }` `      }` `    }` `    stage('Build Image Artifacts') {` `      steps {` `        script {` `          packer.build(template: '.')` `        }` `      }` `    }` `  }` `}` ## Circle Here we see a standard example for managing Packer artifacts within Circle. The assumption here is that the build agent Docker image contains bindings for retrieving secrets, the Packer statically linked executable binary, and Ansible installed via `pip`. We have basic caching for any installations on top of the image during preparation. We also have split jobs for validation on pull requests versus merged. Packer builds are scheduled at the beginning of the month to produce updated artifacts (assumes extrinsic to an immutable infrastructure pipeline or combined with a downstream trigger). `---` `version: 2.1` `defaults: &defaults` `  working_directory: /tmp/project` `  docker:` `  - image: myrepo/myrepoimage:mytag` `  environment:` `    PACKER_LOG: 1` `    SECRETS_REGION: 'us-west-1'` `default_pre_steps: &default_pre_steps` `  pre-steps:` `  - attach_workspace:` `      at: /tmp/project` `  - checkout` `  - restore_cache:` `      keys:` `      - v1-packer-ansible` `  - run:` `      name: initialize packer` `      command: packer init .` `  ...` `  - save_cache:` `      paths:` `      - ~/.cache/pip` `      - /usr/local/lib/python3.9/site-packages` `      - /usr/local/lib/site-python` `      - /usr/local/bin` `      key: v1-packer-ansible` `jobs:` `  validate:` `    <<: *defaults` `    steps:` `    - run:` `        name: validate packer templates` `        command: packer validate .` `    - run:` `        name: check packer formatting` `        command: packer fmt -diff -check .    ` `  build:` `    <<: *defaults` `    steps:` `    - run:` `        name: tasks here to retrieve and utilize secrets for aws` `        command:` `    - run:` `        name: build packer templates` `        command: packer build .` `workflows:` `  version: 2.1` `  validate_template:` `    jobs:` `    - validate:` `        <<: *default_pre_steps` `        filters:` `          branches:` `            ignore: main` `  build_ami:` `    jobs:` `    - build:` `        <<: *default_pre_steps` `        context:` `        - AWS_SECRETS` `        - OTHER_SECRETS` `    triggers:` `    - schedule:` `        cron: "0 4 1 * *"` `        filters:` `          branches:` `            only: main` ## Concourse This example uses the [Concourse Packer resource](https://github.com/mitodl/concourse-packer-resource) that I forked, and then enhanced and updated for MITODL. The pipeline was also constructed for MITODL’s artifact management. The resource is contained inside two Docker images specifically built as Concourse agents; one for validating and another for building. The Packer templates and configs are validated when pushed, and the artifacts are built on a schedule. The validations and builds each execute in parallel relative to each other. ``` ---resource_types:- name: packer type: docker-image source: repository: mitodl/concourse-packer-resource tag: latest- name: packer-builder type: docker-image source: repository: mitodl/concourse-packer-resource-builder tag: latestresources:- icon: github name: packer_templates source: branch: main paths: - src/bilder/images/ uri: https://github.com/mitodl/ol-infrastructure type: git- name: artifact-build-schedule type: time icon: clock-outline source: start: 4:00 AM stop: 5:00 AM days: [Sunday]- name: packer-validate type: packer- name: packer-build type: packer-builderjobs:- name: packer-validate-workflow plan: - get: packer_templates trigger: true params: depth: 1 - in_parallel: - put: packer-validate params: template: packer_templates/src/bilder/images/. objective: validate vars: app_name: consul only: - amazon-ebs.third-party - put: packer-validate params: template: packer_templates/src/bilder/images/edxapp/edxapp_base.pkr.hcl objective: validate vars: node_type: web - put: packer-validate params: template: packer_templates/src/bilder/images/. objective: validate vars: app_name: vault only: - amazon-ebs.third-party- name: packer-build-workflow plan: - get: artifact-build-schedule trigger: true - get: packer_templates trigger: false params: depth: 1 - in_parallel: - put: packer-build params: template: packer_templates/src/bilder/images/. objective: build vars: app_name: consul env_vars: AWS_REGION: us-east-1 only: - amazon-ebs.third-party - put: packer-build params: template: packer_templates/src/bilder/images/edxapp/edxapp_base.pkr.hcl objective: build vars: node_type: web env_vars: AWS_REGION: us-east-1 - put: packer-build params: template: packer_templates/src/bilder/images/. objective: build vars: app_name: vault only: - amazon-ebs.third-party env_vars: AWS_REGION: us-east-1 ``` ## Travis Since Travis cannot use custom build agents supplied by a user like the previous CI pipeline tools, we can demonstrate how to provision dependencies in this example. With an Ubuntu 20 distribution and Python 3.9, we can install the statically linked binary executable for Packer. We then use pip to install Ansible and Boto3. Note that Travis (also Github Actions, by the way) bakes in Packer into their build agent images for some unknown reason, and therefore you must supply a pathing preference for Packer if implicitly executed by other software, or explicitly path to it instead if possible. We have an example here of pathing preference to cover the more difficult scenario. We validate and format check Packer on pull requests and build an artifact on merge. `dist: focal` `language: python` `python: 3.9` `os: linux` `branches:` `  only:` `  - master` `notifications:` `  email: false` `git:` `  depth: 5` `cache: pip` `env:` `  global:` `  - PACKER_LOG=1` `  - PACKER_VERSION='1.7.10'` `  - ANSIBLE_VERSION='2.11'` `install:` `# packer needs an alternate install location that is preferred over other paths since the travis images bake in a version of packer by default` `- mkdir -p /home/travis/bin` `- if [ ! -f /home/travis/bin/packer ]; then curl "https://releases.hashicorp.com/packer/${PACKER_VERSION}/packer_${PACKER_VERSION}_linux_amd64.zip" -o packer.zip && unzip packer.zip -d /home/travis/bin/; fi` `- pip install ansible~=${ANSIBLE_VERSION}.0 boto3` `before_script:` `- ansible --version` `script:` `- if [ $TRAVIS_PULL_REQUEST != "false" ]; then` `    packer validate . && packer fmt -diff -check .;` `  fi` `- if [ $TRAVIS_PULL_REQUEST == "false" ]; then` `    packer build .;` `  fi` ## Conclusion In this article, we explored configurations for four major CI pipeline tools for automated integration, build, and publication with Packer. You should now be able to confidently implement pipelines for your Packer usage. If your CI pipeline tool varies from the four explained above, then it should still be straightforward to transpose the design into a feasible implementation with the provided information. If your organization is interested in robust codified management for images for the servers and containers for your infrastructure and automated pipelines for integrating, building, and publishing your server and container images, then [contact Shadow-Soft](https://www.shadow-soft.com/contact)   Share: [linkedin-in icon](http://www.linkedin.com/shareArticle?mini=true&url=https://shadow-soft.com/content/packer-image-pipelines) [twitter icon](https://twitter.com/intent/tweet?url=https://shadow-soft.com/content/packer-image-pipelines) [envelope icon](mailto:?body=https://shadow-soft.com/content/packer-image-pipelines) ## Is Your Platform Ready for What's Next? Get unfiltered perspectives on modern platforms, automation, and observability. The Platform Pulse newsletter delivers actionable insights to help you build with confidence. [Get the insights](https://shadow-soft.com/content/packer-image-pipelines#popform_hero) ### Subscribe to Platform Pulse One email per month. Unfiltered perspectives on modern platforms, automation, and observability. [![](https://shadow-soft.com/hs-fs/hubfs/Shadow-Soft-Logo-1.png?width=1878&height=431&name=Shadow-Soft-Logo-1.png)](https://shadow-soft.com) Shadow-Soft is a Red Hat Specialized Partner and Dynatrace Premier Partner helping mid-to-large enterprises modernize infrastructure through Kubernetes platforms, VMware to OpenShift Virtualization migrations, infrastructure automation, and observability solutions. Headquartered in Atlanta and serving enterprises nationwide. **Address:** 4501 North Point Pkwy, Suite 110 Alpharetta, GA 30022 USA 🔔 [Subscribe to Platform Pulse Newsletter](https://shadow-soft.com/platform-pulse) [linkedin-in icon](https://www.linkedin.com/company/shadow-soft/) [Follow us on Facebook](https://www.youtube.com/c/Shadow-soft) **Company**   [About](https://shadow-soft.com/about) [Contact](https://shadow-soft.com/contact) [Careers](https://shadow-soft.com/careers) [Content Hub](https://shadow-soft.com/content-hub) [Case Studies](https://shadow-soft.com/client-stories) [Icinga Support](https://shadow-soft.com/icinga) [Privacy Policy](https://shadow-soft.com/privacy) **Services** [VMware to OpenShift Migration](https://shadow-soft.com/solutions/vmware-red-hat-openshift-virtualization-migration) [Red Hat Consulting](https://shadow-soft.com/expertise/red-hat) [OpenShift Consulting](https://shadow-soft.com/expertise/openshift) [Kubernetes Consulting](https://shadow-soft.com/expertise/kubernetes) [Ansible Consulting](https://shadow-soft.com/expertise/ansible) [Observability Consulting](https://shadow-soft.com/expertise/observability-monitoring) [Dynatrace Consulting](https://shadow-soft.com/expertise/dynatrace) [HashiCorp Terraform Consulting](https://shadow-soft.com/hashicorp-terraform-consulting) **Solutions** [VMware Alternative](https://shadow-soft.com/solutions/vmware-migration-program) [Automate IT Operations](https://shadow-soft.com/solutions/it-operations-automation-program) [Unified Observability](https://shadow-soft.com/solutions/unified-observability-program)   **Workshops** [VMware Migration Workshop](https://shadow-soft.com/vmware-migration-workshop) [Observability Workshop](https://shadow-soft.com/observability-tool-sprawl-assessment) [Automation Strategy Workshop](https://shadow-soft.com/automation-strategy-workshop)   Copyright © 2026, Shadow-Soft ![](https://www.facebook.com/tr?id=433738280336898&ev=PageView&noscript=1) ```json { "@context" : "https://schema.org", "@type" : "BlogPosting", "author" : { "@type" : "Person", "name" : "Shadow-Soft Team", "url" : "https://shadow-soft.com/content/author/shadow-soft-team" }, "dateModified" : "2024-08-28T15:38:10.090Z", "datePublished" : "2022-07-05T16:21:07.000Z", "headline" : "Packer Image Pipelines", "image" : [ "https://shadow-soft.com/hubfs/Imported_Blog_Media/Hashi-Shadow-Soft-3.png" ], "mainEntityOfPage" : { "@id" : "https://shadow-soft.com/content/packer-image-pipelines", "@type" : "WebPage" }, "publisher" : { "@type" : "Organization", "logo" : { "@type" : "ImageObject", "url" : "https://shadow-soft.com/hubfs/Shadow-Soft-Logo-1.png" } } } ```