PuppetConf 2017

Continuous Integration (CI) testing provides instant feedback as you develop Puppet code. While it’s great that your rspec-puppet and beaker tests pass using fixtures and stubs, you can get a false sense of security from unanticipated edge cases, unrealistic assumptions, or the lack of complete coverage. You may be tempted to address these limitations by writing even more tests, but that would only exacerbate your frustration. Instead, you could use some alternate techniques to make your CI pipeline predict real-world behavior more accurately, and thereby gain confidence in your test results. In this session, we will explore some less traditional yet more powerful techniques for automated testing of Puppet code. Real-time catalog compilation testing obtains more realistic results by simulating your production environment with the latest data within CI. Learn how GitHub scaled this strategy to test hundreds of unique combinations while keeping build times reasonable. Catalog difference testing with GitHub’s open source catalog difference tool (octocatalog-diff) increases development velocity and reduces deployment risk. The speaker, who is the original author of the tool, will explain why and how octocatalog-diff was developed and demonstrate how it can be used to streamline code reviews and validate Puppet upgrades. Real host testing of catalog compilation on actual nodes can exercise external dependencies and custom facts to provide the most accurate results. Overcome the limitations of human-defined fact fixtures and assumptions, avoiding situations that defy fixtures without you realizing it. Throughout the session, pick up other useful ideas, including how and why to run puppet-lint as a CI job, how to test project setup, and how Hubot keeps Dockerfiles up to date at GitHub. You will leave this session with fresh ideas to improve your CI pipeline, which will lead to confident and reliable deployments.