Gene will share his favorite principles and prescriptive patterns from the upcoming "DevOps Cookbook," coauthored by Patrick DeBois, Damon Edwards, Jez Humble, Mike Orzen, and John Willis. Our goal is to create a prescriptive playbook that organizations can follow to replicate the extraordinary culture and outcomes enabling IT Operations can operate at scale and win in the marketplace.
This talk will be a straight-dope report of our real-world experiences using Puppet to manage our infrastructure over the last 3 years. We run Puppet in masterless mode to manage the environments that support our payments processing platform . Our domain provides especially stringent requirements when it comes to security and availability, and I'll be speaking about our practices around Puppet that support these requirements. When you're in the trenches, it's not always pretty. I'll spend some time talking about the spectacular ways we've screwed up with Puppet over the past few years. I'll link each of those war stories with lessons we've learned about good Puppet practices. I'll provide a few examples of where we feel like applying these lessons have yielded some cool modules (which will hopefully be open sourced and pushed to the Forge by PuppetConf). There will be no stunning conclusions. As I get up to the more recent lessons we've learned, I'll talk about a few of our unsolved problems, where we run up against the edges of what Puppet itself is meant for as a tool. I'll talk about unanswered questions around where truth should be managed, and outline the ideas we have about where we're headed. In the end, I'll be plugging for the Puppet community as a place for us to collaboratively solve the Hard Problems of managing infrastructure.
""#monitoringsucks."" I can appreciate the sentiment. I used to use Nagios too! However, I can't agree that monitoring sucks. Monitoring is awesome! We observe our systems to understand their behaviour. We do this in various ways like reading logs or taking measurements and, more recently, storing them in a timeseries database such as collectd or graphite. However, the standard practice for alerting is still to check the measurement at the time that it is taken and it is this ""check script"" model of monitoring that is long due for an overhaul. In this talk, I'll start over from first principles: what do we want monitoring to do for us? I'll deconstruct the ""check script"" and rebase its essentials on the humble timeseries. I'll demonstrate simple aggregation and apply some maths and stats to show how monitoring can scale to cluster size without increasing maintenance costs. With worked examples based on real-world situations, you'll learn techniques that you can use to make your monitoring systems more usefu
At Puppet Labs we try to do work that directly improves the day-to-day lives of sysadmins. One of the most important ways we do this is with a team of designers connected deeply with our users and with our engineers. I'll talk about how and why we do that. I'll also talk about our existing products and the work and workflows they encourage and (sometimes) make too difficult. What are we proud of and want to continue? What do we n
In a previous life I ran a large Puppet Enterprise install on high hundreds of physical boxes and 1000's of VMs. We had a 90-95% internally written module base. Our problems mainly revolved around our multiple OS provisioning workflow and scaling the infrastructure. Several months ago I left to join a very small startup. I left with nothing, no modules, no configs and arrived to nothing, no servers, and no time to setup infrastructure. At that point I realized starting with Puppet can be overwhelming. Where do you start? Should you use OSS or PE? Should you get the bits from PuppetLabs or the distro? How do you bootstrap? What about PuppetDB? Have you heard of Hiera? What about that best practices? How am I going to have time to write all these modules? Take a deep breath and just get started. This talk covers my approach. Starting with a target environment of 20 servers and planning for growth. We cover the setup I have now and why I made the choices I did: Puppet Enterprise, PuppetDB, Hiera, Razor, mostly forge based modules. I'll also cover tools I use outside of the Puppet environment to tie it all together FPM, Pulp, Vagrant, etc.
Everyone hears about automation of OS installs and configuration of services on top of the OS yet no one talks about automating the configuration of firmware. There are a number of venders who provide tools that allow the intrepid systems administrator to never again have to arrow through BIOS menus to toggle some less known but needed setting. Gone are the days of manually entering disk configurations into a RAID controller. This session is for SysAdmins or hardware geeks that want to free up their time to enjoy the finer things in life. You will be presented with the knowledge gained from two years of experience managing firmware configuration as a part of hardware discovery and provisioning.
Ops at GitHub has a unique challenge - keeping up with the rabid pace of features and products that the GitHub team develops. In this talk, we'll focus on tools and techniques we use to rapidly and confidently ship infrastructure changes/features with Puppet using Puppet-Rspec, CI, Puppet-Lint, branch puppet deploys, and Hubot.
Testing Puppet is a unique level of difficulty in that we must manage multiple platforms, test client/server behavior, multiple versions of ruby and maintain a large infrastructure environment all without the help of our favorite system admin tool. How do we do it?
Learn about the Puppet Forge, get insight into product development at Puppet Labs and hear what the brand new Forge team has been up to lately.
Building out Infrastructure with the Puppet Forge + Module Writing Tips & Tricks
Ryan will demonstrate several things you can do with Puppet straight from the Puppet Forge and will discuss some things that might help you in creating your next Forge module
In this session, Morphlabs will explain how we use Puppet as a global service to monitor hundreds of disparate OpenStack deployments around the world. mCloud Optimum, Morphlabs global remote service, is powered by Puppet Enterprise. It maintains configurations and redefines what customers expect from a converged cloud infrastructure company.
At VMworld® 2012, Puppet Labs, announced its integration solution for VMware vFabric™ Application Director™, a cloud-enabled application provisioning solution that simplifies how to create and standardize portable application deployment topologies across cloud services. In this session we will showcase the Puppet integration service, which is currently available for download in the beta release of the VMware Cloud Application Management marketplace. It enables VMware vFabric™ Application Director™ customers to use Puppet's configuration language to describe their infrastructure. The solution also gives users the option to leverage Puppet Forge modules as building blocks to create services for VMware vFabric™ Application Director™, further accelerating the deployment and management of applications on VMware infrastructure.
In the four years that I have been on the Fedora Infrastructure team, the amount and complexity of the services that we offer has grown, and we're (mostly) a volunteer team. This talk will be an interactive discussion of lessons learned, and how that can be applied to other puppet deployments.
Geppetto 3.0 is hot of the presses and in addition to supporting the new language features in Puppet 3.0, Geppetto 3.0 comes with a puppet code formatter, integrated documentation hovers, and more stylistic- and problematic-statements validation. The Geppetto 3.0 release is the biggest release in terms of functionality since the start of the project. Come see the new features in action, and learn how Geppetto can help you develop your puppet logic.
Wikimedia Foundation uses Puppet to automate its infrastructure and does so in a way that anyone in the world can participate. Through the combination of Puppet, Gerrit, and OpenStack, any one in the world can modify the Wikimedia Infrastructure or build new infrastructure from scratch. This presentation will show how Wikimedia is using Puppet to enable contributions, to manage its infrastructure, and how we're integrating Puppet with OpenStack in Wikimedia Labs.
I would like to present on Puppet External Data. This talk will cover Hiera and the External Node Classifier (ENC) interface. I also plan to cover the new data bindings in Telly and the working towards building an ENC on top of Hiera.
A case study of how Pinterest uses Puppet to manage its infrastructure. Pinterest has hundreds of Amazon EC2 virtual servers and uses Puppet Dashboard as the "source of truth" about its server inventory. Pinterest built a REST API for this database, which powers tools and automated scripts that integrate Puppet with internal systems and with Amazon Web Services.
Adam Kosmin will return to Puppetconf this year to speak about Reliant Security's evolution and shift to Hiera with Redis to manage some 10,000 physical and virtual nodes in a distributed puppet environment. Fueled by the never-ending need to scale along with a strong desire to empower customers, Reliant is constantly refining their approach to managing their Puppet source. Hiera has opened the door separating code from data and Redis has been identified as ""the perfect fit"" due for its support of important data types, intutive command set, and overall flexibility. Adam will cover our approach to building hiera-redis, along with using Redis at scale with puppet.
As a sysamin you need to be agile, you need to do something once and then automate it so you can keep pace. Apache CloudStack committer and recovering sysadmin will detail how to build an open source cloud computing platform using CloudStack. Additionally, he'll outline how to integrate CloudStack with tools to automate the management of a private cloud computing environment.
In the process of creating new software and releasing it to customers, both Development/QA, and Operations play critical roles. Historically the groups have worked in isolation to achieve the joint goal of delivering new software. When software couldn't be produced quickly enough to meet customer demand, Development largely turned to Agile development practices to speed their process. CI and other dev process automation tools have helped Dev to achieve their Agile goals. This increase in output removed Dev as the bottleneck and put pressure on Ops to deliver the required infrastructure at the rate Development was producing. With the help of tools like Puppet, Ops has been able to remove themselves as the bottleneck, providing Infrastructure as a Service to meet the increased development demands. But the two solutions alone are not enough to achieve true, fluid, end to end DevOps. The challenge (and hence new bottleneck) is consistently taking the output from development, combined with the necessary system and middleware configurations and processes, and repeatedly deploying these to the infrastructure made available by Ops. Come find out how Puppet can play a role in the Dev side of DevOps. Come hear about best practices in achieving frequent deployment of quality applications. Methods and benefits of implementing development process automation.
What's new in Puppet 3.0? Eric delivers a fast-paced technical rundown of new features and important changes in the upcoming release of Puppet 3.0.
More and more startups/companies are deploying their infrastructure directly and exclusively in EC2 or similar cloud provider. With that comes a whole new set of challenges and paradigms around scalability, reliability and availability. This talk will focus on how to leverage all the infrastructure parts of AWS, augment them with great (affordable) third party services and solid Open Source Software to create an operations environment that will scale with you, be as reliable as it can be, providing you and your peers with all the data you need to make good decisions to support (rapid) changes while letting you sleep through the night. And all that using a tiny operations team. It may make you coffee in the morning too.
This talk will skip the basics of Vagrant and dive right into the deep end by taking a look at some advanced use cases of Vagrant in a Puppet environment. We'll cover some of the basic use cases to get people warmed up, but quickly get into automated testing with Vagrant, developing Puppet modules that depend on a Puppet master, using Vagrant as a CI tool, and more.
The key pattern in continuous delivery is the deployment pipeline, which controls - and makes visible - the progress of all changes, whether code, configuration, or infrastructure - from check-in to realease. In this talk I'll describe the deployment pipeline pattern and discuss how continuous integration, configuration management, and automated testing fit in to it. I'll also give some examples of controlling infrastructure changes using Puppet in the context of a deployment pipeline.
Nokias' Location and Commerce division is heavily utilizing puppet to manage services in multiple datacenters around the world. Last year we set out to build a multi tenant puppet-infrastructure to provide deployment services to a multitude of R&D teams with a really diverse set of services that they develop. Uptake of the platform has been stunning ever-since. During the last year we had to overcome challenges around scalability due to growth in organizational and technical areas: - Amount of teams we support - Growth of the machine-base - Diversity of customers - Puppet-infrastructure - Test-infrastructure
Using out-of-the-box Puppet for non-sysadmin work - steps from going from no config management to managing 100 nodes and allowing non-sysadmin tasks to be performed.
RSpec-Puppet is taking off for module testing. This talk will show you how to extend that approach your entire Puppet configuration, and how to do so efficiently within the context of continuous delivery. We'll cover the basics of running RSpec-Puppet, and show you how to use facts for complete, end-to-end testing without actually having to deploy your configuration.
This talk will describe the evolution of how we've used Puppet at Demonware, a subsidiary of Activision Blizzard, to run the infrastructure of some of the world's biggest games, supporting millions of concurrent users for titles such as Call of Duty.
Want to learn how to use Puppet to manage your systems, but don't have a lot of systems to play with? Learn how to use the Eucalyptus Community Cloud to build up and tear down instances in a snap, all using cloud-init and Puppet.
This talk will review CERN's objectives and how the computing infrastructure is evolving to address the challenges at scale using community supported software such as Puppet and OpenStack.
SpaceX manages a large number of machines using puppet. A quick overview of where puppet fits and how we came to use it.
In this presentation, I'll be presenting on what's happened in the Puppet community for the last year. We'll discuss the victories, failures and lessons from the past, and ideas for the future. In addition, if we've hired a community manager by this time, I'd like to this presentation jointly. This will focus on state of the union, how to get started in the community, and how to contribute to docs and dev
This talk will cover the topic of managing packages across several environments and staging. We'll cover topics including the package resources included in Puppet core, how to hack on the Yum and Deb providers, and how to extend Puppet via custom resources and basic principals of idempotent operations. Some knowledge of the package management and Puppet resources is ideal for users attending the talk. We'll build on that knowledge and move into more advanced topics like custom providers and resources.
The OpenStack project is one of the fastest growing Open Source projects ever. In the first two years it has picket up hundreds of active developers. Managing a project like that takes a pretty large set of development resources. Additionally, since the project is a collaborative Open Source project, there is no reason that the development infrastructure can't be too. What started as one machine being manually managed by me has, over the last two years, turned in to many static machines and a giant elastic build farm - and it's all managed by puppet. We've learned a few things along the way, but two challenges are particularly interesting: - keeping as much of our puppet as possible in a completely public repo so that anyone in the project can theoretically help in the administration - directly collaborating with other project, like Wikipedia, who are doing similar things in a similar manner.
The CentOS.org infrastructure is one of the largest amongst any open source projects. The talk will be about some of the challenges we faced bringing puppet into the ecosystem, both on a technical and a people level. And highlight the major benefits we have seen over the years; finishing up with a state of affairs at the moment.
I will discuss the evolution of our organisation's automated host monitoring. Specifically, I will discuss the performance issues and custom solutions we implemented whilst progressing from a static and inconsistent monitoring solution to centralising our logic and requirements in Puppet through the use of exported resources. Unhappy with the performance of a more traditional solution using exported resources with Nagios across a large number of nodes, we opted for a home grown and somewhat crazy solution. We now rely on Puppet to automatically generate each host's customised monitoring configuration on the host itself; we do not use Puppet's stored config database as a transport mechanism. We implemented a hybrid solution that collects and validates pre-generated monitoring configuration out of band, whilst maintaining full process automation. Our solution compresses and transports configuration using NRPE, but the general idea could easily be applied to other monitoring solutions. The consequence is that we avoid the performance impact of exporting a large number of monitoring resources per host; rather we simply export one resource: is this host monitored? In a large, highly customised environment, this greatly improved our run times and scalability. Our solution wouldn't be appropriate for everyone, but is an interesting story for those using Puppet and trying to automate Nagios or other monitoring configuration for sanity. Our requirements are likely to align in many areas with the community at large; especially for other providers managing larger clusters of machines with Puppet.
There's a zillion of tools out there that can help developers and operations people work closer together. Technology moves fast, most people are so busy they don't have time to keep up with what's new, or sometimes don't really understand the need for these tools, until they take 5 minutes and listen to somebody using them. This talk will go over a bunch of unmissable open source system tools tools, some of them didn't even exist 2 years ago, We won't spill which ones we'll cover .. but rest assured .. you'll learn a few, And guess what ... they all either involve Puppet , or have been Puppetized
While there are a number of ways of building and deploying the OpenStack cloud computing stack, there are very few models that support day two management of those systems, from physical resource addition/removal, to the eventual need to upgrade and migrate pieces of the environment to newer versions. Since the OpenStack system isn't just a single point product, the use of puppet modules to manage the interaction of numerous pieces of an openstack environment will also be covered, including the addition of additional services, such as assurance integration and PaaS automation on top of the OpenStack environment.
Come learn how to use Razor to turn large sets of physical or virtual hardware into fully-built stacks managed by Puppet.
As interns, we feel we have a unique perspective about contributing to Puppet and Facter. We'll discuss the problems you may encounter as a non-developer, and the ways you can solve these problems. We'll cover the following items in our talk:
* The steps needed to get started as a contributor
* Good places to start contributing
* How to move forward when you inevitably run into problems
* Good habits, practices, and processes to follow as a contributor
* Common stumbling blocks
* What to expect from Puppet Labs once you submit a pull request
We will present some of our own personal stories from which we took away valuable knowledge, as well as what we have found rewarding from our work on Puppet and Facter. Even if the learning curve seems steep, we believe it is worth it to contribute to Puppet. As interns, we've had experience with how intimidating the submission process may seem from the outside. After just a few days, with no prior knowledge of Puppet at all, all three of us were able to begin contributing to the Puppet and Facter code bases. If we can do it, we think anyone can.
Attend this presentation to learn why technology professionals and entrepreneurs are relocating to Portland, Oregon. You will hear firsthand from Eric Sorenson, who recently left Apple in Cuppertino, California, to join Puppet Labs in Portland. You will also hear about economic trends, startup initiatives and unique industry resources that paint a compelling picture of Portland as a talent rich, fast-growing, affordable and competitive tech center. Boasting its own TV show, Portlandia, Portland is home to Pink Martini, The Dandy Warhols, The Decembrists, and The Shins, world class coffee houses, breweries, wineries, and distilleries, the center of the universe for the athletic and outdoor industry, innovative food carts, and James Beard Award-winning chefs. Undoubtedly, Portland has a lot going for it. What a lot of people are starting to find out is that Portland is also a serious tech center, with top tier creative talent and leading companies in mobile, open source, social, and enterprise software. For example, Portland is home to Intel's largest campus in the world with over 17,000 employees, as well as its software group, with over 3,500 employees. Portland is also home to eBay Mobile, IBM's open source division, Mozilla, Autodesk, Walmart Labs, Salesforce, McAfee, Symantec, Jive Software, Thetus Corporation, Caoxis/Viewpoint, Tripwire, Webtrends, WMWare, Puppet Labs, Urban Airship, Elemental Technologies, Janrain, ShopIgniter, Giftango, Simple, Cloudability, Meridian Apps, Cvent, Acquia, the Drupal Association and of course Puppet Labs. The list goes on beyond these great companies and includes the recently announced Salesforce expansion that will bring hundreds of jobs to the City. Portland is home to some of the most prestigious leaders in software and online development including Linus Torvalds, the creator of Linux, Ward Cunningham, the inventor of the wiki, and Russell Kirsch, the inventor of the world's first internally programmable computer.
Using Puppet's custom report processor capabilities, you can automatically keep all of your cloud images up to date and in sync.
Puppet Enterprise and Spiceworks for managing your heterogenous networks of Windows and Unix hosts. Puppet Enterprise as employed by Spiceworks for production and QA use. Spiceworks is a free Windows-based network inventory and management application.
People are deploying servers in cloud environments faster than ever before but most are still not doing so in a safe and secure manner. Too few server instances are hardened as a part of the provisioning process; often leaving the technological doors wide open for potential service disruption by malicious threat agents -- such as malware, automated attack tools and human attackers. This talk will explain how Puppet can be used to automate the creation and maintenance of secure server baselines as a foundation for securely operating in cloud environments.
At GitHub, we've got an army of developers, designers, and just plain amazing people. They've all got laptops, of course, and for a long time they were all managed painstakingly by hand about a billion different ways. Hold on a second! I thought we were developers. We should know better! That's why John Barnette and I wrote "The Setup". The Setup is our infrastructure for managing every machine GitHubbers use. The purpose of this talk is to talk about how The Setup came to be, some of the choices we've made, and some of the amazing things yet to come with this project — including lots of sweet, sweet open source love.
Patterns and Antipatterns to create Puppet Modules that can be used, reused and abused. Points of Views about a Holistic approach to modules design for an integrated infrastructure development.
At Mozilla we manage thousands of nodes with puppet with different environments, modules, SLAs and teams. Keeping on top of it all is a difficult task for a small team of sysadmins. We'd like to share and talk about how we ma
This presentation will cover topics of when software/configurations should be delivered as native packages (rpm, deb, etc) vs just placing configurations and files into Puppet Manifests. This will include distribution of web content, jar files, stuff on the JVM, ruby applications, etc.
In 2003 CERN helped write a config management system, called Quattor, in order to manage the compute grid. At the time, our scale made our problem unique, and that there wasn't any configuration management solutions available that suited us. In 2011 we looked again, and concluded that if our use case really had been all that special it wasn't anymore, and there were other widely used mature tools available, such as puppet. We have started an ""agile infrastructure"" project, which aims to use industry standard tools, including puppet, to solve what should be common problems. This talk will cover how we have traditionally done configuration management, the experiences of the pilot of our new infrastructure, and looking forward to the challenge of scaling this out to the thousands of physical machines in our computer centre and the VMs that they host.
Puppet Labs recently launched its new IT automation industry certification program and is introducing two new certifications, the Puppet Professional certification for System Admins, and the Puppet Developer certification for Ruby Developers at this conference. Find out more about the methodology and the business benefits behind this benchmark standard for the rapidly growing Puppet user base, and the technology objectives that will allow qualified candidates to be successful in Puppet's two new high-stakes certification exams offered at PearsonVUE certification testing centers world-wide starting the 1st of October, 2012.
The developers you work with have a new service they're building, and it is your job to add the appropriate Puppet modules to provision it. Where do you start? A base image, Vagrant and some tinkering? Editing files in ""/etc/puppet"" on a staging Puppet host? What is the most maintainable work-flow which won't saddle you with technical debt in the future? In this talk, I will take you from the ""outside"" inward, writing tests along the way with Cucumber and RSpec. This approach encourages you to think more about where the lines are drawn between the various services and resources you're configuring. While the work-flow is a departure from the ""traditional sysadmin"" role, the resulting modules can be more easily re-used and re-factored.
PayPal is a relatively recent newcomer to the Puppet world, but is enthusiastic about its possibilities. In this presentation, we will show how Puppet Enterprise is being used to manage application and system configuration at PayPal. We will also outline how we use MongoDB and Hiera to specify configuration, and our approach to application deployment.