Video recording and production done by DockerCon.
Since July 2014 Shopify's been serving thousands of requests per second of production web traffic from Docker containers. This was an 8 month effort, with multiple pivots of direction from the team—and we're only getting started. This talk covers the lessons learned through the trial and error of an in-flight architecture redesign, spanning hundreds of hosts, as well as the technical vision of the future of our platform.
Combining my passions for automation and games, I will discuss the opportunity and challenge for automating and containerizing game servers. The necessity to prioritize scale and performance makes game servers a perfect candidate for the container revolution. However many aspects of game servers and apps make this pretty challenging. Starting from the perspective of a typical transition to in-house Docker based micro services at Shopkeep, I’ll take a deep dive into the problems with, benefits of and approaches to containerizing these “opinionated” applications in a wider setting.
We set out to solve the problems of quickly building high quality games for a fragmented mobile market. Taking advantage of HTML5 allowed a fast, familiar and highly iterative local development process, and a hybrid build process for native apps meant high performance games on mobile. Our product is designed to comprehensively handle complex UI flows, related server tasks as well as deep integrations with any social platform. This is necessarily complex piece of engineering, with dozens of large dependencies, and 5 local web servers powering a single user’s experience. When we set out to make this easily available to 3rd parties, we used Docker to solve to major challenges: 1) Fitting many users, each with a unique development environment, on to one machine; 2) Managing all of these development environments in a scalable way.
Docker Compose is the last piece of the orchestration puzzle. After provisioning Docker daemons on any host in any location with Docker Machine and clustering them with Docker Swarm, users can employ Docker Compose to assemble multi-container distributed apps that run on top of these clusters.
Docker Compose is still a work-in-progress and we want your help to design it. In particular, we want to know whether or not you think this should be a part of the Docker binary or a separate tool. Head over to the proposal on GitHub to try out an alpha build and have your say.
Docker Hub Enterprise is a Powerful turn-key solution for creating multi-container distributed applications behind the firewall. “Docker Hub Enterprise is Docker’s foundation for establishing relationships with our rapidly expanding enterprise customer base, who view the Docker open platform as the cornerstone of their distributed application strategy. These organizations want a behind-the-firewall solution that enables them to leverage both the broader ecosystem and the more dynamic development environment that Dockerization has enabled,” said Ben Golub, CEO of Docker.
In this video, Ken Cochrance talks about Docker Hub and the new features announced during DockerCon Europe including Webhooks 2.0 and
Terraform is a tool for building and safely iterating on infrastructure, while Consul provides service discovery, monitoring and orchestration. In this talk we discuss using Terraform and Consul together to build a Docker-based Service Oriented Architecture at scale. We use Consul to provide the runtime control plane for the datacenter, and Terraform is used to modify the underlying infrastructure to allow for elastic scalability.
The ModCloth Platform team has been building a Docker-based continuous delivery pipeline. This presentation discusses that project and how we build containers at ModCloth. The topics include what goes into our containers; how to optimize builds to use the Docker build cache effectively; useful development workflows (including using fig); and the key decision to treat containers as processes instead of mini-vms. This presentation will also discuss (and demo!) the workflow we’ve adopted for building containers and how we’ve integrated container builds with our CI.
There are several large scale deployments of microservices in production. This talk will summarize the differences and commonalities across these architectures and show how they are evolving.
Opening words by Ben Golub (CEO of Docker) and Solomon Hykes (Founder and CTO). Welcoming the community to the European edition of DockerCon.
The real value of Docker is to get people to agree on something. In that spirit, Solomon talks about the open source project, the people building it and review the latest numbers highlight how active the Docker project is.
Docker Swarm is native clustering for Dockerized distributed apps. It picks-up where Docker Machines leaves off by optimizing host resource utilization and providing failover services. Specifically, Docker Swarm allows users to create resource pools of hosts running Docker daemons and then schedule Docker containers to run on top, automatically managing workload placement and maintaining cluster state.
In this video from DockerCon Europe 2014, Derek McGowan explains the work that is currently being done around trust and Docker Image provenance.
Partners panel at DockerCon EU:
Ben Golub - CEO, Docker, Inc
Kit Colbert - VP & CTO, Cloud-Native Apps, VMware
John Gossman - Architect, Microsoft
Jason McGee - CTO Cloud Services, IBM
The future of Micro services panel at DockerCon Europe 2014 with Ben Golub (CEO Docker), Scott Johnston (SVP Docker), Michael Coté (Analyst 451 Research and Adrian Cockcroft (Fellow at Battery Ventures)
Operating apps at web scale has become the new normal, but has been out of reach for most companies. Join us as we show you how to deploy and manage your Docker containers at scale. See how easy it is to build highly-available, fault-tolerant web scale apps using Docker with the Mesos cluster scheduler. Docker plus Mesos is a new way to scale applications. Together they give you capabilities similar to Google’s Borg, the Googleplex’s secret weapon of scalability and fault tolerance.
At BBC News, we've had a lot off issues with using CI environments for building and testing. Recently, we've taken a lot of time in order to solve these issues. We've created a new Jenkins setup to simplify our environment and in build & test our software inside on containers, even if ultimately we're not deploying in a container. This workflow has saved us potentially days of lost developer time.
Scientific results are shared as manuscripts which researchers read and interpret in their own work. Based in the field of genomics, this talk will show how genome assembly software can be shared with Docker containers using a common API. Containerising scientific software leads to fungible scientific methods which can be objectively compared and easily interchanged. This leads to "standardised parts" for bioinformatics software and allows scientists to focus more on their research.
Just like the hosted Docker Hub you know and love, Docker Hub Enterprise integrates into your build/test/deploy pipeline and makes development a joy. And just like Docker Hub, when you base your application strategy on the Docker ecosystem and Docker Hub Enterprise, you have the confidence that your container based workflows will remain solid, even with a rapidly changing Docker ecosystem. Why’s this the case? Because with your help, Docker Hub Enterprise will be the product defining the future of enterprise distributed applications.
Fig (http://www.fig.sh/) is an Docker-based development environment tool which is owned by Docker. Originally, we can only deploy to one host at one time. My hack in Docker Global Hack Day #2 is to enable Fig to deploy multiple hosts at one time. In this talk, I'll give a brief introduction to Fig first. Then describe my hack in the hack day. Finally I'll give a short demo about deploying apps to multi hosts at one time.
At Societe Generale GBIS, time to market & quality matters; hence we do love continuous delivery. In this context, we’re considering the Container as a Service pattern: artifacts produced by the continuous integration chain would become self-sufficient “dockerized” application modules, onboarding both code and subsequent system requirements; then, a CaaS cloud would enable to host these containers. In this talk, I’ll present our usecase and current findings, considering both technical & operational aspects. We’ll talk about software factories, immutable IT, registries, containers configuration, API-driven infrastructure, DevOps roles shifts. Finally, we’ll discuss pros/cons of this solution toward regular IaaS and PaaS.
Docker doesn't only revolutionize your application hosting, it also revolutionizes your development pipeline. ING has over 250DevOps teams, thousands of applications and a complex application landscape. ING is simplifying its application landscape in record time while introducing a webscale architecture based on anti-fragility patterns. Speed is vital in this transformation and one of ING’s key assets is it Continuous Delivery Pipeline. In this talk, we will show how ING uses test containers for confidence checks, integration testing on up and downstream services, creating dev/test environments for every feature branch, reverse proxy and CI servers. As a result, we are able to automate test processes and reduce our integration testing costs.
Hey curious friend, let’s play a game. How can we bring together two different companies, an established enterprise with traditional dev and ops having cultural differences when working together with a DevOps champion startup. In the middle exists a number of real use cases on how we are bringing DevOps culture with Docker to Atos Worldline. In my talk I will discuss the first use cases for Docker at Atos Wordline, where we are today, learnings and benefits until now, our future technology stack and how Docker is changing our human stack a.k.a. how we communicate and work together.
As a scientific software engineer in the field of radio astronomy, I'm involved in creating, improving and maintaining a broad range of tools used by scientists. These tools are used to process and analyse data coming from various radio telescopes, particularly LOFAR (LOw Frequency ARray) and SKA (Square Kilometre Array). Radio astronomy has a long and rich computing related history, and with that comes a big pile of poorly written, fragile, and badly maintained legacy code. This problem is not unique to radio astronomy but is a common in various scientific fields. Although no quick solution exists when it comes to re-writing lots of legacy code, Docker helps to contain this fragile software, simplifies the installation and ensures that the software works. In this talk I will discuss how Papino, our umbrella project for containing various radio astronomy libraries and RODRIGUES, an online radio telescope simulator leverage Docker.
Docker Machine takes you from zero-to-Docker in seconds with a single command.
Before Docker Machine, a developer would need to log in to the host and follow installation and configuration instructions specifically for that host and its OS. With Docker Machine, whether provisioning the Docker daemon on a new laptop, on virtual machines in the data center, or on a public cloud instance, the same, single command …
% machine create -d [infrastructure provider] [provider options] [machine name]
… gets the target host ready to run Docker containers. Then, from the same interface, you can manage multiple Docker hosts regardless of their location and run any Docker command on them.
Furthermore, the pluggable backend of Docker Machine allows users to take full advantage of ecosystem partners providing Docker-ready infrastructure, while still accessing everything through the same interface. This driver API works for provisioning Docker on a local machine, on a virtual machine in the data center, or on a public cloud instance. In this Alpha release, Docker Machine ships with drivers for provisioning Docker locally with Virtualbox as well as remotely on Digital Ocean instances; more drivers are in the works for AWS, Azure, VMware, and other infrastructure.
Note that Docker Machine is a separate project from the Docker Engine. To try out the Alpha build and contribute to Docker Machine and its drivers, go to its repository.
For your convenience, we’ve divided the keynote into four discrete sections:
Introduction: The future of the Docker project by Solomon Hykes
Docker Machine by Ben Firshman
Docker Swarm by Victor Vieux and Andrea Luzzardi
Docker Compose by Aanand Prasad
his talk introduces Clocker and shows how to bootstrap a Docker Cloud that is responsive and scalable, across a dynamic cluster of hosts and cloud providers. Clocker is an Apache licensed open source project that demonstrates intelligent placement, on-demand provisioning and autonomic management of containers using Apache Brooklyn as the central nervous system. The Clocker stack enhances the standard Docker installation using best practices for configuration and integrates Weave networking capabilities plus Apache jclouds for provisioning on any infrastructure. We will show how to use Clocker to deploy, monitor and scale complex applications defined using Brooklyn blueprints across a network of Docker containers in the cloud.
Green field projects might be able to take advantage of containers from the start, but how can we take a monolithic existing code-base and make the move to Docker? We want to run our code as a collection of small collaborating containers, but we have a large existing code-base, and don’t want massive disruption to product releases. We’ll take a walk through some of the challenges we’ve faced, and techniques used to solve taking a set of large collaborating Rails applications into containers. In this process we’ve aimed to progressively move towards our ideal.