Organization leaders expect technology features and updates delivered to clients and customers faster than ever before. For technology teams to have that ability, deployments to the production environment should be a smooth and pain-free process. It should be as trivial as pushing those features to a staging or test environment. A production deployment taking hours is taxing on engineers, IT staff, QA folks and everyone else involved.
So, how does an organization get from "once-a-quarter" release to several deployments a week or even several a day?
Here’s a look at the deployment processes for two different teams:
These cases may seem "too manual" or "too automated" but they are definitely realistic. Your particular team or organization could be anywhere in between. For example, you could add QA testing to any phase in Team B if you don’t want to fully rely on automated testing.
Getting from Team A’s process to Team B’s might not be the top priority for your team since it requires significant time, effort and changes in infrastructure. However, it really is in the best interest of everyone involved:
Project managers can spend time on upcoming work and quarters instead of planning for production deployments.
With small increments, product managers can be more aggressive with new features and have extended versions of A/B testing to see what changes users respond to best.
Engineers don’t have to spend hours planning and prepping for a production deployment.
Changes can be delivered in smaller increments so rollbacks and troubleshooting are easier.
Delivery vs. Deployment
Depending on need, a team can choose between Continuous Deployment and Continuous Delivery.
Continuous Delivery means every update that goes through the automated process is in a production-ready state and can be deployed anytime. The team decides if it should be deployed to production.
Continuous Deployment (followed by Team B above) is an extension of Continuous Delivery. In this process, every update is deployed to production immediately.
Achieving Continuous Delivery
Continuous Delivery is a result of a fully functional DevOps practice within an organization. DevOps requires tight integration between engineers, QA and IT personnel as well as constant communication and understanding of deployment processes.
The road to Continuous Delivery starts with:
Continuous Integration (CI): Tools, like Git, SVN and Perforce, allow developers to all work in perfect harmony, most of the time. In the CI model, several development and test engineering resources can work on their individual features in isolation and continuously sync it with the mainline code base. As part of CI, a release-ready build is created during every commit or check-in. The main line is always deployable to production at any time. Developers can work off a branch and integrate to the main line when ready.
Unit and integration tests: CI relies on a highly-detailed and extensive test suite. Without a QA team being involved at every stage, these automated tests are essential in confirming that these builds are ready to be deployed to production. Having a minimum code coverage requirement ensures that no new functionality makes it into your main code line without having the adequate test cases accompanying it.
Configuration management (CM): Being able to reliably move your functionality, tools and settings as well as being able to track changes throughout a product lifecycle is key to achieving a Continuous Delivery model. There are several tools like Chef, Puppet and traditional scripting that can assist with managing and tracking all changes and defects. Open source tools like Docker are changing the traditional CM methods as well.
Performance metrics: Using tools, like Splunk, Sumo-logic and Graphite, can help confirm that the latest deployment did not disrupt the existing ecosystem. Splunk alerting and pager duty can also let you know immediately if your new deployment is causing any issues such as increased error rate, longer response times etc. Live metrics and alerting provide insight into whether:
The newly deployed services are within the acceptable SLAs
The error rate is not higher than usual
The CPU usage, load average, DB usage etc. are normal
IT support: The core purpose of DevOps is to break down the walls between the development and IT teams to better understand existing methods and create new standards. Working in silos is a practice of the past and doesn’t fit in with the needs of today’s fast-moving business organizations. IT teams should be involved at every stage since they will be able to provide information on existing infrastructure and help build out the new one.
Culture: Just like DevOps, Continuous Delivery will always evolve based on the needs of the organization, complexity of services offered and new technologies. It is a culture change as much as it is a process change. Team members have to "buy in" to the concept of automating certain tasks that are otherwise second nature to them. Automation doesn’t make someone less valuable to their team; it enhances their value as a team member since they can now make even greater contributions by innovating and executing on new ideas. It’s not one specific goal but a constantly improving model.