In the 1980’s, German and American auto manufacturers were shocked by the sudden popularity of reliable, inexpensive imports from Japan. A now-famous article by John F. Krafcik entitled “Triumph of the Lean Production System” quantified the problem. The Japanese companies, led by Toyota, had squeezed out every last little bit of waste from the process, had automated as much as they could and had changed their management styles to include, among other things, feedback from their workers. The Americans and Germans quickly adopted these new methodologies, called “Lean Manufacturing”. They then rapidly reached, and in some cases exceeded, the same levels of quality.
So, why am I talking about making cars in a software engineering blog?
Because some of the ideas developed for Lean Manufacturing continue to be applied to how we develop and deploy software.
It wasn’t that long ago that product release cycles were measured in months, if not years. What came after Windows 95? Windows 98! Three years later! And it wasn’t just software products. Generating financial reports could take weeks as data was collected. Provisioning a new system in the data center required procurement, approvals, finding data center space, electrical and more.
Slow, inefficient processes led to slow and inefficient businesses—and it all had to change.
<<< Start >>>
<<< End >>>
DevOps, a term which seems to have been coined in 2009, was the first in a series of these revolutions. In the past, developers would create an application, QA would test it, and, if stable enough, would turn it over to an Operations team to deploy and run. In a DevOps environment, of course, every check-in kicks off a set of automated tests and, perhaps, an automated deployment to the cloud.
We all now recognize this as a Continuous Integration/Continuous Deployment pipeline. And a whole ecosystem of tools—Jenkins, Ansible, Gitlab CI, Travis CI, and numerous others—has sprung up to help organizations create their own pipelines.
To work in such a fashion means that organizational barriers come down. The application team in this case consists of development, QA and operations all working together: hence DevOps.
It didn’t take long. Soon, teams recognized that security professionals and their processes could and should be incorporated into the delivery pipeline. Vulnerability scanning, automated testing of authentication and authorization, patching and other security practices would evolve DevOps into DevSecOps.
As organizations became mature in their use of the cloud, and careful optimizations of cloud resources began to yield significant cost savings, a new discipline—FinOps—was born. In the same way that DevOps broke down organizational barriers, so too did FinOps. To quote the FinOps Foundation, “FinOps increases the business value of cloud by bringing together technology, business and finance professionals with a new set of processes.” To attain maximum financial value from the cloud, technology professionals must collaborate with business and financial teams to, for example, ensure applications are “rightsized,” that discounts such as Reserved Instances are taken advantage of when appropriate.
And data! Every organization is faced with an explosion in the amount of data they maintain. There has been an increase in demand for insights from analytics and machine learning, real-time dashboards, predictive alerts and more. These imply that the underlying data must be current, cleansed and available to be used for experimentation by data scientists.
DataOps describes a set of processes, many automated, for turning data into information.
There are more: InfraOps is about optimizing cloud infrastructure to the furthest extent possible. This could include using hyperconverged hardware or software containers. ArchOps aims to streamline the IT architecture process by connecting people and tools in new ways. For example, changes in an enterprise architecture diagram could trigger a change in a configuration management database. AIOps uses artificial intelligence to improve IT operations. It could correlate alerts to determine a root cause, for instance. BizOps is based on the idea of cross-functional teams that set strategy and improve company performance based on data and analytics.
Are these “disciplines” well defined? Do they have clear boundaries between them? Certainly not!
But what’s important to recognize is the original concept of Lean Manufacturing as it’s now applied. Automating wherever possible, driving collaboration, rapid feedback and turnaround: which are applicable to all aspects of cloud computing.
How are you using ***Ops?