I overheard a comment from an energetic agile team: “Architecture, the big design upfront, thank goodness that’s in the past.” I stopped and pondered, is it, really? Then I reminded myself that although refactoring has emerged as an important software engineering technique, it is not a replacement for sound architectural design; if an architecture is decent you can improve it, but refactored junk is still junk. But what exactly were they trying to say? In this blog I will discuss the early and continuous focus on the architecture of the system, to enable scaling up Agile development and minimizing unanticipated roadblocks.
Let’s quickly review the origin of Architecture. Architecture (Latin architectura, from the Greek ἀρχιτέκτων arkhitekton "architect", from ἀρχι- "chief" and τέκτων "builder") is both the process and the product of planning, designing and constructing of buildings and other physical structures. What exactly is the relationship with software development, you may ask?
Following the definition, we can define software architecture as the high-level structure of a software system, the discipline of creating such a high-level structure and the documentation of the structure.
True to the Agile Manifesto, “individuals and interactions,” let’s explore the role of the “chief builder or architect” as a counterpart to the “product owner.” In the practice of Agile development, architects work on three distinct but interdependent structures,
The Architecture (A) of the system under design, development or refinement, which we have called the traditional system or software architecture.
The Structure (S) of the organization, which are the teams, partners, subcontractors and others.
The Production Infrastructure (P) used to develop and deploy the system. This last activity is especially important in contexts where Development and Operations are combined and the system is deployed more or less continuously.
I will argue that these structures must be kept aligned to support agility and avoid common aches and pains of Agile development. Turning our attention back to the chief builder as a role in Agile software development and the architecture of the system (A), let’s explore the alignment of the two remaining structures. The relationship of A to S is known as socio-technical congruence1. When A is lagging, we face a situation of technical debt; when S is lagging, we have a phenomenon called “social debt,” akin to technical debt, which slows down development. The alignment of A with P is seeing renewed interest with increased focus on Continuous Integration and Deployment and the concept of “DevOps” 2. When P is lagging, we witness a case of “infrastructure debt,” which is another source of friction in software development. A, S and P must be “refactored” regularly to be kept in sync so that they can keep supporting each other. On the other hand, too much emphasis on any of the three structures upfront will potentially result in excessive delays.
Circling back to the introduction, the team was correct, the "big design upfront" is something of the past, however the need for a well-balanced A, S and P relationship is not.
At scale development needs a healthy, proactive and early focus on both system architecture and software architecture. Agile approaches work well for projects in a “sweet spot” with certain enabling characteristics: small teams of 5-12 people, preferably co-located; a stable underlying architecture; frequent deliveries; and low to medium criticality of the system.
But how do we scale Agile development for larger, bolder software development projects?
There are many possible answers to the question “how do we scale Agile up” and “how do we use it outside of its sweet spot”?3 They often take the form of modifying an Agile development practice to make it work “at scale," e.g., Scrum scaled up to Scrum of Scrums (SAFe4).
The role of chief builder and architecture introduces into an Agile development team a common vocabulary and culture around the "system" and how it "functions"; a systematic approach to controlling dependencies; ability to identifying and gradually reducing technical debt; and lastly, guiding release planning and configuration management.
This is where I will argue that an early and iterative focus, led by an architect, on architecture of the system, is required. It allows for scaling up Agile development and removes obstacles along the way. Having an interwoven A, S and P congruence solves four aches and pains in scaled Agile development endeavors, namely:
An architecture of the product supports multiple waves of enhancement to accommodate a constant flow of new needs.
An evolving architecture supports product enhancements.
An architecture, supportive of Agile team dynamics, allows for the team to operate in the sweet spot.
Synergy between Development and Operation groups allows for seamless handover.
I will conclude by saying that the "Big Upfront Design" is something of the past.
However, projects that need the ability to scale on all three dimensions (scope, team size, and duration) will require tools to organize the work, make the right decisions, communicate these decisions, implement and validate them, and define guidelines and processes applicable across the project. I believe that these tools do not exist (maybe partially) in the traditional toolkit of Scrum, XP and Lean, but they can be found in the architects’ toolkit. The key to scale is embedded in the need to acknowledge a “product owner” counterpart role, e.g. Architecture Owner (or Chief Builder), that will assist with the keeping the A-S-P congruency by following an Agile development approach. This is where the architecture helps to alleviate some of the aches and pains of Agile today.
Please subscribe for more posts.
Footnotes:1 A metric used to investigate how good the alignment or fit is between the organizational structure and the software architectures