Modular Architecture

Modularity plays an important role in software architecture. It fills a gap that has existed since we began develop enterprise software systems in Java. Here, we’ll discuss that gap and explore how modularity is an important intermediary technology that fills the gap.

4.1 – Defining Architecture

There are numerous definitions of architecture. But within each lies a common theme, and some key phrases. Here are a few of the definitions. From Booch, Rumbaugh, and Jacobson in UML User Guide (Addison-Wesley, 1999):

An architecture is the set of significant decisions about the organization of a software system, the selection of the structural elements and their interfaces by which the system is composed, together with their behavior as specified in the collaborations among those elements, the composition of these structural elements and behavioral elements into progressively larger subsystems, and the architecture style that guides this organization — these elements and their interfaces, their collaborations, and their composition.

Now, the ANSI/IEEE Std 1471-2000:

The fundamental organization of a system, embodied in its components, their relationships to each other and the environment, and the principles governing its design and evolution.

In TOGAF, architecture has two meanings depending on context:

1.)   A formal description of a system, or a detailed plan of the system at component level to guide its implementation

2.)   The structure of components, their inter-relationships, and the principles and guidelines governing their design and evolution over time.

Examining these definitions reveals many common keywords that span each, which I’ve highlighted in bold in the various definitions. There exist important underlying currents embodied by these keywords. But these keywords also lead to some important questions that must be answered to more fully understand architecture. What makes a decision architecturally significant? What are the elements of composition? How do we accommodate evolution of architecture? And what does this have to do with modularity? As we begin to delve into answering these questions, I’d like to start with a story on software architecture.

4.2 – A Software Architecture Story

The story of software architecture reminds me of the following story.

“A well-known scientist (some say it was Bertrand Russell) once gave a public lecture on astronomy. He described how the earth orbits around the sun and how the sun, in turn, orbits around the center of a vast collection of stars called our galaxy. At the end of the lecture, a little old lady at the back of the room got up and said: “What you have told us is rubbish. The world is really a flat plate supported on the back of a giant tortoise.” The scientist gave a superior smile before replying, “What is the tortoise standing on?” “You’re very clever, young man, very clever”, said the old lady. “But it’s turtles all the way down!”

- A Brief History of Time by Stephen Hawking

Software architecture is “‘turtles all the way down.”

4.2.1 – The Ivory Tower

Many of us can relate. In dysfunctional organizations, architects and developers fail to communicate effectively. The result is a lack of transparency and a lack of understanding by both sides. As shown in Figure 1, architects tend to bestow their wisdom upon developers who fail to grasp the high level concepts in terms that make sense to them. The failure often occurs (though I recognize there are other causes) because architecture is about breadth and development is about depth.  Each group has disparate views of software architecture, and while both are warranted, a gap between these views exists. The architect might focus on applications and services while the developer focuses on the code. Sadly, there is a lot in between that nobody is focused on. It is this gap between breadth and depth that contributes to ivory tower architecture.

Figure 1: The Ivory Tower

4.2.2 – Turtles and the Tower

Without question, the ivory tower is dysfunctional, and systems lacking architectural integrity are a symptom of ivory tower architecture. So assuming good intent on the part of the architect and the developer, how can we bridge the gap between breadth and depth? How can we communicate more effectively? How do we increase understanding and transparency?

Let’s revisit the definition of software architecture by exploring another. My favorite definition of software architecture was offered by Ralph Johnson in an article by Martin Fowler (Who Needs an Architect? citation needed). He states:

In most successful software projects, the expert developers working on that project have a shared understanding of the system design. This shared understanding is called ‘architecture.’ This understanding includes how the system is divided into components and how the components interact through interfaces. These components are usually composed of smaller components, but the architecture only includes the components and interfaces that are understood by all the developers…Architecture is about the important stuff. Whatever that is.

The key aspect of this definition that differentiates it from the earlier definitions in this chapter is that of “shared understanding”, which implies that there is a social aspect to software architecture. We must have a shared understanding of how the system is divided into components, and how they interact. Architecture isn’t just some technical concept, but also a social construct. And it is through this social aspect of architecture that we can break down the divide between architects and developers.

To ensure shared understanding, we have to architect “all the way down.” Architects cannot worry only about services and developers cannot worry only about code. There is a huge middle ground that each must also focus on, as illustrated by the diagram in Figure 2.

Figure 2: Architecture all the way Down

Focusing exclusively on top level abstractions is not enough. Emphasizing only code quality is not enough either. We must bridge the gap through other means, including module and package design. Often times, when I speak, I ask the audience to raise their hands if they spend time on service design. Most do. I also ask them to raise their hand if they spend time on class design and code quality. Again, most do. But then I ask if they also spend time on package and module design. Usually, only a small percentage leave their hands raised.

This is unfortunate, because module and package design are equally as important as service and class design. But somewhere along the way, with our emphasis on services and code quality, we’ve lost sight of what lies in between. Within each application or service awaits a rotting design, and atop even the most flexible code sits a suite of applications or services riddled with duplication and lack of understanding. A resilient package structure and corresponding software modules help bridge the divide between services and code. Modularity is an important intermediate technology that helps us architect all the way down, and is the conduit that fills the gap between breadth and depth.

We need to start focusing on modularity to ensure a consistent architecture story is told. It is the glue that binds. It’s the piece that helps bridge low level class design with higher level service design. It’s the piece that helps bring down the ivory tower, enhance communication, increase transparency, ensure understanding, and verify consistency at multiple levels. It is the piece that allows us to “architect all the way down”, and allows to realize the goal of architecture.

4.3 – The Goal of Architecture

Modularity helps address the social aspect of software architecture, but it also helps us design more flexible software systems. That is, a system with a resilient, adaptable, and maintainable architecture. Examining the earlier definitions of architecture leads us to the goal of architecture. The Johnson definition of architecture as quoted by Fowler makes it’s apparent that architecture is about the important stuff. And in the following statement, Booch makes (citation needed) it clear that something is architecturally significant if it’s difficult to change.

All architecture is design but not all design is architecture. Architecture represents the significant design decisions that shape a system, where significant is measured by cost of change.

Based on these statements, it’s fair to conclude that the goal of software architecture must be to eliminate the impact and cost of change, thereby eliminating architectural significance. We attempt to make something architecturally insignificant by creating flexible solutions that can be changed easily, as illustrated in Figure 3. But herein lies a paradox.

Figure 3 – The Goal of Architecture

4.3.1 – The Paradox

The idea behind eliminating architecture isn’t new. In fact, Fowler mentions “getting rid of software architecture” in his article, Who Needs an Architect? (citation needed). And the way to eliminate architecture by minimizing the impact of cost and change is through flexibility. The more flexible the system, the more likely that the system can adapt and evolve as necessary. But herein lies the paradox, and I’ll use a statement by Ralph Johnson to present and support the idea.

…making everything easy to change makes the entire system very complex…

So as flexibility increases, so too does the complexity. And complexity is the beast we are trying to tame because complex things are more difficult to deal with than simple things. It’s a battle for which there is no clear path to victory, for sure. But what if we were able to tame complexity while increasing flexibility, as illustrated in Figure 4? Taming the best we call complexity is the emphasis of Chapter 5. But first, let’s explore the possibility of designing flexible software without increasing complexity. Is it even possible? In other words, how do we eliminate architecture?

Figure 4 – Maximize Flexibility, Manage Complexity

4.3.2 – Eliminating Architecture

As the Johnson quote clearly points out, it’s not feasible to design an infinitely flexible system. Therefore, it’s imperative that we recognize where flexibility is necessary to reduce the impact and cost of change. The challenge is that we don’t always know early in the project what might eventually change, so it’s impossible to create a flexible solution to something we can’t know about. This is the problem with Big Architecture Up Front (BAUF), and it’s why we must make architectural decisions temporally. In other words, we should try to defer commitment to specific architectural decisions that would lock us to a specific solution until we have the requisite knowledge that will allow us to make the most informed decision.

It’s also why we must take great care in insulating and isolating decisions we’re unsure of, and ensuring that these initial decisions are easy to change as answers to the unknown emerge. For this, modularity is a missing ingredient that helps minimize the impact and cost of change, and it’s a motivating force behind why we should design software systems with a modular architecture. In the UML User Guide (P. 163), Booch talks about “modeling the seams in a system.” He states:

Identifying the seams in a system involves identifying clear lines of demarcation in your architecture. On either side of those lines, you’ll find components that may change independently, without affecting the components on the other side, as long as the components on both sides conform to the contract specified by that interface.

Where Booch talks about components, today we talk about modules. Where Booch talks about seams, we’ll talk about joints. Modularity combined with design patterns and SOLID principles, represent our best hope to minimize the impact and cost of change, thereby eliminating the architectural significance of change. The joints of a system are discussed in Chapter 5, Section 5.6. The SOLID principles are discussed in Appendix A.

4.4 – Modularity – The Missing Ingredient

Two of the key elements of the architectural definitions are component and composition. Yet there is no standard and agreed upon definition of component (reminding me of architecture, actually), and most use the term pretty loosely to mean just “a chunk of code”. But that doesn’t work, and in the context of OSGi, it’s clear that a module is a software component. We devoted Chapter 2 to defining module. Developing a system with an adaptive, flexible, and maintainable architecture requires modularity because we must be able to design a flexible system that allows us to make temporal decisions based on shifts that occur throughout development. Modularity has been a missing piece that allows us to more easily accommodate these shifts, as well as focus on specific areas of the system that demand the most flexibility, as illustrated in Figure 5.

Figure 5 – Encapsulating Design

Modularity, in conjunction with design patterns and SOLID principles, represent our best hope to minimize the impact and cost of change. It’s easier to change a design embedded within a module than it is a design that spans modules.

4.5 – Answering Our Questions

At the beginning of this chapter, we posed the following questions after introducing the three definitions of software architecture. Through explanation, we answered each of these questions. But to be clear, let’s offer concise answers to each of these questions.

• What makes a decision architecturally significant? A decision is architecturally significant if the impact and cost of change is significant.
• What are the elements of composition? The elements of composition include, classes, modules, and services.
• How do we accommodate evolution of architecture? Evolution is realized by designing flexible solutions that can adapt to change. But flexibility breeds complexity, and we must be careful to build flexibility in the right areas of the system.

And what does this have to do with modularity? Modularity is an important intermediate component that helps increase architectural agility. It fills a gap that exists between architects and developers. It allows us to create a software architecture that can accommodate shifts. Modularity helps us architect all the way down.