Modular Architecture

Reuse is software development’s unattainable panacea. The ability to compose systems from reusable elements has long been our achille’s heel.  We want reuse so badly, yet our failures are spectacular. Almost all major technology trends of the past 20 years touts reuse as the saving grace. Vendors have sold billions of dollars in software through the broken promise of increased reusability.

What happened? Reuse was supposed to save software development. In the early 90’s, object-orientation promised to save us. It hasn’t. In the late 90’s, component-based development promised to save us. It didn’t either, and the movement died. Shortly after the turn of the millenium, SOA promised to save us. It didn’t, though we’re still trying. Why is reuse so difficult?

Foremost, reuse is achieved by designing flexible software entities that can be tailored based on need. In other words, we reuse an entity by configuring it to a specific context. But as we’ve seen, flexibility breeds complexity. And this leads us to the Use/Reuse Paradox.

6.1 – The Use/Reuse Paradox

The problem stems from the following rather simple statement, which is depicted in Figure 1:

Maximizing Reuse complicates Use. (1)

In general, the more reusable we choose to make a software module, the more difficult that same software module is to use. In the extreme, an infinitely reusable module is infinitely difficult to use. The driving force behind this was discussed in Chapter 4, Section 4.3.1. Developing a module that’s reusable demands that module be flexible, and with the increase in flexibility comes a corresponding increase in complexity. Likewise, increasing the ease with which a module can be used, managed, and deployed often decreases a module’s reusability. Dealing with the tension between reuse and use is a complex issue, and often, we fail. Largely, the problem has to do with dependencies.

Figure 1: The Use/Reuse Paradox

6.2 – The Reuse Disclaimer

I recognize that we’ve done a fair job in achieving reuse at certain levels, and we’re much farther along the reuse curve than we were 20 years ago. Today, we have a plethora of frameworks to choose from that aid development. Web frameworks, ORM frameworks, and security frameworks to name just a few. But most of these frameworks are horizontal, not vertical. That is, they address problems related to infrastructure and plumbing code, not business problems. And I want to focus explicitly on vertical reuse, because that’s the unattainable panacea we’ve been struggling with for so long. That’s the broken promise. Why have we struggled to create reusable business software?

6.2.1 – Granularity

Granularity is the extent to which a system is broken down into parts. Coarse-grained modules tend to be richer in behavior than fine-grained modules. Because coarse-grained modules do more, they tend to be larger than fine-grained modules. To maximize reuse, we try composing coarse-grained modules from fine-grained modules. Of course, this results in a lot of dependencies between the fine-grained modules, making the fine-grained modules more difficult to use. In general, we can say the following:

Coarse-grained modules are easier to use, but fine-grained modules are more reusable.

Time for an example. Let’s say we’re creating a module that processes health insurance claims. Let’s keep the business process relatively simple to maintain our sanity. There are four steps in the process. First, the system is fed the claim information. Second, the system checks to make sure it’s not a duplicate submission. Third, the system reprices the claim based on HMO and PPO agreements. Fourth, the system remits payment. A coarse-grained module would perform all four of these steps.

In doing this, we’ve made it easy to use since we only need to invoke one operation to complete the entire process. Additionally, if any part of the process changes, we only have a single module that needs to be built and redeployed. But it’s also more difficult to independently reuse only a portion of this process, such as the remit payment code. The logical solution is to create four fine-grained modules (one for each step in the process) and one coarse-grained module composed of the four others that pulls everything together. The fine-grained modules make things more reusable, but are also more difficult to use since we have to do more to pull them all together to perform a unit of work. Additionally, managing the deployment of four separate modules is more complex than deployment of a single module. In this situation, the coarse-grained module is an example of applying the Module Facade pattern.

6.2.2 – Weight

Weight is the extent to which a module depends on it’s environment. A heavyweight module depends on it’s operating environment, while a lightweight module avoids these dependencies. When creating a module that runs in multiple environments, we’re forced to move the environment specific dependencies (ie. context dependencies) from code to configuration. This makes the module more reusable, but it’s also a bit more difficult to use since the module must be configured for each environment. The Independent Deployment and Container Independent patterns discuss this in more detail.

Designing and configuring a lightweight module is more difficult than simply dropping in a module programmed to operate in that specific environment. In general, we can say the following:

Lightweight modules are more reusable, but heavyweight modules are easier to use.

Let’s elaborate using the example above, where the solution was to create one coarse-grained module composed of four fine-grained modules. If each of these modules only needs to run within a single application in a single operating environment, we can encapsulate all of this environmental code into each module, making each heavyweight. But if we want to reuse these modules across applications and operating environments, then we have to move this code outside of the module and ensure it can be configured for each environment in which we want it to operate.

6.3 – Reuse or Use

The challenge we run into when attempting to create a highly reusable module is to manage the tension between reusability and usability. In our example above, breaking out the coarse-grained modules into fine-grained modules makes it more difficult to use each of the resulting fine-grained modules. Likewise, creating a lightweight module makes using the module more difficult since the module must be configured each time the module is used.

In general, we can say that fine-grained modules have more module dependencies and lightweight modules require configuration based on context. Each makes a module more reusable, but also more difficult to use. The key is to strike a balance.

6.4 – Class and Module Design

Almost all principles and patterns that aid in software design and architecture address class design. Identifying the methods of a class, relationships between classes, and a system package structure are all examples of class design. Since most principles and patterns emphasize class design, it’s no surprise that the majority of developers spend their time dealing only with class design issues. Other examples of class design include deciding if a class should be a Singleton, determining if an operation should be abstract, or deciding if you should inherit from a class versus contain it. Developers live in the code, and are constantly dealing with class design issues.

Making good use of object oriented design principles and patterns is important. Accommodating the complex behaviors required by most business applications is a challenging task, and failing to create a flexible class structure can have a negative impact on future growth and extensibility. But class design is just one piece of the software design and architecture puzzle. The other piece of the puzzle is module design. Module design represents the deployable units composing a software system. Module design is equally important as logical design, and module design is all about dealing with the design paradigm of modularity.

6.5 – Benefits of Modularity

Figure 2: Benefits of Modularity

Almost all discussions on modularity mention reuse as a prime advantage. But there are other advantages, of course. There is a reduction in complexity that helps ease the maintenance effort, as we discussed in Chapter 5. Cohesive modules encapsulate behavior and expose it only through well-defined interfaces. Because modules are cohesive, change is isolated to the implementation details of a module as discussed in Chapter 4. Because behavior is exposed through interfaces, new modules containing alternative implementations can be developed without modifying existing modules. There are other benefits to modularity that extend beyond design to runtime, as well. These include the ability to hot deploy new modules without restarting an application and deploy multiple versions of a module. But from the design perspective, modularity helps increase reuse, ease maintenance, and increase extensibility.

6.6 -Modular Tension

Let’s explore the tension between use and reuse through an example. Assume we define an interface to decouple client classes from all classes implementing the interface. In doing this, it’s easy to create new implementations of the interface without impacting other areas of the system. The principle surrounding this idea is the Open Closed Principle – systems should be open for extension but closed to modification. For more information on OCP, see Appendix A and the discussion on the SOLID design principles. Logical design makes extending the system easier, but it’s also only half of the equation. The other half is how we choose to modularize the system.

Let’s assume the interface we’ve created has three different implementations, and that each of the implementation classes have underlying dependencies on other classes. We’re faced with a contentious issue. On one hand, grouping all the classes into a single module guarantees that change is isolated to only that module (ie. easier to use and maintain). If anything changes, we’ll only have one module to worry about. Yet, this decision results in a coarse-grained and heavyweight module (ie. harder to reuse), and a desire to reuse a subset of that module’s behavior leaves us with one of two choices. Duplicate code or refactor the module into multiple lighterweight and finer-grained modules. In general, logical design impacts extensibility, while physical design impacts reusability and useability.

As we refactor a coarse-grained and heavyweight module to something finer-grained and lighter weight, we’re faced with a set of tradeoffs. In addition to increased reusability, our understanding of the system architecture increases! We have the ability to visualize subsytems and identify the impact of change at a higher level of abstraction beyond just classes.  Grouping all classes into a single module may isolate change to only a single module, but understanding the impact of change is more difficult. With modules, we not only can assess the impact of change among classes, but modules, as well.

Unfortunately, if modules become too lightweight and fine-grained we’re faced with the dilemma of an explosion in module and context dependencies. Modules depend on other modules and require extensive configuration to deal with context dependencies! Overall, as the number of dependencies increase, modules become more complex and difficult to use, leading us to the corollary we presented above:

Maximizing reuse complicates use.

Creating lighter weight and finer-grained modules increases reuse but also increases module and context dependencies, while creating fatter modules decreases dependencies but also decreases reuse. Modules that are too lightweight provide minimal value and may require other modules to be useful. Modules that are too heavyweight are difficult to reuse because they do more than what the client needs. Of course, there are other challenges beyond reuse, such as the ability to understand and maintain the system.

Coarse-grained and heavyweight modules may do a good job of encapsulating change to a single module, but understanding the impact of change is more difficult. Conversely, fine-grained and lightweight modules make it easier to understand the impact of change, but even a small change can ripple across many modules. The key is to balance this tension, especially when developing large software systems where these challenges are even more pronounced.

6.7 – Modular Design

Large software systems are inherently more complex to develop and maintain than smaller systems. In addition to increasing reuse, breaking a large system into modules, makes the system easier to understand. By understanding the behaviors contained within a module, and the dependencies that exist between modules, it’s easier to identify and assess the ramification of change. We saw an example of this Chapter 5, Section 5.7.1.

For instance, software modules with few incoming dependencies are easier to change than software modules with many incoming dependencies.  Likewise, software modules with few outgoing dependencies are much easier to reuse than software modules with many outgoing dependencies. This tension surrounding module weight and granularity is an important factor to consider when designing software modules.

Today, frameworks like OSGi aid in designing modular software systems. While these frameworks can enforce runtime modularity, they cannot guarantee that we’ve modularized the system correctly. Correct modularization of any software system is contextual and temporal. It depends on the project, and the natural shifts that occur throughout the project impact moduarlity. As shown in Figure 3, the modularity patterns help address the design paradigm surrounding modularity, and provide balance in dealing with the tension between use and reuse.

Figure 3: Modularity Patterns

6.8 – Conclusion

Modularity patterns provide guidance and wisdom in helping design modular software. They explain ways that we can minimize dependencies while maximizing reuse potential. They help balance module weight and granularity to make a system easier to understand, maintain, and extend. For those who have attempted to design modular software, it’s common for modularity to drive the class design decisions.

The chapters that follow present a list of modularity patterns that I’ve used on past projects. These patterns also build atop proven object-oriented design concepts. Some of the patterns, such as Manage Relationships, may be intuitively obvious, while others less so. Certainly, other patterns exist that are not listed, and I hope as you discover your own patterns, you’ll add share them with others. I’m also hopeful that technology specific idioms emerge that show how these patterns can be applied using specific module frameworks and technologies, such as OSGi. Before we delve to deeply into the pattern discussions, I’d like to walk through a sample system to illustrate the wonders of modularity.