Yet module design and service design are incredibly complementary in other ways, as well. In Chapter 2, we defined a module and illustrated how modules are excellent candidates for intra-process reuse, while service reuse crosses process boundaries. In Chapter 4, we talked about modularity in a broader architectural context, illustrating how it helps us achieve the goal of architecture. Before we progress onto the pattern discussion, it’s important to discuss modularity in the context of other architectural principles, and illustrate how modularity helps us realize many of the benefits we’ve discussed thus far.

7.1 – All The Way Down, Revisited

Figure 7.1 – Social Architecture All The Way Down

In Chapter 4, I talked about how important it is that we “architect all the way down”. It helps increase transparency and understanding between developers and architects by emphasizing a lot of the middle ground that noone ever seems to focus on. It’s just another reason why modularity is so important. I used the diagram in Figure 7.1 to illustrate the point. Look at the huge gap that exists if we don’t focus on the stuff highlighted by the gray bar in the middle.

One reason I like this visual is that it illustrates the social aspect of software architecture. Yet, there are other significant advantages to architecture all the way down that we haven’t explored yet. Another is structural flexibility.

7.2.1 – Structural Flexibility – Different Entities, Different Purpose

Another benefit of module design in filling that middle ground is that modules can offer different benefits than classes and services. The diagram in Figure 7.2 illustrates some of the capabilities of different types of entities, which is a variation of the diagram we used when defining a module in Chapter 2.

Figure 7.2 – Structural Architecture All The Way Down

For example, classes are pretty easily reused within an application, but because classes aren’t a unit of deployment, it’s difficult to use them across applications. Of course, intra-application reuse is a sweet spot of services, since they’re method of invocation is through some distributed protocol (SOAP, HTTP, even RMI/IIOP). Yet because services are invoked remotely, we typically like to make them coarser-grained because of the performance implications of distributed requests. So this begs the question – If a service is too coarse-grained to reuse (ie. it does more than what we want), how do I reuse some desirable behavior across applications? Well, without modules, our only other choice is the class. Since a class can’t be reused intra-process, we do one of either two things. Expose it as a service or copy the class (ie. the code). Given the circumstances, neither of these may be ideal. Another option is desirable.

Modules represent that other option. They are a finer level unit of granularity than services, and are a unit of deployment. Since each of these different types of entities are units of composition, we have tremendous flexibility in how we assemble applications. Possibly more important though is our increased ability to accommodate the architectural shifts that naturally occur as requirements evolve. Because one of the most difficult aspects of architecture is designing entities at the right level of granularity.

7.2 – Granularity, Architecture’s Nemesis

Granularity is the extent to which a system is broken down into it’s behavioral entities. Coarse-grained entities tend to be richer in behavior than fine-grained entities. Because coarse-grained entities do more, they tend to be larger than fine-grained entities, and are easier to use. In general, a fine-grained entity is more reusable than a coarse-grained entity because it does a little bit less.  If it does less, then it’s more likely to apply across a wider variety of usage scenarios. We spent some time discussing this tension in Chapter 6. Now, I’d like to go a bit deeper which will help more clearly illustrates the concept of “architecture all the way down.”

Unfortunately, while there are a lot of patterns, principles, and idioms that offer really good guidance surrounding many aspects of architecture and design, there isn’t a lot of guidance surrounding granularity. Let’s start with a really simple example that illustrates the challenge.

7.2.1 – A Really Simple Example

Consider the following save method:

public class Person {
private String firstName;
private String lastName;
private String ssNumber;
public void save() {
ValidationErrors errors = this.validate();
      if (errors != null) {
         //handle the errors somehow
      } else {
//save to the database.
      }
}
private ValidationErrors validate() {
//perform validation
}
}

As can be seen, the save method invokes the validate method before saving the information to the database. Generally speaking, this makes saving the information easier (and possibly safer) since whatever invokes save doesn’t have to worry about validating. And this seems to be a good thing.

Yet, what happens when a new consumer of this method wants to save the information but apply a slightly different set of rules than the current validation method provides? Here’s where the existing save method is simply too coarse-grained. It does just a little bit too much. We’ve made it easier to use, but also less reusable.

There are quite a few variations I could go with at this point to improve the design. One would be to make the validate method public, and have clients invoke the validate method before invoking save. Then, if validate didn’t do what it was supposed to, clients could always opt out of invoking the validate method and apply their own validation before invoking save. Certainly less safe though because validation is optional.

Yet another alternative might be the following:

public class Person {
   private String firstName;
   private String lastName;
   private String ssNumber;
   public void save(Validator validator) {
      ValidationErrors errors = validator.validate();
      if (errors != null) {
         //handle the errors somehow
      } else {
         //save to the database.
      }
   }
}

This seems to offer the best of both worlds. I now require a Validator be passed in before I can save, and if the Validator is an interface, I can certainly allow clients to pass in the Validator they need. Of course, I could easily pass in a NOP validator.

Whatever…there are lots of different options, and each has their own set of tradeoffs. I’d like to avoid extensive debate surrounding which approach to save and validation is best and instead focus on the main point here, which is that achieving the right level of granularity for an entity can be very challenging. It requires more than just looking at the problem from a code level viewpoint and demands that we possess contextual information that will help us answer the question, “What’s the right level of granularity?”

7.2.2 – Bring It Up A Level

The code above was a pretty simple example that illustrates the challenges of granularity at the method level. But the same challenges exist when developing classes, packages, modules and services, and we’ll see an example of this shortly. So what I really want to know is

How do I determine the appropriate level of granularity?

This is the million dollar question. Because granularity is a significant inhibitor to creating reusable software that’s easy to use. (Oh, and managing dependencies too.) If something does too much, it’s less reusable. If something does too little, it’s more difficult to reuse.

MAKE MENTION OF ROTTING DESIGN AND DEPENDENCY MANAGEMENT.

The diagram in Figure 7.1 does a good job illustrating one view of granularity. Services are more coarse-grained than modules which are more coarse-grained than packages which in turn are slightly more coarse-grained than classes. This begins to help answer the question, “What’s the right level of granularity?”

If I’m concerned that a service I create is simply too coarse-grained and fails to maximize its reuse potential, I can break the behaviors of the service out into modules that are a bit finer grained and more reusable (of course, one might consider doing this in general). Then I can compose the services from the modules and reuse the modules across services. The result is different entities at different levels of granularity that lends tremendous flexibility to how I compose, use, and reuse software entities.

This provides some guidance on the level of granularity at which different types of software entities should be defined. However, it still doesn’t offer enough guidance to determine the right level of granularity for the save method in our example above.

7.2.3 – Another Dimension

There’s another dimension that is relevant, and that ties in nicely (though in a rather subtle way) to the initial coding example. It has to do with the way we traditionally layer our software. The diagram in Figure 7.3 illustrates the typical architectural layers commonly found in many systems. Generally, as we move down the layered hierarchy, entities become finer grained.

Figure 7.3 – Architectural Layers

While this isn’t absolute, and depends on your architectural principles and style, entities in higher level layers tend to be more coarse-grained than entities in lower level layers because they are an amalgamation of their own behavior and the behavior of entities in the lower level layers.

Armed with this information, we can determine the right level of granularity for the save method as long as we understand in which layer the code containing the save method lives. If it’s in the data access layer, and we place architectural constraints on business rules living in the data access layer, then we shouldn’t have any validation code that lives in the data access layer. If the save method lives in a class in the domain layer, however, it may be suitable for the save method to contain validation code in the domain layer and invoke another method in the data access layer that actually performs the save operation.

7.2.4 – The Complete Picture

At this point we can make the following general statements:

• Entities in higher level layers are more coarse-grained than entities in lower level layers.
• Services are coarser than modules which are coarser than packages which are coarser than classes.

And if we combine these two ideas into a single thought, and overlay the two diagrams, we can begin to visualize the level of granularity for different types of software entities. Figure 7.4 illustrates how entities become finer-grained as we move from both left to right and top to bottom.

Figure 7.4 – Granularity and Architecture All The Way Down

Certainly this doesn’t offer hard guidance. Realistically, there are very few architecture and design principles and patterns that are universally applicable. In fact, I’m not certain you’d ever create a “presentation service.” But certainly you might create a data service that is composed of multiple data access modules. And you might have separate data services that reuse a data access module. And you might have a business service that uses a data service and is also composed of multiple domain modules. But you definitely do not want a data access module that invokes a business service, nor a business service that references a presentation module.

So this general guidance can serve as useful information to help answer our question, “How do I determine the appropriate level of granularity for a software entity?” And it can also serve as guidance when establishing architectural principles and constraints that help determine where software entities with specific behaviors should reside. Because without some scheme that will help determine the appropriate level of granularity, it’s hopeless to imagine that we’ll be able to design reusable software entities that are also easy to use.

7.2.5 – A Service Example

Let’s take another example. Say I have a business function called Pay Bill for which I develop a web service that can be invoked by many different consumers. That service happens to be relatively coarse-grained, and performs all the steps involved in paying the bill. These happen to include the following:

• audit bill – apply a discount to the bill based on payee
• check for duplicate – ensure the bill hasn’t already been paid
• remit payment – cut the check
• reconcile payment – reconcile with accounts payable financials

This seems reasonable. We have a nice little service that we can reuse any time we want to pay a bill. Unfortunately, the real world often gets in the way of our idealistic solutions. In fact, there are two problems that will eventually surface, and modularity benefits both scenarios. Let’s start by looking at the first scenario.

What should I do when a different scenario arises that demands I follow a slightly modified Pay Bill function?

As part of the remit step, I have a new payee that demands electronic payment. This is pretty easy actually. I simply modify the service to support electronic payments and then configure the service to context for that specific payee. So how does modularity help here?

If the service is composed of modules, it’s going to be much easier for me to understand the structure of the service, assess the impact of what’s it’s going to cost to change the service, and then introduce a new module (or modify the existing module) to provide the new capability. Without modules, I’m simply wading through the code trying to figure out all of these things. Now, the 2nd scenario.

What should I do when I want to reuse just one step of the Pay Bill function?

Let’s say another new requirement emerges. Whereas traditionally bills were entered by data entry personnel, we now have to support electronic delivery of bills. We also know that bills delivered electronically are often duplicates. It’s just one of those business things, you know? If we don’t pay the bill on the day it’s received, the billing party sends us the bill again, asking for payment. So we need to check for duplicates before we save the bill to the database and prepare it for processing. What do we do?

We could take the same approach as before and modify the Pay Bill service so that the duplicate check could be invoked separately from the higher level pay bill function. But that’s a bastardized design solution. Why? We are exposing behavior of the Pay Bill service that shouldn’t be exposed. The API is coarse-grained in some areas and fine-grained in others.

Maybe exposing the finer-grained capabilities of the Pay Bill function isn’t a severe compromise, but it is a compromise nonetheless. And we are forced to compromise because of architectural inflexibility. We don’t have architecture all the way down, and are therefore left with limited choice as to how we support the new requirement. We can either modify the service to reuse what we know is already encapsulated within the service, or copy the code to create something new. But those are the two options we have, and neither may be ideal.

As we continue to hack the system in this manner, with a multitude of other similar changes, the design will rot. Eventually, our Pay Bill service is transformed into a utility that performs all general-purpose bill functions. It’s API is a mix of coarse-grained and fine-grained operations. It’s become the monolith that we’re trying to avoid. While the Pay Bill service is still pretty reusable (it does everything now), it isn’t that usable. That tension between reuse and use that we discussed in Chapter 6 surfaces again.

Our decision to modify the Pay Bill service to expose the duplicate check was driven by one thing – ease. It was our easiest option. Really, it was our only option. But it isn’t the best option.

If we architect all the way down, we have another option. A Pay Bill service composed of modules that audit the bill, check for duplicates, remit payment, and reconcile the payment means we can choose the desirable solution over the easiest short term choice. We can avoid the long term compromises that degrade architectural integrity.

Shown in Figure 7.5, we see the service composed of modules. If we have a check for duplicates module, we can simply reuse that module in the new service or application that’s going to process electronic bills. Or we might expose the capabilities of the check for duplicates module as a separate service. We have multiple reuse entry points, as well as different levels of granularity through which our software entities can be managed, deployed, built, and much more.

My point here isn’t to debate the original design nor the decisions made to reuse the check for duplicates functionality in another service or application. Certainly, debating architectural and design decisions is a favorite past-time of developers. There are a variety of different ways that we can support new requirements as they emerge. Some are better than others and all are contextual.

Figure 7.5 – Service Composed of Modules

7.3 – An Alternative View

Figure 7.X –

7.6 – Summary

The notion of architecture all the way down gives us options, and these options help maintain architectural integrity. They increase our options when making decisions and allow our system to accommodate unforeseen architectural shifts. I can modify an existing application or service to give me what I need. Or I can reuse an existing module that’s already available for me. Or I can compose a new service from an existing set of modules. Or I can break apart existing modules to create new modules that result in new services. And I can do a lot of this refactoring without significant impact on the existing code. We’ll see this idea in action in the next chapter, as well as throughout the pattern discussions.

8.1 – Why No OSGi?

The example that follows doesn’t use OSGi. One would think that if I’m going to modularize my system, I’d want to use a proven module framework, such as OSGi. There are a couple of reason I chose not to.

• While OSGi is a module system, OSGi will not help you design more modular software. In other words, it doesn’t help us with the design paradigm. Among other things, designing modular software requires that we understand the weight and granularity of individual modules (discussed in Chapter 6), and use the right techniques to manage the coupling between modules (dependencies are discussed in Chapter 5). In other words, designing good software is our job. Tools and technologies may help, but make no guarantee. For more on the differences between the runtime and development model, see Chapter 3.
• Many of us aren’t able to leverage OSGi today because the platforms and languages we use don’t support it. I want to use the same tools and techniques that we can leverage in the enterprise right now.
• Mostly though, I didn’t want to shroud the examples in technology. Instead, I want to focus on pure modularity. For the interested reader, you can view the final version of the system, in all it’s OSGi glory, in Appendix X. You’ll find that the module structure hasn’t changed. Instead, OSGi allows us to take advantage of the runtime benefits of modularity, such as hot deployments.

In general, I want to focus this chapter on the design paradigm, not the tools and technologies. So instead of answering, “How do I use OSGi?”, I want to focus on “How do I develop a system with a modular architecture”. Hopefully, that resonates with you.

8.2 – Let’s Get Started – The System

Interestingly, I’ve found that when designing modular software, it’s tough to identify the modules early in the lifecycle. Instead, shifts typically occur, and as things unfold, the modules become more apparent as development progresses. With a SOLID OO design, it’ll make it much easier to move things around and create new modules. So to start, while the system is small, I favor coarser-grained and heavier-weight modules.

As specific needs emerge, we’ll break larger modules out into a bunch of finer-grained and lighter-weight modules that address specific functional and non-functional needs. Remember, the material in the first six chapters describe the essence of my motivation here. It’s deep and very interesting, and impacts how we understand the software, maintain the system, reuse software entities, and more. If you haven’t read any of the previous material, you might want to check it out now, or you can wait and see what I’m talking about, as we’ll experience this phenomenon as we move through the exercise.

Here’s a simple, high-level description of the system we’ll develop. It’s the common bill payment sample system often used.

We’ve been asked to develop a system to handle payment of of bills. Prior to paying the bill, the system should apply a discount to the bill in an amount that has been negotiated with the payee (we call this the process of auditing the bill). Applying this discount is a fairly complex process, and a 3rd party vendor has been commissioned that will apply this discount. Additionally, we must integrate with a legacy financials system that must be fed payment information for reconciliation.

We’ll flesh out additional details as development progresses.

8.3 – Version One

The initial version for this system uses Struts as the web framework, and packages everything into a single WAR file. The initial class diagram can be seen in Figure 1. It’s not a complete class diagram, but it does show the main set of classes. I’ve greatly simplified the system for purposes of example.

Figure 1: Initial Version Class Diagram

As you can see, there are some Action and ActionForm classes that leverage Struts. There are also a couple of JSP you don’t see here. A Customer has a list of Bills, and each Bill has a reference to an AuditFacade and Payment class. The AuditFacade integrates with the 3rd party vendor software that applies the discount, and the Payment class integrates with a legacy financial system. Of particular interest, note the bi-directional relationship between Bill and the AuditFacade and Payment classes – a sure sign of a problem that will haunt us later. But don’t worry, we’ll fix it.

For the sake of simplicity, I’ve hardcoded the database into the data access layer, which is represented by the Loader interfaces. This way, you can experiment with the system without running any scripts to create the database. If you are compelled to do so, feel free to add a real database backend. It wouldn’t be very difficult, but it’s not what I want to focus on here.  While we’ll show some code snippets as the example progresses, I have setup a Google Code Repository that shows each step in the evolution. That repository can be found at http://code.google.com/p/kcode/source/browse/#svn/trunk/billpayevolution/billpay, and each of the subprojects represent a single step in the example.

8.4 – First Refactoring

So packaging everything up into a single WAR file for deployment certainly isn’t modular. In most systems we develop, we try to design layers that encapsulate specific behaviors and isolate certain types of change. Typical layers include a UI layer, a business or domain object layer, and a data access layer. In this system, we have these three layers. The Struts action and Form classes, along with the JSP, represent a part of the UI layer. The UI layer is represented by the red classes in Figure 1. The Customer, Bill, Name, AuditFacade, and Payment form the business object layer, and the Loader classes form the data access layer. These classes are shown in blue in Figure 1. Now, here’s a key statement that you need to take with you.

If I truly have a layered system, then I should be able to break out each layer into a separate module where modules in the upper layers depend on modules in lower layers, but not vice versa.

If you try this for one of your systems, it’s likely you’ll find it’s not so easy. Most development teams feel they have a layered architecture, but in reality, they don’t because somewhere deep within the bowels of the system lies an import or reference to a class higher up in the food chain that we aren’t aware of.

Figure 2: Applying the Physical Layers Pattern

In fact, if I really do have a layered system, then I shouldn’t have to change anything other than my build script to break the layers out into separate JAR files. If I do have to change more than a build script, then I didn’t have a layered system to begin with, and I should perform some architectural refactoring to clean things up. Anyway, the end result is relatively simple to understand. No code changes. Only a build script change so that certain classes are allocated to specific modules. The structure is shown in Figure 2. This refactoring was an example of applying the Physical Layers pattern.

Listing 1 illustrates a portion of the initial build script where all of the classes were bundles into the WAR file. Listing 2 shows the changes we’ve made to the build script to create modules for the various layers. Here, we only show the business object layer. We created a new build target where we create the module and then added a new line to the dist target where that module is now included in the WAR file.

<target name="compile" depends="init">
   <javac srcdir="${javasrc}:${testsrc}" destdir="${build}">
      <classpath refid="project.class.path"/>
     </javac>
</target>

<target name="bundle" depends="dist">
   <mkdir dir="${deploy}"/>
   <war destfile="${deploy}/billpay.war" webxml="WEB-INF/web.xml">
       <fileset dir="jsp"/>
       <webinf dir="WEB-INF">
          <exclude name="web.xml"/>
          <exclude name="lib/servlet-api.jar"/>
       </webinf>

        <classes dir="${build}"/>
    </war>
</target>

Listing 1: Initial Build Script

<target name="compile" depends="init">
   <javac srcdir="${javasrc}:${testsrc}" destdir="${build}">
      <classpath refid="project.class.path"/>
   </javac>
</target>

<target name="dist" depends="compile">
    <mkdir dir="${bindist}"/>
    <jar jarfile="${bindist}/bill.jar" basedir="${build}" excludes="com/extensiblejava/bill/test/**, com/extensiblejava/ui/**"/>
</target>

<target name="bundle" depends="dist">
   <mkdir dir="${deploy}"/>
   <war destfile="${deploy}/billpay.war" webxml="WEB-INF/web.xml">
      <fileset dir="jsp"/>
      <webinf dir="WEB-INF">
          <exclude name="web.xml"/>
          <exclude name="lib/servlet-api.jar"/>
      </webinf>
      <lib dir="${bindist}" excludes="test.jar"/>
      <classes dir="${build}" includes="com/extensiblejava/ui/**"/>
   </war>
</target>

Listing 2: Build Script with Physical Layers

8.4.1 – Wrapping Up and Getting Ready for the Next Refactoring

This first refactoring was quite simple, but has significant implications. Foremost, it proves that my class level architecture was decent. I was able to break the system out into modules for the various layers without having to change a bunch of code. Really, that’s the reason why it was so simple…because the design was decent. Had there been violations in the layered structure, it would have been significantly more difficult pulling off this refactoring because I would have been forced to remove the unwanted dependencies.

Yet, as we’ll see, the existing design may meet the needs of today, but it’s going to have to evolve as change emerges. In the second refactoring, we’ll take a look at what we need to do to integrate with another auditing system, and how modularity can help us do this. And as we progress, the amazing transformation of a system lacking modularity to a highly modularized version will unfold. In the next couple of steps, we’re going to apply two refactorings using two different modularity patterns – Abstract Modules and Acyclic Relationships. First, we’re going to separate the bill and audit functionality out into separate modules so we can manage (develop, deploy, etc.) them independently. Second, we’re going to remove the cyclic dependency between these two modules.

8.5 – Second Refactoring

In the initial class diagram, the Bill class has a bi-directional relationship to the AuditFacade class. This design has two fundamental flaws. The Bill is tightly coupled to the concrete AuditFacade class and the relationship is bi-directional. Bad all around! This can be seen in the following code snippet in Listing 3 illustrating the Bill’s audit method.

public void audit() {
   AuditFacade auditor = new AuditFacade();
   this.billData.setAuditedAmount(auditor.audit(this));
   this.persist();
}

Listing 3: Audit method of the Bill Class

Notice that the audit method actually creates the AuditFacade, calls the audit method, and passes a reference to Bill. Ugly. Let’s clean this up a little bit. While there are obvious technology reasons why we need to clean this up, there is also a motivating business force.

The system needs to go live with the current vendor’s auditing system, but the business has indicated that they aren’t renewing the contract with the vendor and are in ongoing negotiation with another vendor. The contract expires in 6 months, but we deliver the initial version of the system in 3 months.

So, three months after deployment, we know we’ll need to swap out auditing systems.

We’re going to apply the Abstract Modules pattern, which states that we should depend upon the abstract elements of a module. We’ll start by refactoring the AuditFacade class to an interface and create a separate AuditFacade1 implementation. This solves the first half of our problem – the tight coupling between the Bill and AuditFacade implementation. The result is the class diagram shown in Figure 4.

Figure 4: Refactoring AuditFacade to an Interface

Of course, the Bill can no longer create the AuditFacade implementation. Doing so would compromise the work we’ve done since Bill would still be coupled to AuditFacade1. To deal with this challenge, the AuditFacade interface is now passed into the Bill’s audit method, allowing us to swap out AuditFacade implementations. The modified audit method is shown in Listing 4.

public void audit(AuditFacade auditor) {
   this.billData.setAuditedAmount(auditor.audit(this));
   this.persist();
}

Listing 4: The New Audit Method Accepting the AuditFacade Interface.

If we take this flexible class structure and continue to deploy in the single bill.jar module, we have the flexibility to swap out AuditFacade implementations at the class level, but we’re still required to package everything up into a single bundle and deploy it as a single module. So what we really must do is separate the audit functionality out into a module separate from the bill. Separating the AuditFacade interface and AuditFacade1 implementation out into separate modules results in the diagram illustrated in Figure 5. Here’s where the bi-directional relationship between Bill, the AuditFacade interface, and AuditFacade1 implementation rears it’s ugly head. We have a cyclic dependency between our bill.jar and audit.jar modules. Remember our discussion from Chapter 5 – excessive dependencies are bad and cyclic dependencies are especially bad. We need to fix this problem.

Figure 5: Abstract Modules

This change will also be reflected in the build file that packages up these modules, as shown in Listing 5, where we can see the audit.jar module created.

<target name="dist" depends="compile">
   <mkdir dir="${bindist}"/>
   <jar jarfile="${bindist}/audit.jar" basedir="${build}" includes="com/extensiblejava/audit/**"/>
   <jar jarfile="${bindist}/bill.jar" basedir="${build}" excludes="com/extensiblejava/bill/test/**, com/extensiblejava/ui/**"/>
</target>

Listing 5: The Build Script Creating the Audit.jar Module

8.6 – Third Refactoring

Since cyclic dependencies are so bad, we need to remove the cyclic dependency between the bill.jar and audit.jar modules. To do this, we’ll apply the Acyclic Relationships pattern, which states that module relationships must be acyclic. We’ll introduce an additional abstraction, called Auditablethat our Bill class implements. Upon applying this little trick, we can see that we have now removed the bi-directional relationship between Bill and the AuditFacade interface, as shown in Figure 6.

Figure 6: Acyclic Relationships between Classes

Whereas previously the AuditFacade accepted a Bill to the audit method, it now accepts an Auditable type. The new AuditFacade interface can be seen in the code snippet shown in Listing 6.

package com.extensiblejava.audit;
import java.math.*;
public interface AuditFacade {
   public BigDecimal audit(Auditable auditable);
}

Listing 6: AuditFacade Interface

The Bill will pass itself to the AuditFacade interface as an Auditable type. The key element at this point is how we allocate these classes to their respective modules. Here’s a simple rule to abide by when determining how to allocate classes and interfaces to modules.

Interfaces should be closer to the classes that use them, and farther away from the classes that implement them.

The Extensibility Patterns discuss this idea in more detail and we’ll leverage this concept later to address another problem that has surfaced. But applying this rule now, it’s clear that Auditable should be bundled with the AuditFacade interface, not in the same module as the Bill class. While we haven’t talked about the new financial.jar module, we applied a similar refactoring to the Payment class by implementing a Payable interface, and breaking it out into a separate module. Allocation of classes to modules is done at build time, so after modifying our build script, we have the module structure as shown in Figure 7.

Figure 7: Acyclic Relationships

As I’m sure you’ve noticed at this point, we broke our rule above and bundled the AuditFacade1 implementation close to the interface it implements, which is the other problem that has surfaced. The problem, however, is that if we apply our rule in this situation, we’d bundle the AuditFacade interface and Bill class in the same bundle, which won’t work since it would result in the cyclic dependency between the bill.jar and audit.jar modules that we’ve worked so hard to remove.

8.6.1 – Wrapping Up and Getting Ready for the Fourth Refactoring

Since our last recap, we applied two separate refactorings. The first was to separate the bill and audit functionality out into separate modules. The second was to remove the cyclic dependency between the bill and audit modules.This offers us decent flexibility. We can introduce a new AuditFacade implementation, knowing that we’d only need to rebuild the audit.jar module. We can test the audit.jar module independent of any other module because it doesn’t have any outgoing dependencies. And if we wanted to, we could deploy the audit functionality separately (ie. we can reuse it elsewhere). So overall, some decent progress.

But some problems remain. If we really want the ability to swap out Audit systems, bundling the AuditFacade interface and AuditFacade1 implementation into the same module doesn’t give us the flexbility we need. While we can easily create a new AuditFacade implementation and allocate it to the audit.jar module, this requires us to redeploy audit.jar unnecessarily. In the next post, we’ll explore how we can change the module structure to allow a new AuditFacade implementation while also allowing the existing AuditFacade1 implementation to be removed from the system, and a new module to be deployed. Thus far, we’ve applied the following patterns:

• Physical Layers
• Abstract Modules
• Acyclic Relationships

And while we haven’t specifically mentioned it, we could also easily apply the following at this point:

• Levelized Modules
• Levelized Build

8.7 – Fourth Refactoring

At this point, if we want to deploy a new AuditFacade implementation, we have two options. We can bundle it in the existing audit.jar module or deploy it in a new module. The second approach offers us some flexibility because we wouldn’t need to modify any classes in the existing audit.jar module. Unfortunately, we cannot remove the existing AuditFacade implementation because it’s bundled in the same module as the AuditFacade interface. Obviously, if we deploy a new AuditFacade implementation, we need the interface. Additionally, if both of the implementations of AuditFacade live in the same module, we can’t rip one out (ie. uninstall) when we add the other. In other words, because they are in the same module, they can only be managed together.

To solve this tricky little challenge, we’re going to apply the Separate Abstractions pattern, which states that we should separate abstractions from the classes that realize them. We actually applied this pattern when allocating the Auditable interface to the audit.jar module, but we didn’t apply it to the AuditFacade1 class. So we’ll apply it to the AuditFacade1 implementation, as well as a new AuditFacade2 implementation. First, we have to answer the following question.

Where do we put the AuditFacade interface so that new implementations of the interface can be managed separately from the interface and other implementations?

Our answer can be found by looking at the module structure shown in Figure 9. Note that I’ve also included the financial.jar module in this diagram, which was actually introduced in a previous step when we introduced the Payable interface. We’ll use this soon, but let’s ignore it for now.

Figure 9: Separate Abstractions

We separate the AuditFacade interface out into its own module – auditspec.jar.The AuditFacade1 and AuditFacade2 implementations are also placed in their own modules. Aside from moving a few things around, no real coding changes are necessary beyond the new AuditFacade2 implementation we’ve created. Note that we’ve also separated each implementation out into different packages to avoid splitting packages across modules. After adding this new class, we modified our build script to allocate the classes to the appropriate modules.

8.7.1 – A Note on the Benefit of OSGi

At this point, we have increased the architectural flexibility of our system. The AuditFacade implementation classes are allocated to separate modules, allowing us to manage the two independently. From a maintenance perspective, we can work on each of the implementations separately, resting assured that changes to one implementation won’t negatively impact changes to the other. We can also test each independently and reuse each independent of the other.

This change also increases the resiliency of the bill.jar module since it’s no longer tightly coupled to any AuditFacade implementation. We can test the bill.jar module using only the auditspec.jar module by creating a mock implementation of AuditFacade for testing purposes. While the billpay.jar module is still dependent on the financial.jar module, we’re going to solve that problem in the next refactoring. In general, we have completely eliminated the dependencies between the bill.jar module and the AuditFacade implementations in the audit1.jar and audit2.jar modules. Remember, at the beginning of Section 7.6, these modules containing the bill and audit behavior had a cyclic relationship.

While this example doesn’t leverage OSGi, it is important to point out the benefit that OSGi can provide. Because we don’t have a runtime module system we still need to deploy these modules within the WAR file. Changes to any of the modules still requires that we redeploy the entire WAR, even though it’s possible to only rebuild the modules that have changed. The presence of OSGi, however, brings the same degree of flexibility to the runtime that we have at development time, which we discussed in Chapter 3. With OSGi, we would be able to install and uninstall the audit1.jar and audit2.jar modules without redeploying the entire system. Appendix B shows us how OSGi allows us to do this.

8.7.2 – Wrapping Up and Getting Ready for the Next Refactoring

So we’ve made quite a bit of progress from our initial version of the system, which lacked a modular architecture. By breaking the system out into modules, we ease the maintenance effort and increase overall system flexibility. In the next refactoring, we’re going to to turn our attention to the financial.jar module that focuses on payment. Recall that in the third refactoring in Section 7.6, we separated the Payment class out into a separate module and decoupled Bill and Payment through the Payable interface. In the next step we will examine how we can decouple the bill.jar module from the financial.jar module so bill.jar can be managed, tested, and deployed independent of financial.jar.

Soon, we’re going to focus on the financial.jar module and try to decouple bill.jar from financial.jar so the two are completely independent of each other. I think you’ll really like this one…But first, some additional housekeeping.

8.8 – Fifth Refactoring

Before we decouple the bill.jar module from the financial.jar module, we’re going to talk about exceptions. In general, we need to answer the question.

Where do exceptions belong?

Let’s introduce an AuditException that is thrown if an error is encountered. The Collocate Exceptions pattern says that exceptions should be close to the classes (or interfaces) that throw them. Since the AuditFacade interface throws the exception, as shown in Listing 7, we should put the AuditException in the same module as the AuditFacade interface, which is the auditspec.jar module. Note that if we put the exception anywhere else, we’d create an unwanted dependency between modules.

package com.extensiblejava.audit;

import java.math.*;
public interface AuditFacade {
   public BigDecimal audit(Auditable auditable) throws AuditException;
}

Listing 7: AuditFacade Method that throws an AuditException

8.9 – Sixth Refactoring

The previous refactoring was pretty simple. This next refactoring is rather interesting. Here’s the driver behind this next step.

A new requirement has emerged, and we need to use the bill.jar module in another system. In this other system, we don’t need the financial.jar module to make the payment. We just want to use the bill.jar functionality. So, what do we do?

First, recall our initial version of the class structure shown in Figure 1. While we didn’t spend much time discussing it, in Section 7.6, we introduced a Payable interface and  created the financial.jar module. After we finished fourth refactoring shown in Figure 9. Our goal now is to eliminate the dependency between the bill.jar and financial.jar modules so that we can deploy the bill.jar module without the financial.jar module. We’re going to apply a little trick called escalation, which is discussed in more detail in the Acyclic Relationships pattern. Essentially, we’re going to escalate the dependency to a higher level module. First, let’s take a look at the new class structure that’s going to allow us to remove the module dependency.

Figure 10: Independent Deployment

As shown in Figure 10, we see the new class structure that’s going to allow us to decouple the bill.jar and financial.jar modules. Things are starting to get a bit tricky here, so before we allocate these classes to their respective modules, let’s walk through the steps to illustrate how these classes will callaborate to give us what we need.

We start by passing a BillPayAdapter instance to the Bill as a BillPayer type. The BillPayAdapter is going to manage the relationship between Bill and Payment. When the pay method on Bill is called, it will turn around and invoke a generateDraft method on BillPayAdapter. The BillPayAdapter invokes a generateDraft method on Payment, passing itself in as a Payable. This allows the Payment to callback on the Payable and invoke the appropriate method (either getAmount or getAuditedAmount) on BillPayAdapter. The BillPayAdapter can now query the Bill to obtain the audited amount and pass it back to the payment. Once the Payment has this amount, it can make the payment.

A fairly complex callaboration, but enough to ensure we can decouple the bill.jar module from the financial.jar module. The code for the new BillPayerAdapter class, which mediates (hmm..possibly naming it a mediator would have been better, huh?) the relationship between Bill and Payment, is shown below.

public class BillPayerAdapter implements BillPayer, Payable {
   private Bill bill;
   public BillPayerAdapter(Bill bill) {
      this.bill = bill;
   }

   public BigDecimal generateDraft(Bill bill) {
      Payment payer = new Payment();
      return payer.generateDraft(this);
   }

   public BigDecimal getAmount() {
      return this.bill.getAmount();
   }

   public BigDecimal getAuditedAmount() {
      return this.bill.getAuditedAmount();
   }

Listing 8: The BillPayerAdapter Class

Now it’s time to allocate these classes to the appropriate modules. The key decision here is where we place the BillPayerAdapter class. Because this controls the Bill and Payment interaction, we can’t put it in either of those two modules. In fact, we’re going to create a new module that contains the BillPayAdapter. The new module, which we’ll call billpay.jar, will depend on both the bill.jar and financial.jar modules, as shown in Figure 11. This refactoring is an example of the Indpendent Deployment pattern, which states that modules should be independently deployable units. That’s exactly what we’ve done.

Figure 11: Independently Deployable Modules

For the most part, our system is complete now. But we do have one nasty little problem that we haven’t solved yet. We have two different AuditFacade implementations, but the current system hardcodes creation of AuditFacade1 within the AuditAction class. All this flexibility compromised by a single line of code. Unnerving how that happens! We really do have to architect all the way down.  Let’s fix this problem.

8.10 – Seventh Refactoring

The problem of creating instances is pretty common, so the solution is relative easy. There are a lot of different ways to do this, but we’re going to take the easiest route (and not the most flexible mind you) by creating an AuditFacadeFactory class. This refactoring is an example of the Implementation Factory pattern, which states that we should use factories to create a modules implementation classes. Now, instead of the AuditAction creating the AuditFacade implementation, it invokes the factory and is returned an instance, as shown in Listing 9. An alternative, and possibly a better solution, would have been to inject the appropriate AuditFacade implementation into the AuditFacade using a dependency injection framework such as Spring.

try {
   bill.audit(AuditFacadeFactory.getAuditFacade(bill));
   request.setAttribute("bill",bill);
   return mapping.fineForward("success");
} catch (AuditException e) {
   throw new ServletException(e);
}

Lisiting 9: The AuditFacade Invoking the AuditFacadeFactory Class

8.11 – The Postmortem

We’ve made considerable progress. Amazing really. From a single monolithic application to a fully modularized architecture with considerable flexibility. But there are a lot of very interesting takeaways that we haven’t talked about yet. A number of positive side affects have resulted that aren’t immediately obvious. In this section, we’ll take a more in-depth look at the impact of modularity.

Through this series of refactorings, we made considerable progress in modularizing the application using a few of the modularity patterns. The diagram in Figure 12 illustrates the progress we’ve made. We started with everything bundled into a single WAR file and wound up with a highly modularized system that satisfied the evolutionary requirements. Aside from the many advantages we spoke about in each post, I want to take a moment to explore a few other thoughts.

Figure 12: Modular Architecture

8.11.1 – A Note On Module Testing

If you’ve explored the system by getting the code from the Google code repository, which we discussed earlier and can be found at, http://code.google.com/p/kcode/source/browse/#svn/trunk/billpayevolution/billpay, you’ll notice that there are a corresponding set of test modules for each module that we’ve created. These can be found in the bin directory after building the final version of the system. Like we do with unit testing, I’ve tried to create a test component for each module in the system. Unfortunately, there’s a flaw in the billtest.jar module.

Similar to unit testing, where we create mocks and stubs to avoid undesirable dependencies, an ideal test module shouldn’t pull in other modules that contain implementation classes. Instead, we should create mocks or stubs to avoid this situation. In other words, a test module should only be dependent on the same set of modules as the module it’s testing. The Test Module pattern discusses this further.

Unfortunately, the billtest.jar module breaks this rule by using the AuditFacade implementations to perform the tests. That means the billtest.jar module is also dependent on the audit1.jar and audit2.jar modules, but the bill.jar module is not. So billtest.jar is really a module integration test, not a module unit test. It could easily be fixed by creating a mock AuditFacade implementation that lived in the billtest.jar module.

This begs another question….

How do we keep track of module relationships so that we recognize when something bad like this happens?

Even for small systems, without a module system like OSGi, it can be incredibly challenging.

8.11.2. – A Note On Managing Modules

Modularizing a system on a platform that doesn’t support modularity is challenging. Modularizing a system on a platform that does support modularity is challenging! One of the greatest challenges is in managing module dependencies. Tracking the dependencies between modules is quite difficult.

This is where module systems like OSGi really help by enforcing the declared module dependencies. In Java today, and most platforms used in the enterprise today, there is no module system available so there is nothing to help enforce modularity. The first unwanted dependency that creeps into our system compromises architectural integrity. This is where a tool like JarAnalyzer can be helpful. You can find JarAnalyzer at http://code.google.com/p/jaranalyzer/. By incorporating JarAnalyzer into the build script, we’re able to more easily manage the dependencies between modules.

Figure 13: JarAnalyzer Visual Output

JarAnalyzer has two output formats. The first is a GraphViz compliant dot file that can be easily converted to an image showing module relationships. The image shown in Figure 12, which includes the test modules, clearly illustrates the problem with the billtest.jar module discussed above.

As can be seen, the bill.jar module has only a single outgoing dependency on the auditspec.jar module. So the module that tests the bill.jar module should not be dependent on any other modules, either. However, if you look at the billtest.jar module, you’ll see that it depends upon the audit1.jar and audit2.jar modules. So instead of using a mock or stub to test the bill.jar module, I got lazy and used the various AuditFacade implementations. Look at a few of the other modules, and you’ll discover that none include additional dependencies beyond the dependencies already present within the modules they test.

The second output format for JarAnalyzer is an html file that provides some key design quality metrics, as well as listing the dependencies among modules. Essentially, it’s a textual view of the same information provided by the visual diagram. I’ve included the Summary header of the JarAnalyzer report for the system, which can be seen in Figure 14. You can also browse the complete JarAnalyzer HTML report for the final version of the system. There is also a version that omits the test modules.

Figure 14: JarAnalyzer HTML Output

Look at the auditspec.jar module. Note that it has 8 incoming dependencies (afferent coupling) and 0 outgoing dependencies (efferent coupling). It’s abstractness is 0.67 and Instability is 0.00. This is a pretty good sign. Why? It’s instability is very low, implying it’s highly resistant to change. It possesses this resistance to change because of the large number of incoming dependencies. Any change to this module may have serious implications (ie. the ripple effect of change). But because it’s quite abstract, it’s less likely to change than a module with a lot implementation classes. The Distance for the module is 0.33 (ideal is 0.00), so we’re not far from where we ideally want to be.

In case you’re wondering about all these metrics I’m rambling about, you might want to take a look at the Martin Metrics. In general, without a utility like JarAnalyzer (or a module framework like OSGi), it would have been incredibly difficult to manage the modules composing this system.

8.11.3 – A Note on Module Reuse

The reuse/release equivalency principles states that the unit of reuse is the unit of release. Modules are a unit of release, and therefore are a unit of reuse. Naturally, the devil is in the details, and we’re going to discuss these details here.

In Chapter 6, we spoke of the tension between reuse and use. That tension is evidently at play here. Earlier versions of the system had coarser-grained modules that were easier to use but more difficult to reuse. As we progressed, we broke these coarser-grained modules out into finer-grained modules, increasing their reusability but decreasing their ease of use. A perfect example of this is the bill.jar module. In the final version, it was quite reusable, since it was only dependent on the auditspec.jar module. However, this came at the price of useability.

To elaborate a bit more. In the sixth refactoring in Section 7.9, we decoupled the bill.jar and financial.jar modules so the two could be deployed independently (ie. increase reuse). But the runtime structure still has some dependencies. In order to reuse bill.jar, we need a BillPayer type. While an alternative BillPayer implementation could be created, the existing implementation is the BillPayAdapter in the mediator.jar module, which also has a relationship to the financial.jar module. This means that to use the bill.jar module without the mediator.jar and financial.jar modules would require a new consuming module to implement the BillPayer interface.

So what do we do if we want to break this runtime coupling? We should move the pay method on the Bill up to the BillPayAdapter class, and get rid of the BillPayer interface. Now the Bill class has no dependency on the BillPayer interface, but it also can’t make payments. Every action has an equal an opposite reaction, heh?

8.11.4 – A Note on The Build

The build was a key element in helping enforce modularity (note: JarAnalyzer helped me manage module relationshps; the build enforced module relationships). Even a framework such as OSGi is only going to manage module relationships at runtime. We talked about the design paradigm in Chapter 3, and it’s why we need really good tools that help us design more modular software. It’s our responsibility to craft the modules, and the build is one way to help put in place a system of checks and balances that help enforce modularity before discovering at runtime that one module has a relationship to another. As part of the third refactoring in Section 7.6, we could have easily refactored our build script to a Levelized Build. If you take a look at the code in the Google code repository, you’ll notice how the build changes after we applied the Acyclic Relationships pattern.

This means that as we build each module, we include only the required modules in the build classpath. We speak more about this when discussing the Levelized Build pattern. To elaborate on this a bit, we build the modules with fewer outgoing dependencies first, following by those that are dependent on the modules previously built. For example, when we build the auditspec.jar module, we include nothing else in the build classpath because the auditspec.jar module doesn’t require anything. When we build the audit1.jar module, we’ll only include the auditspec.jar module in its build classpath. This pattern recurs throughout the remainder of the build scripts. Introducing a module dependency that violates the dependency structure enforced by the build results in a failed build.

8.11.5 – A Note on Object Orientation

The way we managed, massaged, and modified module relationships was through object-oriented techniques. By introducing interfaces and abstraction and allocating them to their respective modules, we were able to significantly change the module structure of the system. While we used object-orientation to do this, that is not a prerequisite. We could just as easily have used other techniques, such as aspects (AOP). The key is that we are managing the dependencies between modules.

8.11.6 – The Final Wrap

I recognize that this was a pretty lengthy and involved tutorial. But it’s necessary to leverage a relatively involved example to show the positive affect of a modular architecture. It also shows how we can use many of the modularity patterns, which we’ll now turn our attention towards. Again, for the version of the system that uses OSGi, see Appendix B.

What happened? Reuse was supposed to save software development. In the early 90’s, object-orientation promised to save us. It hasn’t. In the late 90’s, component-based development promised to save us. It didn’t either, and the movement died. Shortly after the turn of the millenium, SOA promised to save us. It didn’t, though we’re still trying. Why is reuse so difficult?

Foremost, reuse is achieved by designing flexible software entities that can be tailored based on need. In other words, we reuse an entity by configuring it to a specific context. But as we’ve seen, flexibility breeds complexity. And this leads us to the Use/Reuse Paradox.

6.1 – The Use/Reuse Paradox

The problem stems from the following rather simple statement, which is depicted in Figure 1:

Maximizing Reuse complicates Use. (1)

In general, the more reusable we choose to make a software module, the more difficult that same software module is to use. In the extreme, an infinitely reusable module is infinitely difficult to use. The driving force behind this was discussed in Chapter 4, Section 4.3.1. Developing a module that’s reusable demands that module be flexible, and with the increase in flexibility comes a corresponding increase in complexity. Likewise, increasing the ease with which a module can be used, managed, and deployed often decreases a module’s reusability. Dealing with the tension between reuse and use is a complex issue, and often, we fail. Largely, the problem has to do with dependencies.

Figure 1: The Use/Reuse Paradox

6.2 – The Reuse Disclaimer

I recognize that we’ve done a fair job in achieving reuse at certain levels, and we’re much farther along the reuse curve than we were 20 years ago. Today, we have a plethora of frameworks to choose from that aid development. Web frameworks, ORM frameworks, and security frameworks to name just a few. But most of these frameworks are horizontal, not vertical. That is, they address problems related to infrastructure and plumbing code, not business problems. And I want to focus explicitly on vertical reuse, because that’s the unattainable panacea we’ve been struggling with for so long. That’s the broken promise. Why have we struggled to create reusable business software?

6.2.1 – Granularity

Granularity is the extent to which a system is broken down into parts. Coarse-grained modules tend to be richer in behavior than fine-grained modules. Because coarse-grained modules do more, they tend to be larger than fine-grained modules. To maximize reuse, we try composing coarse-grained modules from fine-grained modules. Of course, this results in a lot of dependencies between the fine-grained modules, making the fine-grained modules more difficult to use. In general, we can say the following:

Coarse-grained modules are easier to use, but fine-grained modules are more reusable.

Time for an example. Let’s say we’re creating a module that processes health insurance claims. Let’s keep the business process relatively simple to maintain our sanity. There are four steps in the process. First, the system is fed the claim information. Second, the system checks to make sure it’s not a duplicate submission. Third, the system reprices the claim based on HMO and PPO agreements. Fourth, the system remits payment. A coarse-grained module would perform all four of these steps.

In doing this, we’ve made it easy to use since we only need to invoke one operation to complete the entire process. Additionally, if any part of the process changes, we only have a single module that needs to be built and redeployed. But it’s also more difficult to independently reuse only a portion of this process, such as the remit payment code. The logical solution is to create four fine-grained modules (one for each step in the process) and one coarse-grained module composed of the four others that pulls everything together. The fine-grained modules make things more reusable, but are also more difficult to use since we have to do more to pull them all together to perform a unit of work. Additionally, managing the deployment of four separate modules is more complex than deployment of a single module. In this situation, the coarse-grained module is an example of applying the Module Facade pattern.

6.2.2 – Weight

Weight is the extent to which a module depends on it’s environment. A heavyweight module depends on it’s operating environment, while a lightweight module avoids these dependencies. When creating a module that runs in multiple environments, we’re forced to move the environment specific dependencies (ie. context dependencies) from code to configuration. This makes the module more reusable, but it’s also a bit more difficult to use since the module must be configured for each environment. The Independent Deployment and Container Independent patterns discuss this in more detail.

Designing and configuring a lightweight module is more difficult than simply dropping in a module programmed to operate in that specific environment. In general, we can say the following:

Lightweight modules are more reusable, but heavyweight modules are easier to use.

Let’s elaborate using the example above, where the solution was to create one coarse-grained module composed of four fine-grained modules. If each of these modules only needs to run within a single application in a single operating environment, we can encapsulate all of this environmental code into each module, making each heavyweight. But if we want to reuse these modules across applications and operating environments, then we have to move this code outside of the module and ensure it can be configured for each environment in which we want it to operate.

6.3 – Reuse or Use

The challenge we run into when attempting to create a highly reusable module is to manage the tension between reusability and usability. In our example above, breaking out the coarse-grained modules into fine-grained modules makes it more difficult to use each of the resulting fine-grained modules. Likewise, creating a lightweight module makes using the module more difficult since the module must be configured each time the module is used.

In general, we can say that fine-grained modules have more module dependencies and lightweight modules require configuration based on context. Each makes a module more reusable, but also more difficult to use. The key is to strike a balance.

6.4 – Class and Module Design

Almost all principles and patterns that aid in software design and architecture address class design. Identifying the methods of a class, relationships between classes, and a system package structure are all examples of class design. Since most principles and patterns emphasize class design, it’s no surprise that the majority of developers spend their time dealing only with class design issues. Other examples of class design include deciding if a class should be a Singleton, determining if an operation should be abstract, or deciding if you should inherit from a class versus contain it. Developers live in the code, and are constantly dealing with class design issues.

Making good use of object oriented design principles and patterns is important. Accommodating the complex behaviors required by most business applications is a challenging task, and failing to create a flexible class structure can have a negative impact on future growth and extensibility. But class design is just one piece of the software design and architecture puzzle. The other piece of the puzzle is module design. Module design represents the deployable units composing a software system. Module design is equally important as logical design, and module design is all about dealing with the design paradigm of modularity.

6.5 – Benefits of Modularity

Figure 2: Benefits of Modularity

Almost all discussions on modularity mention reuse as a prime advantage. But there are other advantages, of course. There is a reduction in complexity that helps ease the maintenance effort, as we discussed in Chapter 5. Cohesive modules encapsulate behavior and expose it only through well-defined interfaces. Because modules are cohesive, change is isolated to the implementation details of a module as discussed in Chapter 4. Because behavior is exposed through interfaces, new modules containing alternative implementations can be developed without modifying existing modules. There are other benefits to modularity that extend beyond design to runtime, as well. These include the ability to hot deploy new modules without restarting an application and deploy multiple versions of a module. But from the design perspective, modularity helps increase reuse, ease maintenance, and increase extensibility.

6.6 -Modular Tension

Let’s explore the tension between use and reuse through an example. Assume we define an interface to decouple client classes from all classes implementing the interface. In doing this, it’s easy to create new implementations of the interface without impacting other areas of the system. The principle surrounding this idea is the Open Closed Principle – systems should be open for extension but closed to modification. For more information on OCP, see Appendix A and the discussion on the SOLID design principles. Logical design makes extending the system easier, but it’s also only half of the equation. The other half is how we choose to modularize the system.

Let’s assume the interface we’ve created has three different implementations, and that each of the implementation classes have underlying dependencies on other classes. We’re faced with a contentious issue. On one hand, grouping all the classes into a single module guarantees that change is isolated to only that module (ie. easier to use and maintain). If anything changes, we’ll only have one module to worry about. Yet, this decision results in a coarse-grained and heavyweight module (ie. harder to reuse), and a desire to reuse a subset of that module’s behavior leaves us with one of two choices. Duplicate code or refactor the module into multiple lighterweight and finer-grained modules. In general, logical design impacts extensibility, while physical design impacts reusability and useability.

As we refactor a coarse-grained and heavyweight module to something finer-grained and lighter weight, we’re faced with a set of tradeoffs. In addition to increased reusability, our understanding of the system architecture increases! We have the ability to visualize subsytems and identify the impact of change at a higher level of abstraction beyond just classes.  Grouping all classes into a single module may isolate change to only a single module, but understanding the impact of change is more difficult. With modules, we not only can assess the impact of change among classes, but modules, as well.

Unfortunately, if modules become too lightweight and fine-grained we’re faced with the dilemma of an explosion in module and context dependencies. Modules depend on other modules and require extensive configuration to deal with context dependencies! Overall, as the number of dependencies increase, modules become more complex and difficult to use, leading us to the corollary we presented above:

Maximizing reuse complicates use.

Creating lighter weight and finer-grained modules increases reuse but also increases module and context dependencies, while creating fatter modules decreases dependencies but also decreases reuse. Modules that are too lightweight provide minimal value and may require other modules to be useful. Modules that are too heavyweight are difficult to reuse because they do more than what the client needs. Of course, there are other challenges beyond reuse, such as the ability to understand and maintain the system.

Coarse-grained and heavyweight modules may do a good job of encapsulating change to a single module, but understanding the impact of change is more difficult. Conversely, fine-grained and lightweight modules make it easier to understand the impact of change, but even a small change can ripple across many modules. The key is to balance this tension, especially when developing large software systems where these challenges are even more pronounced.

6.7 – Modular Design

Large software systems are inherently more complex to develop and maintain than smaller systems. In addition to increasing reuse, breaking a large system into modules, makes the system easier to understand. By understanding the behaviors contained within a module, and the dependencies that exist between modules, it’s easier to identify and assess the ramification of change. We saw an example of this Chapter 5, Section 5.7.1.

For instance, software modules with few incoming dependencies are easier to change than software modules with many incoming dependencies.  Likewise, software modules with few outgoing dependencies are much easier to reuse than software modules with many outgoing dependencies. This tension surrounding module weight and granularity is an important factor to consider when designing software modules.

Today, frameworks like OSGi aid in designing modular software systems. While these frameworks can enforce runtime modularity, they cannot guarantee that we’ve modularized the system correctly. Correct modularization of any software system is contextual and temporal. It depends on the project, and the natural shifts that occur throughout the project impact moduarlity. As shown in Figure 3, the modularity patterns help address the design paradigm surrounding modularity, and provide balance in dealing with the tension between use and reuse.

Figure 3: Modularity Patterns

6.8 – Conclusion

Modularity patterns provide guidance and wisdom in helping design modular software. They explain ways that we can minimize dependencies while maximizing reuse potential. They help balance module weight and granularity to make a system easier to understand, maintain, and extend. For those who have attempted to design modular software, it’s common for modularity to drive the class design decisions.

The chapters that follow present a list of modularity patterns that I’ve used on past projects. These patterns also build atop proven object-oriented design concepts. Some of the patterns, such as Manage Relationships, may be intuitively obvious, while others less so. Certainly, other patterns exist that are not listed, and I hope as you discover your own patterns, you’ll add share them with others. I’m also hopeful that technology specific idioms emerge that show how these patterns can be applied using specific module frameworks and technologies, such as OSGi. Before we delve to deeply into the pattern discussions, I’d like to walk through a sample system to illustrate the wonders of modularity.

A well-defined segmentation of the project effort ensures system modularity. Each task forms a separate, distinct program module. At implementation time each module and its inputs and outputs are well-defined, there is no confusion in the intended interface with other system modules. At checkout time the integrity of the module is tested independently; there are few scheduling problems in synchronizing the completion of several tasks before checkout can begin. Finally, the system is maintained in modular fashion; system errors and deficiencies can be traced to specific system modules, thus limiting the scope of detailed error searching.

Parnas would go on to discuss the important decisions developers must make when modularizing their software systems. Encapsulating design decisions within autonomous modules allowed for modules to evolve independently and reveal little about their inner workings. The designs were encapsulated, shielding other developers from this unnecessary complexity, and helping them to more easily understand the software system.

While a lot has changed since the Parnas paper was first published, one constant remains – software is still incredibly difficult to design and develop. Unfortunately, in the decades that have passed since modular programming was first introduced, these ideas were lost. Today, few of us consciously decompose our software systems into modules. Fortunately, we have the luxury of using a platform with an excellent unit of modularity built in, and it’s time we start using it.

5.1 – Enterprise Complexity

Most software systems don’t begin their life as complex entities, but grow complex over time. Lehman’s 2nd law summarizes this phenomenon:

As a system evolves, it’s complexity increases unless work is done to maintain or reduce it.

Lehman chose to use the evolution of IBM’s OS/360 for his work by examining the growth of the system over a series of releases. His findings revealed that for each release, the size of the software system increased. The logical conclusion? As software system size increases, so too does the complexity.

Many of us have experienced this phenomenon. In almost every way, larger software systems are inherently more difficult to design, build, and manage than are smaller software systems. Examining more modern evidence of Lehman’s work, we turn to the evolution of the Spring framework. The diagram in Figure 1 illustrates this evolution between release 0.9.1 and release 2.5. Since the earliest releases of the framework dating back to 2003, we’ve seen almost a six-fold increase in the size of the code. Examining other projects, such as Linux, Tomcat, or FireFox reveals similar growth.

Figure 1: Evolutionary Growth of Spring Framework in Lines of Code

Such phenomenal growth is not only expected, but desired. We can only hope that the systems we create are in such high demand that we have the opportunity to grow them over time. A software system that doesn’t change, dies. In fact, studies (http://users.jyu.fi/~koskinen/smcosts.htm) suggest that the cost to maintain software and manage its evolution exceeds 90% of the total cost of a system. Evolution is a big deal!

So evolution is not only expected, but desirable. It’s in our best interests to encourage change in software systems. But as we’ve seen, change breeds complexity. What is the cause of all this complexity, and what can we do to manage this complexity?

5.2 – Technical Debt

Software tends to rot over time. When you establish your initial vision for the software’s design and architecture, you imagine a system that is easy to modify, extend, and maintain. Unfortunately, as time passes, changes trickle in that exercise your design in unexpected ways. Each change begins to resemble nothing more than another hack, until finally the system becomes a tangled web of code that few developers care to venture through. Sadly, design rot is self-inflicted, and technical debt describes the affect of rotting design.

Technical debt is a metaphor developed by Ward Cunningham to describe the design trade-off’s made in order to meet schedules and customer expectations. Martin Fowler offers the following analogy when comparing technical debt to financial debt.

Like a financial debt, the technical debt incurs interest payments, which come in the form of the extra effort that we have to do in future development because of the quick and dirty design choice. We can choose to continue paying the interest, or we can pay down the principal by refactoring the quick and dirty design into the better design. Although it costs to pay down the principal, we gain by reduced interest payments in the future.

There are situations where leveraging suboptimal designs, thereby incurring technical debt, is warranted to meet short term demands. For instance, the schedule may not not allow longer term designs to be used. However, if we ignore this debt, it continues to build over time and incurring too much debt leads to significant inefficiencies surrounding our ability to effectively change the software system. Lehman’s law in action, for sure. In fact, at some point we hit a wall where we are no longer able to afford the interest payments. It’s imperative that we manage technical debt to ensure long term survivability of our software system. Too much debt will cause the system to crumble. At some point, we must deal with the technical debt we’ve incurred. If we don’t, our design will continue to rot!

5.3 – Design Rot

The most common cause of rotting software is tightly coupled code with excessive dependencies. Of course, any interesting software system must have dependencies for it to do anything. Yet for large teams and large applications, managing dependencies is especially important. Excessive dependencies increase technical debt and cause numerous problems. And as Lehman’s law suggests, the problems will persist and likely increase unless work is done to maintain it.

5.3.1 – Hinder Maintenance

Dependencies hinder the maintenance effort. When you’re working on a system with heavy dependencies, you typically find that changes in one area of the application trickle to many other areas of the application. In some cases, this cannot be avoided. For instance, when you add a column to a database table that must be displayed on the user interface, you’ll be forced to modify at least the data access and user interface layers. Such a scenario is mostly inevitable. However, applications with a well thought dependency structure make change easier since developers have a more complete understanding of the impact of change.

5.3.2 – Prevent Extensibility

Dependencies prevent extensibility. The goal of flexible software architecture is to remain open for extension but closed to modification. The desire is to add new functionality to the system by extending existing abstractions, and plugging these extensions into the existing system without making rampant modifications. One reason for heavy dependencies is the improper use of abstraction, and those cases where abstractions are not present are areas that are difficult to extend. Abstraction aids large teams by exposing well-defined extension points, as well as encapsulating implementation complexity.

5.3.3 – Inhibit Reusability

Dependencies inhibit reusability. Reuse is often touted as a fundamental advantage of well-designed object oriented software. Unfortunately, few applications realize this benefit. Too often, we emphasize class level reuse. To achieve higher levels of reuse, careful consideration must also be given to the package structure and deployable unit structure. Software with complex package and physical dependencies minimize the likelihood of achieving higher degrees of reuse.

5.3.4 – Restrict Testability

Dependencies restrict testability. Tight coupling between classes eliminates the ability to test classes independently. Unit testing is a fundamental principle that should be employed by all developers. Tests provide you the courage to improve your designs, knowing flaws will be caught by unit tests. They also help you design proactively and discourage undesirable dependencies. Heavy dependencies do not allow you to test software modules independently. Teams with few tests cannot respond to change easily due to the inability to access the impact of change.

5.3.5 – Hamper Integration

Dependencies hamper integration. It’s easy for separate teams to plow forward, usually under tremendous pressure from looming deadlines. They operate under the false assumption that if they can simply reach the final feature destination, they can quickly pull things together toward the end of a project. This Big Bang approach to integration does not work. As individual modules are pulled together, common issues that surface include degradation of overall system performance, incorrect levels of behavioral granularity provided by system modules, and transactional incompatibilities. More frequent integration brings many of these issues to the forefront earlier in the project.

5.3.6 – Limit Understanding

Dependencies limit understanding. When working on a software system, it’s important that you understand the system’s structural architecture and design constructs. A structure with complex dependencies is inherently more difficult to understand. Clear and concise dependencies that are well-thought allow teams to more easily access the impact of change.

5.4 – Cyclic Dependencies – The Death Knell

Excessive dependencies are bad. But cyclic dependencies are especially bad. Cyclic dependencies are manifest in various ways at different levels within a system. It’s also possible that acyclic relationships at one level cause cycles at another. For more information on cyclic and acyclic dependencies, be sure to check out the Acyclic Relationships pattern.

5.4.1 – Types of Cycles

Cycles exist across a variety of entities; notably class, package and modules. Class cycles exist when two classes, such as Customer and Bill shown in Figure 2, each reference the other. In this example, we can assume a Customer has a list of Bill instances, and Bill references the Customer to calculate a discount amount. This is also known as a bi-directional association. It’s a maintenance and testing issue, since you can’t do anything to either class without affecting the other.

Figure 2: Cyclic Dependency Between Classes

The code in Listing 1 shows the customer class while the code in Listing 2 shows the Bill class. Certain portions of each class have been omitted to improve clarity. The bi-directional relationship can be clearly identified.

package com.kirkk.cust;

import java.util.*;
import java.math.BigDecimal;
import com.kirkk.bill.*;

public class Customer {
   private List bills;

   //Derivation of the discount depends on the number of bills for a specific Customer.
   public BigDecimal getDiscountAmount() {
      if (bills.size() > 5) {
         return new BigDecimal(0.1);
      } else {
         return new BigDecimal(0.03);
      }
   }

   public void createBill() {
      Bill bill = new Bill(this);
      if (bills == null) {
         bills = new ArrayList();
      }
      bills.add(bill);
    }
}

Listing 1: The Customer Class

package com.kirkk.bill;

import com.kirkk.cust.*;
import java.math.BigDecimal;

public class Bill {
   private Customer customer;

   public Bill(Customer customer) {
      this.customer = customer;
   }

   public BigDecimal pay() {
      BigDecimal discount = new BigDecimal(1).subtract(this.customer.getDiscountAmount()).setScale(2, BigDecimal.ROUND_HALF_UP);
      //Apply the discount and make the payment...
      return paidAmount;
   }
}

Listing 2: The Bill Class

Class cycles can be broken a few different ways, one of which is to introduce an abstraction that breaks the cycle, as shown in Figure 3. Now, the Bill class can easily be tested with a mock DiscountCalculator. Testing Customer, of course, still requires the presence of Bill. This is not a cyclic issue, it’s a different type of coupling issue because Bill is a concrete class, and is fodder for a separate discussion, which we’ll tackle when discussing the Manage Relationships pattern. Clearly, introducing DiscountCalculator has broken the cycle between Customer and Bill…but has it broken all cycles, including those that might exist between modules?

Figure 3: Breaking the Cycle

Listing 3 shows the modified Bill class that implements the DiscountCalculator interface, which is shown in Listing 4.

<pre>package com.kirkk.bill;

import java.math.BigDecimal;

public class Bill implements DiscountCalculator {
   private DiscountCalculator calculator;

   public Bill(DiscountCalculator calculator) {
      this.calculator = calculator;
   }

   public BigDecimal pay() {
      BigDecimal discount = new BigDecimal(1).subtract(this.calculator.getDiscountAmount()).setScale(2, BigDecimal.ROUND_HALF_UP);
      //Apply the discount and make the payment...
      return paidAmount;
   }
}

Listing 3: The Modified Bill Class

package com.kirkk.bill;

import java.math.BigDecimal;

public interface DiscountCalculator {
   public BigDecimal getDiscountAmount();
}

Listing 4: The DiscountCalculator Interface

5.4.2 – Creeping Cycles

We don’t intentionally create cyclic dependencies between modules. Instead, they tend to creep into our design. They commonly surface when cyclic or acyclic relationships at one level cause cycles at another. For instance, if Customer and DiscountCalculator are placed in a customer.jar module, and Bill is placed in a billing.jar module, a cyclic dependency between the customer.jar and billing.jar modules exists even though the bi-directional relationships between classes has been broken. The cyclic dependencies between modules is shown in Figure 4. Allocating the Customer and Billing classes to the customer.jar and billing.jar modules is the cause of the cycle.

Figure 4: Cycles Between Modules

To break the cycle, we must move DiscountCalculator to its own module, or the billing module. Figure 5 illustrates that moving the DiscountCalculator to the billing.jar module has broken the cyclic dependency between the modules. Obviously this is a trivial example. For larger software systems with potentially thousands of classes and numerous modules to manage, the challenge is much greater. This is where the patterns in this book and first class support for modularity on the Java platform will serve as a valuable aid.

Figure 5: Acyclic Between Modules

5.4.3 – Managing Cycles

Fortunately, there are many ways (some easier than others) to manage dependencies and eliminate cycles. Test Driven Development is a great way to manage class cycles assuming we strive to test classes in isolation. JDepend allows you to manage package cycles, either by writing package constraint tests or including JDepend reports within your Ant build script. Cycles between modules can be managed using a Levelized Build, where individual modules are built, including only necessary modules in the build class path. Additional patterns are useful in managing and massaging module dependencies.

JarAnalyzer can also be included in your build script, generating a component diagram illustrating the relationship between JAR files, or a dependency report similar to that of JDepend. Maven and Ivy also provide ways to help manage dependencies. And of course, looming on the horizon is OSGi, which will offer runtime support for modularity and will enforce module dependencies.

5.4.4 – But Are Cycles Always Bad?

Generally speaking, cycles are always bad! But some cycles are worse than others. Cycles among classes are tolerable, assuming they don’t cause cycles among the packages or modules containing them (ie. the classes must be in the same module, essentially encapsulating the design). For a discussion on encapsulating design, see Chapter 4, Section 4.4. Cycles among packages may also be tolerable, assuming they don’t cause cycles among the modules files containing them (again, packages are in the same module). Most important is that we are aware of the relationships existing between the modules. In so many cases, we aren’t.

5.5 – In The Real World

On a project, we were integrating with a vendor product. The vendor product would help us fulfill the needs to the business processing by applying industry standard coding that helped us comply with federal mandates. The relationship with the vendor was volatile, and negotiation with an alternative vendor was underway.

Unfortunately, our project deadline didn’t align well with the vendor contract. We knew we’d have a production release about three months before the vendor decision would be made. In light of this, we encapsulated all vendor code into a single module, and separated out the interface classes to a separate module by applying the Separate Abstractions pattern. When the decision was finally made to establish a relationship with the new vendor, making the switch was less painful. We created a new module that integrated with the new vendor and implemented the interfaces. The ripple effect throughout the system was minimal, and we were able to easily test the integration independently. We’ll see this example play out in front of us when we walk through the reference implementation in Chapter 7.

5.6 – Joints, Modules, & SOLID

It’s imperative to manage dependencies between our modules. The SOLID principles of object-oriented design help minimize class coupling and increase class cohesion, making software easier to maintain and extend. Yet there is also a cost to applying the principles. The additional layers of abstraction increase the complexity of the system. Blindly applying the principles to all classes within a system can actually decrease our ability to maintain and extend the software. Remember the paradox discussed in Chapter 4, SEction 4.3.1 – with flexibility comes complexity. This begs the question, “Where is the most appropriate place to apply the SOLID principles?”

We touched briefly on the joints of the system in Chapter 4, Section 4.4. Every system has joints, which is the point where two modules connect. It is these joints within the system that require the greatest flexibility and resiliency. The reason for this is driven by change. Change that is encapsulated within a single module poses less threat and risk than change that ripples across many modules. This is trivial, and it’s why changing the private methods of a class are easier than changing the public methods. Likewise, change confined to a single module is easier than change that spans modules. That’s logical. So if we’re able to create well-designed modules, it makes our life a little bit easier. And using the SOLID design principles, and other proven design patterns, allows us to bring greater modularity to our applications if we apply the principles when designing our modules.

It is these joints, or module boundaries, within a system that represent the ideal opportunity to leverage the SOLID principles. Using the SOLID principles to minimize coupling between modules offers the greatest opportunity and reward. For example, it’s not realistic to design an entire system open for extension but closed to modification (the Open Closed Principle). But we can use OCP at the module boundaries (aka. the joints) to create modules with enough design flexibility to confine change to a single module. Using the SOLID principles when designing the relationships between modules provides a low barrier point of entry for using these principles to create better designs. For instance, it’s much easier to refactor the classes if we know the refactoring is confined to a single module. With a rich suite of unit tests, we can prove it has no impact to the outside world. But if we have to refactor the classes and the refactoring spans the joints of the system, well…that’s an architectural refactoring. It gets difficult and messy.

This is a major reason why frameworks such as OSGi, and the patterns in this book are so important – they make it just a little bit easier to design really big software systems that we can understand, maintain, and extend. And if we apply the SOLID principles in conjunction with the module patterns, we have guidance on how to best design large software systems. As interest in designing more modular software continues to increase, and especially as OSGi continues to gain momentum, the module design patterns will grow in importance in helping us manage complexity, and new patterns will also likely emerge.

5.7 – Managing Complexity

Many of the benefits of modularity can be gleaned from the definition of a software module. Deployable! Manageable! Reusable! Composable! Indeed, they are substantial benefits. But the greatest benefit of modularity is that it helps us tame the complexity of a software system as it evolves. By taming complexity, it’s much easier to understand the impact of change within large software systems. Since 90% of the total cost of a system involves making change to an existing system, this is a substantial benefit. Modularity is the work we do to help manage the complexity of a system as it evolves. Time for an example that illustrates this benefit.

5.7.1 – Illustrating the Benefit

The diagram in Figure 6 illustrates the benefit of modularity. The top left quadrant shows a sample system with a relatively complex class structure. When change occurs within a single class, shown in red in the bottom left quadrant, understanding the impact of change is difficult. It appears possible that it can propagate to any class dependent on the class highlighted in red.  Assessing the impact of change requires that we analyze the complete class structure. The ripple effect appears significant, and change instills fear.

But if the system is modular with classes allocated to these modules, as shown in the bottom right quadrant, then understanding the impact of change can be isolated to a discrete set of modules. And this makes it much easier to identify which modules contain classes that might also change, as shown in the top right quadrant. Change is isolated to classes within modules that are dependent on the module containing the class that is changing.

This is a simple example, but it serves as evidence of the need for modular architecture, and illustrates one reason why modularity is so important. Modularity makes understanding the system easier. It makes maintaining the system easier. And it makes reusing system modules much more likely. As systems grow in size and complexity, it’s imperative that we design more modular software. That means we need a module system for the Java platform. It means that module system shouldn’t be shielded from enterprise developers. And it means we need to understand the patterns that are going to provide the guidance necessary in helping us design more modular software.

Figure 6: Modules Make Understanding Change Easier

Of course, the object-oriented paradigm promised similar benefits, but experience has shown that objects are too fine-grained. Even a system of moderate size might be composed of thousands of classes, making it virtually impossible for any single person to fully understand the entire system. Does this undermine the benefits of the object-oriented paradigm? Certainly not. But more is needed to help manage the complexity of large enterprise software systems.

Modules are more coarse-grained units than objects that allow us to encapsulate complex design decisions. A module might contain many classes, but the design decisions surrounding the relationships between classes can be encapsulated within a module. Cohesive modules encapsulate behavior and expose it only through well-defined interfaces. Because modules are cohesive, change is isolated to the implementation details of a module. Module behavior that is exposed through interfaces ensures new modules containing alternative implementations can be developed without modifying existing modules. The result is a reduction in complexity that helps ease the maintenance effort. In general, a well-designed modular system is simply easier to understand.

Of course, modularity doesn’t guarantee these benefits any more than using object orientation guarantees the benefits of the object-oriented paradigm. It still requires that we design our modules correctly. We’ll explore these ideas further in the next chapter.

5.8 – Conclusion

Software complexity is our worst enemy. Complexity inhibits our ability to gracefully adapt software systems in response to changing demands. And change is good because it means our software systems are gaining acceptance. Modularity helps tame this complexity, but only if the software modules we create are of high quality. It’s as simple to create a mess using modules as it is with objects and services. This chapter helped illustrate the benefit of modularity. Subsequent chapters discuss how we can realize this benefit.

4.1 – Defining Architecture

There are numerous definitions of architecture. But within each lies a common theme, and some key phrases. Here are a few of the definitions. From Booch, Rumbaugh, and Jacobson in UML User Guide (Addison-Wesley, 1999):

An architecture is the set of significant decisions about the organization of a software system, the selection of the structural elements and their interfaces by which the system is composed, together with their behavior as specified in the collaborations among those elements, the composition of these structural elements and behavioral elements into progressively larger subsystems, and the architecture style that guides this organization — these elements and their interfaces, their collaborations, and their composition.

Now, the ANSI/IEEE Std 1471-2000:

The fundamental organization of a system, embodied in its components, their relationships to each other and the environment, and the principles governing its design and evolution.

In TOGAF, architecture has two meanings depending on context:

1.)   A formal description of a system, or a detailed plan of the system at component level to guide its implementation

2.)   The structure of components, their inter-relationships, and the principles and guidelines governing their design and evolution over time.

Examining these definitions reveals many common keywords that span each, which I’ve highlighted in bold in the various definitions. There exist important underlying currents embodied by these keywords. But these keywords also lead to some important questions that must be answered to more fully understand architecture. What makes a decision architecturally significant? What are the elements of composition? How do we accommodate evolution of architecture? And what does this have to do with modularity? As we begin to delve into answering these questions, I’d like to start with a story on software architecture.

4.2 – A Software Architecture Story

The story of software architecture reminds me of the following story.

“A well-known scientist (some say it was Bertrand Russell) once gave a public lecture on astronomy. He described how the earth orbits around the sun and how the sun, in turn, orbits around the center of a vast collection of stars called our galaxy. At the end of the lecture, a little old lady at the back of the room got up and said: “What you have told us is rubbish. The world is really a flat plate supported on the back of a giant tortoise.” The scientist gave a superior smile before replying, “What is the tortoise standing on?” “You’re very clever, young man, very clever”, said the old lady. “But it’s turtles all the way down!”

- A Brief History of Time by Stephen Hawking

Software architecture is “‘turtles all the way down.”

4.2.1 – The Ivory Tower

Many of us can relate. In dysfunctional organizations, architects and developers fail to communicate effectively. The result is a lack of transparency and a lack of understanding by both sides. As shown in Figure 1, architects tend to bestow their wisdom upon developers who fail to grasp the high level concepts in terms that make sense to them. The failure often occurs (though I recognize there are other causes) because architecture is about breadth and development is about depth.  Each group has disparate views of software architecture, and while both are warranted, a gap between these views exists. The architect might focus on applications and services while the developer focuses on the code. Sadly, there is a lot in between that nobody is focused on. It is this gap between breadth and depth that contributes to ivory tower architecture.

Figure 1: The Ivory Tower

4.2.2 – Turtles and the Tower

Without question, the ivory tower is dysfunctional, and systems lacking architectural integrity are a symptom of ivory tower architecture. So assuming good intent on the part of the architect and the developer, how can we bridge the gap between breadth and depth? How can we communicate more effectively? How do we increase understanding and transparency?

Let’s revisit the definition of software architecture by exploring another. My favorite definition of software architecture was offered by Ralph Johnson in an article by Martin Fowler (Who Needs an Architect? citation needed). He states:

In most successful software projects, the expert developers working on that project have a shared understanding of the system design. This shared understanding is called ‘architecture.’ This understanding includes how the system is divided into components and how the components interact through interfaces. These components are usually composed of smaller components, but the architecture only includes the components and interfaces that are understood by all the developers…Architecture is about the important stuff. Whatever that is.

The key aspect of this definition that differentiates it from the earlier definitions in this chapter is that of “shared understanding”, which implies that there is a social aspect to software architecture. We must have a shared understanding of how the system is divided into components, and how they interact. Architecture isn’t just some technical concept, but also a social construct. And it is through this social aspect of architecture that we can break down the divide between architects and developers.

To ensure shared understanding, we have to architect “all the way down.” Architects cannot worry only about services and developers cannot worry only about code. There is a huge middle ground that each must also focus on, as illustrated by the diagram in Figure 2.

Figure 2: Architecture all the way Down

Focusing exclusively on top level abstractions is not enough. Emphasizing only code quality is not enough either. We must bridge the gap through other means, including module and package design. Often times, when I speak, I ask the audience to raise their hands if they spend time on service design. Most do. I also ask them to raise their hand if they spend time on class design and code quality. Again, most do. But then I ask if they also spend time on package and module design. Usually, only a small percentage leave their hands raised.

This is unfortunate, because module and package design are equally as important as service and class design. But somewhere along the way, with our emphasis on services and code quality, we’ve lost sight of what lies in between. Within each application or service awaits a rotting design, and atop even the most flexible code sits a suite of applications or services riddled with duplication and lack of understanding. A resilient package structure and corresponding software modules help bridge the divide between services and code. Modularity is an important intermediate technology that helps us architect all the way down, and is the conduit that fills the gap between breadth and depth.

We need to start focusing on modularity to ensure a consistent architecture story is told. It is the glue that binds. It’s the piece that helps bridge low level class design with higher level service design. It’s the piece that helps bring down the ivory tower, enhance communication, increase transparency, ensure understanding, and verify consistency at multiple levels. It is the piece that allows us to “architect all the way down”, and allows to realize the goal of architecture.

4.3 – The Goal of Architecture

Modularity helps address the social aspect of software architecture, but it also helps us design more flexible software systems. That is, a system with a resilient, adaptable, and maintainable architecture. Examining the earlier definitions of architecture leads us to the goal of architecture. The Johnson definition of architecture as quoted by Fowler makes it’s apparent that architecture is about the important stuff. And in the following statement, Booch makes (citation needed) it clear that something is architecturally significant if it’s difficult to change.

All architecture is design but not all design is architecture. Architecture represents the significant design decisions that shape a system, where significant is measured by cost of change.

Based on these statements, it’s fair to conclude that the goal of software architecture must be to eliminate the impact and cost of change, thereby eliminating architectural significance. We attempt to make something architecturally insignificant by creating flexible solutions that can be changed easily, as illustrated in Figure 3. But herein lies a paradox.

Figure 3 – The Goal of Architecture

4.3.1 – The Paradox

The idea behind eliminating architecture isn’t new. In fact, Fowler mentions “getting rid of software architecture” in his article, Who Needs an Architect? (citation needed). And the way to eliminate architecture by minimizing the impact of cost and change is through flexibility. The more flexible the system, the more likely that the system can adapt and evolve as necessary. But herein lies the paradox, and I’ll use a statement by Ralph Johnson to present and support the idea.

…making everything easy to change makes the entire system very complex…

So as flexibility increases, so too does the complexity. And complexity is the beast we are trying to tame because complex things are more difficult to deal with than simple things. It’s a battle for which there is no clear path to victory, for sure. But what if we were able to tame complexity while increasing flexibility, as illustrated in Figure 4? Taming the best we call complexity is the emphasis of Chapter 5. But first, let’s explore the possibility of designing flexible software without increasing complexity. Is it even possible? In other words, how do we eliminate architecture?

Figure 4 – Maximize Flexibility, Manage Complexity

4.3.2 – Eliminating Architecture

As the Johnson quote clearly points out, it’s not feasible to design an infinitely flexible system. Therefore, it’s imperative that we recognize where flexibility is necessary to reduce the impact and cost of change. The challenge is that we don’t always know early in the project what might eventually change, so it’s impossible to create a flexible solution to something we can’t know about. This is the problem with Big Architecture Up Front (BAUF), and it’s why we must make architectural decisions temporally. In other words, we should try to defer commitment to specific architectural decisions that would lock us to a specific solution until we have the requisite knowledge that will allow us to make the most informed decision.

It’s also why we must take great care in insulating and isolating decisions we’re unsure of, and ensuring that these initial decisions are easy to change as answers to the unknown emerge. For this, modularity is a missing ingredient that helps minimize the impact and cost of change, and it’s a motivating force behind why we should design software systems with a modular architecture. In the UML User Guide (P. 163), Booch talks about “modeling the seams in a system.” He states:

Identifying the seams in a system involves identifying clear lines of demarcation in your architecture. On either side of those lines, you’ll find components that may change independently, without affecting the components on the other side, as long as the components on both sides conform to the contract specified by that interface.

Where Booch talks about components, today we talk about modules. Where Booch talks about seams, we’ll talk about joints. Modularity combined with design patterns and SOLID principles, represent our best hope to minimize the impact and cost of change, thereby eliminating the architectural significance of change. The joints of a system are discussed in Chapter 5, Section 5.6. The SOLID principles are discussed in Appendix A.

4.4 – Modularity – The Missing Ingredient

Two of the key elements of the architectural definitions are component and composition. Yet there is no standard and agreed upon definition of component (reminding me of architecture, actually), and most use the term pretty loosely to mean just “a chunk of code”. But that doesn’t work, and in the context of OSGi, it’s clear that a module is a software component. We devoted Chapter 2 to defining module. Developing a system with an adaptive, flexible, and maintainable architecture requires modularity because we must be able to design a flexible system that allows us to make temporal decisions based on shifts that occur throughout development. Modularity has been a missing piece that allows us to more easily accommodate these shifts, as well as focus on specific areas of the system that demand the most flexibility, as illustrated in Figure 5.

Figure 5 – Encapsulating Design

Modularity, in conjunction with design patterns and SOLID principles, represent our best hope to minimize the impact and cost of change. It’s easier to change a design embedded within a module than it is a design that spans modules.

4.5 – Answering Our Questions

At the beginning of this chapter, we posed the following questions after introducing the three definitions of software architecture. Through explanation, we answered each of these questions. But to be clear, let’s offer concise answers to each of these questions.

• What makes a decision architecturally significant? A decision is architecturally significant if the impact and cost of change is significant.
• What are the elements of composition? The elements of composition include, classes, modules, and services.
• How do we accommodate evolution of architecture? Evolution is realized by designing flexible solutions that can adapt to change. But flexibility breeds complexity, and we must be careful to build flexibility in the right areas of the system.

And what does this have to do with modularity? Modularity is an important intermediate component that helps increase architectural agility. It fills a gap that exists between architects and developers. It allows us to create a software architecture that can accommodate shifts. Modularity helps us architect all the way down.

3.1 – The Runtime Model

The runtime model focuses on how we manage modular software systems at runtime. It includes support for the dynamic deployment of bundles, different versions of bundles, dependency resolution, enforcement of module boundaries, and the overall dynamism of the runtime environment. The runtime model is relatively mature. The defacto standard module system on the Java platform is OSGi, and many application platforms are leveraging OSGi to take advantage of the runtime capabilities resulting from increased modularity. To an extent, this allows the enterprise to realize the advantages of OSGi without knowing much about it. Faster application startup times and platform adaptability are two advantages organizations will realize as vendors bake OSGi into their products.

Until recently, however, many of the most widely used platforms encapsulated OSGi, and have chosen to keep it hidden from enterprise developers. Because of this, developers were unable to build applications that took advantage of the modular runtime.  However, this is changing as the platforms are exposing the virtues of OSGi and allowing developers to tap into its powerful runtime capabilities. No longer will we be subject to classpath hell and the monolithic applications that plague us today. Instead, modules will be able to discover other modules at runtime, and the artificial walls that separate applications will no longer exist.

Eventually, as support for modularity is baked into the platform, enterprise developers will be able to leverage the frameworks and technologies that aid them in developing more modular software systems. When this happens, the development model will become very relevant.

3.2 – The Development Model

The development model deals with how developers use the runtime framework to build their software applications. The development model can be further broken down into two categories – the programming model and the design paradigm. Each are important in helping developers build more modular software applications.

3.2.1 – The Programming Model

Taking advantage of the runtime module system demands that programmers have the ability to interact with the module system’s API. However, dependencies on the API result in heavyweight modules that are difficult to test and execute outside of the runtime system. To reduce the dependency on the module system API, frameworks and technologies must provide levels of abstraction so that our code doesn’t depend directly upon the framework’s API. Examples of these frameworks and technologies include the OSGi Blueprint Services, Spring Dynamic Modules, and iPojo.

Developers can leverage these frameworks and tools to tap into the capabilities of the runtime module system without worrying about the programming model. These frameworks encapsulate dependencies on the API so your code doesn’t have to talk directly to the API. The separation of concerns achieved through these frameworks ensures Java classes remain POJOs that aren’t dependent on the module system’s framework. This makes programming and testing much easier.

3.2.2 – The Design Paradigm

But the design paradigm must also be addressed. How does an organization create a more modular architecture? What is the right granularity for a module? How heavily dependent should modules be on each other? How do we minimize module dependencies? How do we break apart larger modules into a smaller set of more cohesive modules? When do we do this? These, among others, are the important architectural and design questions that surround modularity in the enterprise. This is the focus of the patterns in this book.

There are important lessons that can be learned from technology’s history. To examine this more deeply, let’s look at two different examples of technologies that prove the pending relevance of the module design paradigm – object-oriented (OO) programming and EJB.

3.2.2.1 – Object-Orientation

In the early 1990’s, OO was touted as the savior. Development teams would be able to build systems by composing reusable objects. This promised significantly reduced time-to-market and higher quality software. The promises were not met, and the benefits of object oriented technologies were never fully realized. There are a few reasons for this. Objects are too granular to serve as the foundation of reuse. Developments teams also had difficulty grasping and applying object oriented concepts correctly. Deep inheritance hierarchies laden with base classes rich in functionality contributed to poorly designed systems. In general, object oriented development was an early failure.

The runtime capabilities of object oriented programming languages provided features such as polymorphism and dynamic binding, and developers were able to easily understand many aspects of the programming model. Using dot notation to invoke methods and defining private member variables were trivial concepts. But it took a long time for us to understand how to design good programs using object oriented design techniques. What we accept today as simple truths surrounding object oriented design (”favor object composition over object inheritance” and “program to an interface, not an implementation”) were unknown, or at least a mystery to us, 15 years ago. Even now, we continue to learn new ways to design better software systems using object technology.

3.2.2.2 – Enterprise JavaBeans

Enterprise Java Beans (EJB), and especially entity beans, were presented as part of Java as a way to componentize business applications. The runtime capabilities of EJB were very alluring – transactions, persistence, security, transparency, etc. – and baked directly into the platform. Unfortunately, there were two glaring problems. The development model was complex and it was not well understood.

It was a number of years ago, but I recall vividly my first experience with EJB. I arrived on the team mid-way through their development effort. At that point, the team had more than 100 entity beans, the local development environment took more than four hours to start-up, and problems were rampant. I stuck around for three weeks before I voluntarily left the project. The project wound up getting canceled shortly thereafter. The problem with EJB was not the runtime model nor the programming model. The runtime model fulfilled many of its promises and the code generation wizards accompanying many tools made working with the programming model relatively straightforward. Frameworks even emerged that allowed developers to decouple their code from the EJB programming model and focus on designing simpler POJOs. However, EJB was a new technology and many developers lacked the design wisdom to leverage the technology effectively.

3.2.2.3 – Lessons Learned

Object oriented programming and EJB were two promising technologies that, arguably, failed to live up to their initial hype. The problems were not that object oriented programming languages, nor with the platform implementations of the EJB specification, but with how we chose to design our applications using these technologies. The most significant challenges were centered around the design paradigm.

These lessons serve as examples of the difficult road that lies ahead for OSGi, and specifically, modularity on the Java platform. If the design paradigm isn’t understood, with principles and patterns that guide how developers leverage the technology, the benefits of the runtime model will not be realized. I firmly believe that modularity is a key element of designing software systems with a flexible and adaptable architecture. There is a need for modularity on the Java platform, especially in developing large enterprise software systems. But if we do not begin to understand how to design more modular applications today, we’ll face significant challenges when platform support for modularity arrives.

3.3 – Modularity Today

It might appear that the runtime model and development model are inextricably linked. Untrue! While a runtime module systems offers additional facilities, such as enforcing module relationships, that help design more modular applications, it is not a requirement. We can still design modular software even in the absence of a runtime module system. In fact, given the significant advantages of modularity discussed in Chatpers 4, 5, and 6, it’s a good idea for development teams to start modularizing their applications right now, even if they aren’t deploying to a runtime module system. How can we do this?

We know a lot about the OSGi runtime model. It’s likely the platform we’re using is leveraging the runtime model internally, even if it’s not exposed to us right now. We know the unit of modularity is the JAR file, and we can start modularizing our applications today by emphasizing the JAR file as the unit of modularity. In Chapter 7, we’ll see firsthand how we can take a system and modularize it by emphasizing the JAR file as the unit of modularity, even in the absence of a runtime module system. The sample code for many of the patterns intentionally avoid using OSGi, though discussion of how the pattern can be implemented when using OSGi is included where it helps offer further clarification. In Chapter 5, Section 5.7, I illustrate visually the advantages of modularity.

Even when OSGi hits your platform, it’s pertinent to remember that the intent of modularizing our applications is not so that we are able to take advantage of OSGi. That’s simply a positive side affect. The real value is the modular architecture that results.

But the role design patterns have played over the last decade should not be diminished. They were a catalyst that propelled object oriented development into the mainstream. They helped legions of developers understand the real value of inheritance and how to use it effectively. Patterns provided insight to how to construct flexible and resilient software systems. With nuggets of wisdom, such as “Favor object composition over class inheritance” and “Program to an interface, not an implementation”, patterns helped a generation of software developers adopt a new programming paradigm.

Patterns are still widely used today, but for many developers, they are instinctive. No longer do developers debate the merits of using the Strategy pattern. Nor must they constantly reference the GOF book to identify which pattern might best fit their current need. Instead, good developers now instinctively design object oriented software systems.

Many patterns are also timeless. That is, they are not tied to a specific platform, programming language, nor era of programming. With some slight modification and attention to detail, a pattern is molded to a form appropriate given the context. Many things dictate context, including platform, language, and the intricacies of the problem you’re trying to solve. As we learn more about patterns, we offer samples that show how to use patterns in a specific language. We call these idioms.

I’d like to think the modularity patterns in this book are also timeless. They are not tied to a specific platform or language. Whether you’re using Java or .NET, OSGi or Jigsaw, or simply want to build more modular software, the patterns in this book will help you do that. I’d also like to think that over time, we’ll see idioms emerge that illustrate how to apply these patterns on platforms that support modularity, and that tools will emerge that help us refactor our software systems using these patterns. Out of this, I’m hopeful that tools will continue to evolve that help aid the development of modular software. For example, tools that provide important refactoring capabilities to help improve modularity. Time will tell.

Over the past several years, there have also been a number of object oriented design principles that have emerged. Many of these design principles are embodied within design patterns. The SOLID design principles espoused by Uncle Bob are prime examples. Further analysis of the GOF patterns reveals that many of them adhere to these principles.

But for all the knowledge shared and advancements made that help guide object oriented development, creating very large software systems is still inherently difficult.  These large systems are still difficult to maintain, extend, and manage. The current principles and patterns of object oriented development fail in helping manage the complexity of large software systems because they address a different problem. They help address problems related to logical design, but do not help address the challenges of physical design.

1.1 – Logical vs Physical Design

Almost all principles and patterns that aid in software design and architecture address logical design. Logical design pertains to language constructs such as classes, operators, methods, and packages. Identifying the methods of a class, relationships between classes, and a system package structure are all logical design issues.

It’s no surprise that since most principles and patterns emphasize logical design, the majority of developers spend their time dealing with logical design issues. When designing classes and their methods, you are defining the system’s logical design. Deciding if a class should be a Singleton is a logical design issue. So to is determining if an operation should be abstract, or deciding if you should inherit from a class versus contain it. Developers live in the code, and are constantly dealing with logical design issues.

Making good use of object oriented design principles and patterns is important. Accommodating the complex behaviors required by most business applications is a challenging task, and failing to create a flexible class structure can have a negative impact on future growth and extensibility. But logical design is not the focus of this book. There are numerous other books and articles that provide the guiding wisdom necessary to create good logical designs. However, logical design is just one piece of the software design and architecture challenge. The other piece of the challenge is physical design. In fact, if you don’t consider the physical design of your system, then your logical design doesn’t really matter all that much.

Physical design represents the physical entities of your software system. Determining how a software system is packaged into its deployable units is a physical design issue. Determining which classes belong in which deployable units is also a physical design issue. Managing the relationships between the physical entities is also a physical design issue. Physical design is equally as, if not more important than, logical design.

For example, defining an interface to decouple clients from all classes implementing the interface is a logical design issue. Decoupling in this fashion certainly allows you to create new implementations of the interface without impacting clients. However, the allocation of the interface and its implementing classes to their containing modules is a physical design issue. If the interface has three different implementations, and each of those implementation classes has underlying dependencies, the placement of the interface and implementation impacts the ease surrounding future reusability. Placing the interface and implementation in the same module introduces the risk of undesirable deployment dependencies. If one of the implementations is dependent upon a complex underlying structure, then you’ll be forced to include this dependent structure in all deployments, regardless of which implementation you choose to use.

Unfortunately, while many teams spend a good share of time on logical design, few teams spend time on physical design. Physical design is about how we partition the software system into a system of modules. Physical design is about software modularity.

1.2 – Modularity

Large software systems are inherently more complex to develop and maintain than smaller systems. Modularity involves breaking a large system into separate parts makes the system easier to understand. By understanding the behaviors contained within a module, and the dependencies that exist between modules, it’s easier to identify and assess the ramification of change.

For instance, software modules with few incoming dependencies are easier to change than software modules with many incoming dependencies.  Likewise, software modules with few outgoing dependencies are much easier to reuse than software modules with many outgoing dependencies. This tension between reuse and maintainability is an important factor to consider when designing software modules, and dependencies play an important factor. But dependencies aren’t the only factor.

Module cohesion also plays an important role in designing high quality software modules. A module with too little behavior doesn’t do enough to be useful to other modules using it, and therefore provides minimal value. Contrarily, a module that does too much is difficult to reuse because it provides more behavior than is desired. When designing modules, identifying the right level of granularity is important. Modules that are too fine-grained provide minimal value and may also require other modules to be useful. Modules that are too coarse-grained are difficult to reuse.

The principles in this book provide guidance on designing modular software. They examine ways that you can minimize dependencies between modules while maximizing a modules reuse potential. Many of these principles would not be possible without the principles and patterns of object-oriented design. However, as you’ll discover, the physical design implications that affect modularity often dictate the logical design decisions.

1.2.1 – Unit of Modularity – The JAR File

Physical design on the Java platform is done by carefully designing the relationships and behavior of Java JAR files. On the Java platform, the unit of modularity is the JAR file. While these principles can be applied to any other unit, such as packages, they shine when applied to designing JAR files. Many of the examples in this book use OSGi. However, OSGi is not a prerequisite for using the principles of modularity. OSGi simply provides a runtime environment that brings modularization to the Java platform.

1.2.2 – OSGi

OSGi Service Platform is the dynamic module system for Java. It provides a framework for managing bundles that are packaged as regular Java JAR files with an accompanying manifest. The manifest contains important metadata that describes the bundles and its dependencies to the OSGi framework. IThe OSGi framework offers the following capabilities:

• Modularity: Enables a modular approach to architecture on the Java platform.
• Versioning: Supports multiple versions of the same software module deployed within the same JVM instance.
• Hot deployments: Permits modules to be deployed and updated within a running system without restarting the application or the JVM.
• Service orientation: Encourages service-oriented design principles in a more granular level within the JVM.
• Dependency management: Requires explicit declaration of dependencies between modules.

1.3 – Chapter Synopsis

Here is a breakdown of the remaining chapters in this book. This book does not need to be read in sequential order. It has been structured so that you can read individual chapters. Chapter 2 introduces OSGi through a general discussion and some simple examples. If you’re familiar with OSGi, you can skip this chapter. If  you’re new to OSGi, I suggest you start here, though the modularity principles don’t require OSGi to be effective.

For an overview of logical design and the SOLID design principles, start with Chapter 4. Most of the material in this chapter is not new, but is provided for reference material that’s referenced when discussing the key principles of modularity. For an overview of physical design, start with Chapter 5. This discusses some of the key elements behind physical design, and many of the challenges we try to solve. Chapter 6 presents a reference implementation that can be used to show many of the key principles.

Chapter 7 begins our discussion of the principles of modularity, and talks about principles that help in managing dependencies between modules. Chapter 8 talks about principles related to the usability of modules, and Chapter 9 examines techniques that can be used to create more extensible modules. The material in chapter 9 makes heavy use of the SOLID principles.

In general, the book is divided into three main parts. Chapters 2 and 3 introduce OSGi. Chapter 4, 5, & 6 discuss logical and physical design. The remaining chapters explore the principles of modularity.

1.3.1 – Part 1 (The Case for Modularity)

Chapter 2 – Module Defined
Chapter 3 – Two Facets of Modularity
Chapter 4 – Architecture and Modularity
Chapter 5 – Taming the Beast
Chapter 6 – Realizing Reuse
Chapter 7 – Reference Implementation

It’s important to provide some decent samples that illustrate some of the concepts discussed. The reference implementation serves two purposes. First, it ties together the material in the first five chapters so you can see how these concepts are applied. Second, it lays the foundation for many of the heuristics presented in chapters 7 – 12. The reference implementation is an example of the heuristics applied in conjunction with other, and this chapter refers to many of these heuristics at the points where they are applied.

1.3.2 – Part 2 (Pattern Catalog)

Module Base Patterns

The base patterns are the fundamental elements upon which many of the other patterns exist. They establish the conscientious thought process that go into designing systems with a modular architecture. They focus on modules as the unit of reuse, dependency management, and cohesion. Each are important elements of well designed modular software systems.

Module Dependency Patterns

I’ve found it personally fascinating that development teams spend so much time designing class relationships, but yet spend so little time creating a supporting physical structure. Here, you’ll find some heuristics that will help you create a supporting  physical structure emphasizing the coupling between components. You’ll also find some discussion exploring how component design impacts deployment.

Module Usability Patterns

While coupling is an important measurement, cohesion is equally important. It’s easy to create and manage component dependencies if I throw all of my classes in a couple of files. But in doing so, I’ve introduced a maintenance nightmare. In this chapter, I introduce some heuristics that will help ensure our components are cohesive units. It’s interesting that you’ll find some contention between the heuristics in chapter 7 and those in chapter 6. I’m going to talk about this contention, what you can do to manage it, and when you want to do it.

Module Extensibility Patterns

A goal in design software systems is the ability to extend the system without making modifications to the existing codebase. Abstraction plays a central role in accomplishing this goal, but simply adding new functionality to an existing system is only part of the battle. We also want to be able to deploy those new additions with redeploying the entire application. The Extensibility Patterns focus on helping us do this.

Module Utility Patterns

The utility patterns aid modular development. Unlike the other patterns, they don’t emphasize reuse, extensibility, or usability. Instead, the base patterns discuss ways that modularity can be enforced and that help address quality related issues.

1.4 – Pattern Form

Each patterns is consistent in structure to help maximize it’s readability.  Each is also accompanied by an example illustrating how the underlying principle it captures is misapplied, followed by a more appropriate application. Not all sections will appear for all principles. In some cases, I’ll suppress certain sections when a previous discussion can be referenced. The general structure of each principle is as follows:

• Pattern Name
: The name of the principles. The name is important, since it helps establish a common vocabulary among developers.
• Pattern Statement: 
A summary describing the principle. This statement helps establish the intent of the principle.
• Sketch: 
A visual representation showing the general structure of the principle applied. Usually, UML is used here.
• Description
: A more detailed explanation describing the problem the principle solves. The description establishes the motivation behind the principle.
• Implementation Variations
: As with any principle or pattern, there are subtle implementation details that quickly arise when applying the pattern to a real world problem. Implementation Variations discusses some of the more significant alternatives you should consider when applying the principle.
• Consequences
: All design decisions have advantages and disadvantages, and like most advice on software design, the use of these principles must be judicious. While they offer a great deal of flexibility, that flexibility comes with a price. The Consequences section discusses some of the interesting things you’ll likely encounter when applying the principle, as well as some of the probable outcomes should you decide to ignore the principle. After reading through the consequences, you should have a better idea of when you’ll want to apply the principle, and when you may want to consider using an alternative approach. Boiled down, this section represents the advantages and disadvantages of using the principle, the price you’ll pay, and the benefits you should realize.
• Sample: It’s usually easier to understand pattern when you can see a focused example. In this section, we’ll walk through a sample that illustrates how the pattern can be applied. Sometimes, we’ll work some code and other times some simple visuals conveys the message clearly. Most important though is that the sample won’t exist in a vacuum. When we apply patterns in the real world, patterns are often used in conjunction with each other to create a more flexible tailored solution. In cases where it makes sense, the sample will build on previous samples illustrated in other patterns. The result will be insight to how you can pragmatically apply the pattern in your work.
• Related Patterns: 
Most of the principles are fundamental aspects of object orientation and design patterns that are used daily. In this section, I’ll examine patterns similar to the one presented, and explore the similarities and differences.

1.5 – Pattern Catalog

Below are the modularity patterns.

1.5.1 – Base Patterns

• ManageRelationships – Design Module Relationships.
• ModuleReuse – Emphasize reusability at the module level.
• CohesiveModules – Create cohesive modules.
• ClassReuse – Classes not reused together belong in separate components.

1.5.2 – Dependency Patterns

• AcyclicRelationships – Module relationships must be acyclic.
• LevelizeModules – Module relationships should be levelized.
• PhysicalLayers – Module relationships should not violate the conceptual layers.
• ContainerIndependence – Consider your modules container dependencies.
• IndependentDeployment – Modules should be independently deployable units.

1.5.3 – Usability Patterns

• PublishedInterface – Make a modules published interface well known.
• ExternalConfiguration – Modules should be externally configurable.
• ModuleFacade – Create a facade serving as a coarse-grained entry point to the modules underlying implementation.

1.5.4 – Extensibility Patterns

• StableModules – Modules heavily depended upon should be stable.
• AbstractModules – Depend upon the abstract elements of a module.
• ImplementationFactory – Use factories to create a modules implementation classes.
• SeparateAbstractions – Separate abstractions from the classes that realize them.

1.5.5 – Utility Patterns

• LevelizedBuild – Execute the build in accordance with module levelization
• TestComponent – For each module, create a corresponding test component that validates it’s behavior and illustrates it’s usage.

2.1 – Defining a Module

I’d like to say that a module is simply a “chunk of software.” Unfortunately, that doesn’t offer enough concision to differentiate a module from other software chunks, like objects, packages, services, or even applications. So we need to focus our definition a bit.

A software module is a deployable, manageable, natively reusable, composable, stateless unit of software that provides a concise interface to consumers.

That’s quite a mouthful (reminds me of the first definition of Java), and I feel badly about dumping this on us so early in a book that I had such high hopes for. But hopefully, after a bit of explanation, you’ll give me a second chance and keep on reading. We can illustrate this definition visually as shown in Figure 1. Let’s tease apart the definition.

Figure 1: Defining a Module

2.1.1 – Deployable

Modules are a unit of deployment. Unlike other software entities such as classes and packages, a module is a discrete unit of deployment that can be deployed alongside other software modules. In this sense, modules represent something more physical than classes or packages, which are intangible software entities. Examples of deployable unis of software include EAR, WAR, and JAR files.

2.1.2 – Manageable

Modules are a unit of management. In the presence of a runtime module system, software modules can be installed, uninstalled, and refreshed. During development, partitioning a system into modules helps ease a number of otherwise complicated activities. This includes improving build efficiency, allowing developers to independently develop autonomous modules, and plan the development effort along module boundaries. Examples of software entities that comply with this segment of the definition include EAR, WAR, and JAR files.

2.1.3 – Testable

A module is a unit of testability. Like a class can be independently tested using test driven development, a module can be independently tested as well. Examples of software entities that comply with this segment of the definition include classes, packages, and JAR files.

2.1.4 – Natively Reusable

Modules are a unit of intra-process reuse. Unlike applications or services, modules are not a distributed computing technology, though SOA principles can be used to design software modules. Instead, modularity are a way to organize units of deployment in a way that they can be reused across applications, but a module is always invoked natively. That is, the  operations exposed by a module are invoked by calling the method directly.

The way we reuse modules is also different than how we reuse services. Typically, a service is deployed only a single time and invoked by multiple consumers. Because modules are used intra-process, a modules is deployed with each process that intends to reuse its functionality. Examples of software entities that comply with this segment of the definition include classes, packages, and JAR files.

2.1.5 – Composable

Modules are a unit of composition. Modules can be composed of other modules. Typically, this involves coarse-grained modules being a composition of finer-grained modules.

2.1.6 – Stateless

Modules are stateless. There exists only a single instance of a specific version of a module. We don’t instantiate software modules, though we do instantiate instances of the classes within software modules, and these classes may maintain state. However, the module itself does not. Examples of software entities that adhere to this segment of the definition include WAR, EAR, and JAR files.

2.2 – A Bit More Succinctly

Before we get too far, I’d like to offer a bit more succinct definition of a software module. Upon applying each of the various segments of the definition, we can more clearly state that:

A Module is a JAR File!