Chapter 7 – Modularity and SOA

At this point, it’s doubtful that any of us will confuse modularity with service oriented architecture (SOA). The two appear to be distinctly different beasts. Yet, they are very similar in nature and each offers tremendous benefit to organizations. At one level, the principles behind service design apply equally to module design. Like we try to create services that are loosely coupled and highly cohesive, we also try to design modules with similar goals.

Yet module design and service design are incredibly complementary in other ways, as well. In Chapter 2, we defined a module and illustrated how modules are excellent candidates for intra-process reuse, while service reuse crosses process boundaries. In Chapter 4, we talked about modularity in a broader architectural context, illustrating how it helps us achieve the goal of architecture. Before we progress onto the pattern discussion, it’s important to discuss modularity in the context of other architectural principles, and illustrate how modularity helps us realize many of the benefits we’ve discussed thus far.

7.1 – All The Way Down, Revisited

TheIvoryTowerAndArcAllTheWay

Figure 7.1 – Social Architecture All The Way Down

In Chapter 4, I talked about how important it is that we “architect all the way down”. It helps increase transparency and understanding between developers and architects by emphasizing a lot of the middle ground that noone ever seems to focus on. It’s just another reason why modularity is so important. I used the diagram in Figure 7.1 to illustrate the point. Look at the huge gap that exists if we don’t focus on the stuff highlighted by the gray bar in the middle.

One reason I like this visual is that it illustrates the social aspect of software architecture. Yet, there are other significant advantages to architecture all the way down that we haven’t explored yet. Another is structural flexibility.

7.2.1 – Structural Flexibility – Different Entities, Different Purpose

Another benefit of module design in filling that middle ground is that modules can offer different benefits than classes and services. The diagram in Figure 7.2 illustrates some of the capabilities of different types of entities, which is a variation of the diagram we used when defining a module in Chapter 2.

ArchitectureAllTheWayDownStructural

Figure 7.2 – Structural Architecture All The Way Down

For example, classes are pretty easily reused within an application, but because classes aren’t a unit of deployment, it’s difficult to use them across applications. Of course, intra-application reuse is a sweet spot of services, since they’re method of invocation is through some distributed protocol (SOAP, HTTP, even RMI/IIOP). Yet because services are invoked remotely, we typically like to make them coarser-grained because of the performance implications of distributed requests. So this begs the question – If a service is too coarse-grained to reuse (ie. it does more than what we want), how do I reuse some desirable behavior across applications? Well, without modules, our only other choice is the class. Since a class can’t be reused intra-process, we do one of either two things. Expose it as a service or copy the class (ie. the code). Given the circumstances, neither of these may be ideal. Another option is desirable.

Modules represent that other option. They are a finer level unit of granularity than services, and are a unit of deployment. Since each of these different types of entities are units of composition, we have tremendous flexibility in how we assemble applications. Possibly more important though is our increased ability to accommodate the architectural shifts that naturally occur as requirements evolve. Because one of the most difficult aspects of architecture is designing entities at the right level of granularity.

7.2 – Granularity, Architecture’s Nemesis

Granularity is the extent to which a system is broken down into it’s behavioral entities. Coarse-grained entities tend to be richer in behavior than fine-grained entities. Because coarse-grained entities do more, they tend to be larger than fine-grained entities, and are easier to use. In general, a fine-grained entity is more reusable than a coarse-grained entity because it does a little bit less.  If it does less, then it’s more likely to apply across a wider variety of usage scenarios. We spent some time discussing this tension in Chapter 6. Now, I’d like to go a bit deeper which will help more clearly illustrates the concept of “architecture all the way down.”

Unfortunately, while there are a lot of patterns, principles, and idioms that offer really good guidance surrounding many aspects of architecture and design, there isn’t a lot of guidance surrounding granularity. Let’s start with a really simple example that illustrates the challenge.

7.2.1 – A Really Simple Example

Consider the following save method:

public class Person {
private String firstName;
private String lastName;
private String ssNumber;
public void save() {
ValidationErrors errors = this.validate();
      if (errors != null) {
         //handle the errors somehow
      } else {
//save to the database.
      }
}
private ValidationErrors validate() {
//perform validation
}
}

As can be seen, the save method invokes the validate method before saving the information to the database. Generally speaking, this makes saving the information easier (and possibly safer) since whatever invokes save doesn’t have to worry about validating. And this seems to be a good thing.

Yet, what happens when a new consumer of this method wants to save the information but apply a slightly different set of rules than the current validation method provides? Here’s where the existing save method is simply too coarse-grained. It does just a little bit too much. We’ve made it easier to use, but also less reusable.

There are quite a few variations I could go with at this point to improve the design. One would be to make the validate method public, and have clients invoke the validate method before invoking save. Then, if validate didn’t do what it was supposed to, clients could always opt out of invoking the validate method and apply their own validation before invoking save. Certainly less safe though because validation is optional.

Yet another alternative might be the following:

public class Person {
   private String firstName;
   private String lastName;
   private String ssNumber;
   public void save(Validator validator) {
      ValidationErrors errors = validator.validate();
      if (errors != null) {
         //handle the errors somehow
      } else {
         //save to the database.
      }
   }
}

This seems to offer the best of both worlds. I now require a Validator be passed in before I can save, and if the Validator is an interface, I can certainly allow clients to pass in the Validator they need. Of course, I could easily pass in a NOP validator.

Whatever…there are lots of different options, and each has their own set of tradeoffs. I’d like to avoid extensive debate surrounding which approach to save and validation is best and instead focus on the main point here, which is that achieving the right level of granularity for an entity can be very challenging. It requires more than just looking at the problem from a code level viewpoint and demands that we possess contextual information that will help us answer the question, “What’s the right level of granularity?”

7.2.2 – Bring It Up A Level

The code above was a pretty simple example that illustrates the challenges of granularity at the method level. But the same challenges exist when developing classes, packages, modules and services, and we’ll see an example of this shortly. So what I really want to know is

How do I determine the appropriate level of granularity?

This is the million dollar question. Because granularity is a significant inhibitor to creating reusable software that’s easy to use. (Oh, and managing dependencies too.) If something does too much, it’s less reusable. If something does too little, it’s more difficult to reuse.

MAKE MENTION OF ROTTING DESIGN AND DEPENDENCY MANAGEMENT.

The diagram in Figure 7.1 does a good job illustrating one view of granularity. Services are more coarse-grained than modules which are more coarse-grained than packages which in turn are slightly more coarse-grained than classes. This begins to help answer the question, “What’s the right level of granularity?”

If I’m concerned that a service I create is simply too coarse-grained and fails to maximize its reuse potential, I can break the behaviors of the service out into modules that are a bit finer grained and more reusable (of course, one might consider doing this in general). Then I can compose the services from the modules and reuse the modules across services. The result is different entities at different levels of granularity that lends tremendous flexibility to how I compose, use, and reuse software entities.

This provides some guidance on the level of granularity at which different types of software entities should be defined. However, it still doesn’t offer enough guidance to determine the right level of granularity for the save method in our example above.

7.2.3 – Another Dimension

There’s another dimension that is relevant, and that ties in nicely (though in a rather subtle way) to the initial coding example. It has to do with the way we traditionally layer our software. The diagram in Figure 7.3 illustrates the typical architectural layers commonly found in many systems. Generally, as we move down the layered hierarchy, entities become finer grained.

architectureallthewaydowngranularitylayersonly

Figure 7.3 – Architectural Layers

While this isn’t absolute, and depends on your architectural principles and style, entities in higher level layers tend to be more coarse-grained than entities in lower level layers because they are an amalgamation of their own behavior and the behavior of entities in the lower level layers.

Armed with this information, we can determine the right level of granularity for the save method as long as we understand in which layer the code containing the save method lives. If it’s in the data access layer, and we place architectural constraints on business rules living in the data access layer, then we shouldn’t have any validation code that lives in the data access layer. If the save method lives in a class in the domain layer, however, it may be suitable for the save method to contain validation code in the domain layer and invoke another method in the data access layer that actually performs the save operation.

7.2.4 – The Complete Picture

At this point we can make the following general statements:

  • Entities in higher level layers are more coarse-grained than entities in lower level layers.
  • Services are coarser than modules which are coarser than packages which are coarser than classes.

And if we combine these two ideas into a single thought, and overlay the two diagrams, we can begin to visualize the level of granularity for different types of software entities. Figure 7.4 illustrates how entities become finer-grained as we move from both left to right and top to bottom.

architectureallthewaydowngranularity

Figure 7.4 – Granularity and Architecture All The Way Down

Certainly this doesn’t offer hard guidance. Realistically, there are very few architecture and design principles and patterns that are universally applicable. In fact, I’m not certain you’d ever create a “presentation service.” But certainly you might create a data service that is composed of multiple data access modules. And you might have separate data services that reuse a data access module. And you might have a business service that uses a data service and is also composed of multiple domain modules. But you definitely do not want a data access module that invokes a business service, nor a business service that references a presentation module.

So this general guidance can serve as useful information to help answer our question, “How do I determine the appropriate level of granularity for a software entity?” And it can also serve as guidance when establishing architectural principles and constraints that help determine where software entities with specific behaviors should reside. Because without some scheme that will help determine the appropriate level of granularity, it’s hopeless to imagine that we’ll be able to design reusable software entities that are also easy to use.

7.2.5 – A Service Example

Let’s take another example. Say I have a business function called Pay Bill for which I develop a web service that can be invoked by many different consumers. That service happens to be relatively coarse-grained, and performs all the steps involved in paying the bill. These happen to include the following:

  • audit bill – apply a discount to the bill based on payee
  • check for duplicate – ensure the bill hasn’t already been paid
  • remit payment – cut the check
  • reconcile payment – reconcile with accounts payable financials

This seems reasonable. We have a nice little service that we can reuse any time we want to pay a bill. Unfortunately, the real world often gets in the way of our idealistic solutions. In fact, there are two problems that will eventually surface, and modularity benefits both scenarios. Let’s start by looking at the first scenario.

What should I do when a different scenario arises that demands I follow a slightly modified Pay Bill function?

As part of the remit step, I have a new payee that demands electronic payment. This is pretty easy actually. I simply modify the service to support electronic payments and then configure the service to context for that specific payee. So how does modularity help here?

If the service is composed of modules, it’s going to be much easier for me to understand the structure of the service, assess the impact of what’s it’s going to cost to change the service, and then introduce a new module (or modify the existing module) to provide the new capability. Without modules, I’m simply wading through the code trying to figure out all of these things. Now, the 2nd scenario.

What should I do when I want to reuse just one step of the Pay Bill function?

Let’s say another new requirement emerges. Whereas traditionally bills were entered by data entry personnel, we now have to support electronic delivery of bills. We also know that bills delivered electronically are often duplicates. It’s just one of those business things, you know? If we don’t pay the bill on the day it’s received, the billing party sends us the bill again, asking for payment. So we need to check for duplicates before we save the bill to the database and prepare it for processing. What do we do?

We could take the same approach as before and modify the Pay Bill service so that the duplicate check could be invoked separately from the higher level pay bill function. But that’s a bastardized design solution. Why? We are exposing behavior of the Pay Bill service that shouldn’t be exposed. The API is coarse-grained in some areas and fine-grained in others.

Maybe exposing the finer-grained capabilities of the Pay Bill function isn’t a severe compromise, but it is a compromise nonetheless. And we are forced to compromise because of architectural inflexibility. We don’t have architecture all the way down, and are therefore left with limited choice as to how we support the new requirement. We can either modify the service to reuse what we know is already encapsulated within the service, or copy the code to create something new. But those are the two options we have, and neither may be ideal.

As we continue to hack the system in this manner, with a multitude of other similar changes, the design will rot. Eventually, our Pay Bill service is transformed into a utility that performs all general-purpose bill functions. It’s API is a mix of coarse-grained and fine-grained operations. It’s become the monolith that we’re trying to avoid. While the Pay Bill service is still pretty reusable (it does everything now), it isn’t that usable. That tension between reuse and use that we discussed in Chapter 6 surfaces again.

Our decision to modify the Pay Bill service to expose the duplicate check was driven by one thing – ease. It was our easiest option. Really, it was our only option. But it isn’t the best option.

If we architect all the way down, we have another option. A Pay Bill service composed of modules that audit the bill, check for duplicates, remit payment, and reconcile the payment means we can choose the desirable solution over the easiest short term choice. We can avoid the long term compromises that degrade architectural integrity.

Shown in Figure 7.5, we see the service composed of modules. If we have a check for duplicates module, we can simply reuse that module in the new service or application that’s going to process electronic bills. Or we might expose the capabilities of the check for duplicates module as a separate service. We have multiple reuse entry points, as well as different levels of granularity through which our software entities can be managed, deployed, built, and much more.

My point here isn’t to debate the original design nor the decisions made to reuse the check for duplicates functionality in another service or application. Certainly, debating architectural and design decisions is a favorite past-time of developers. There are a variety of different ways that we can support new requirements as they emerge. Some are better than others and all are contextual.

NewProcessSample

Figure 7.5 – Service Composed of Modules

7.3 – An Alternative View

When designing software systems, teams often have a tendency to overemphasize service design and the aesthetics of the user experience (UX). While designing the UX and services using SOA principles are incredibly important, neglecting to pay careful attention in designing the service implementation can result in software that is difficult to change. Modularity shines in helping designing flexible service implementations. The diagram in Figure 7.6 illustrates this concept.

ServiceImplementation

Figure 7.6 – Designing Service Implementations

When designing services, teams often pay careful attention to designing the service interface and devote time to understanding and adopting the appropriate interface style, such as REST, POX, SOAP, or messaging. A great benefit of SOA is that individual services can be autonomously reused and evolve independent of other services. Unfortunately, if careful attention isn’t given to how the service implementation is structured, the service loses it’s ability to evolve and survive.

7.6 – Summary

The notion of architecture all the way down gives us options, and these options help maintain architectural integrity. They increase our options when making decisions and allow our system to accommodate unforeseen architectural shifts. I can modify an existing application or service to give me what I need. Or I can reuse an existing module that’s already available for me. Or I can compose a new service from an existing set of modules. Or I can break apart existing modules to create new modules that result in new services. And I can do a lot of this refactoring without significant impact on the existing code. We’ll see this idea in action in the next chapter, as well as throughout the pattern discussions.