Modular Architecture

Statement

Perform the build in accordance with module levelization.

Description

An automated and repeatable build is a critical aspect to most successful development projects. First and foremost, an automated and repeatable build forces you to integrate early and integrate often, so you’re guaranteed to always have a system that works. Of course, there are a couple of rules you have to follow. First, any compile error that ever creeps in must be immediately resolved. Since you should be using a version control system, such as CVS, this means that the CVS projects comprising your application should always be free of any compilation errors. Second, any unit test that fails must be immediately fixed. Unit and acceptance tests should be run after each change you make, and if you ever find that a test fails, you should treat this failure with the same urgency you’d give a compile error. While pretty simple, these two rules are key elements because they are part of what defines an automated and repeatable build. So here, in order, are the characteristics defining an automated and repeatable build [Fowler].

• It must be a clean compile, meaning that no compiler generated syntactical errors occur. While it’s acceptable to encounter certain warnings, such as those generated by deprecated method usage, you should strive to remove these warnings before future builds.
• The build should be done using the most current version of all source files. There are occasions where you’ll want to compile older versions of code to revert back to a previous version. Typically, when compiling these older versions you’ll use the source files consistent with the version of the application you’re building.
• A clean build implies that all application source files are compiled. Conditional compiles or partial builds do not prove syntactical correctness of the entire application. You do not need to compile existing binary components.
• The build should generate all files representing the units of deployment. For Java applications, this means all .jar, .war, and .ear files should be created.
• The build should verify that the application is functionally correct, meaning all unit tests for the application should be run. If all unit tests pass successfully, the build is successful.

A rather significant aspect of an automated and repeatable build is that it be separated and performed outside the confines of your favorite integrated development environment (IDE). Most IDEs allow you to perform conditional compiles, as well as full compiles within the environment itself.  Performing the build within an IDE leaves too many loopholes open for individual developers to produce inconsistent results. In fact, without a repeatable build script, it would be very difficult to enforce the components that compose your application.

The most common manifestation of an automated and repeatable build is a full classpath build, where all source files are compiled as part of the same target1. Since all source is included within the same target, you’re also required to include all dependent components in the classpath of that target. In doing so, you lose your ability to enforce physical dependencies as part of the build process, and the physical dependencies between components that you expect to exist, may in fact, not be entirely accurate. With a full classpath build, there is no good way to enforce your physical dependency structure.

A LevelizedBuild, however, allows you to build each component independently as part of a component specific target, where each target has it’s own build classpath. Since each target specifies it’s own classpath, you are only required to include dependent components for the source being built instead of all components for the complete application. Any developer creating a dependency on a component not in the targets build classpath will cause a compilation error to occur when the build process executes.

A LevelizedBuild can only be performed if components themselves have been levelized and all relationships between components are acyclic. Cycles existing between any two components will require that each component be built at the same time. Given any two or more components, if a cycle exists in the dependency structure among any of those components, then there is no way to get the binary form of one component without also having the binary form of all dependent components. Because of the cycle between the components, you’ll be required to build all as part of the same target. Figure 1 illustrates a Levelized Build.

Figure 1: Levelized Build

Implementation Variations

A LevelizedBuild can be a bit more complex that performing a full classpath build. If you’re using a build tool like Ant, you’ll find that you have quite a few more targets. Like most complex development issues, breaking down the problem can help simplify it significantly. Modularizing your build scripts by breaking a single complex script out into multiple simpler scripts is beneficial. A logical way to break up the build script is to define a separate build script for each component. These component specific build scripts can also build the test component, and even run the test cases. A simplified main build script can be setup to call each of the component build scripts. Additionally, you also have the ability to independently build each component, as well as include the component in other application build scripts.

Another option worth considering is using JDepend to help enforce levelization of components. JDepend offers you the ability to create tests cases that can verify package dependencies.  While you aren’t actually performing the build with JDepend, you still verify levelization of components by defining test cases that verify the package relationships that span component boundaries.The basic idea is that within your test case, you specify the package dependency constraints, and if any relationships exist that violate the constraints in the test case, the test will fail. So if a new relationship is introduced between component boundaries that isn’t allowed, the test case, and ultimately the build, will fail. With this approach, you can actually perform a full classpath build, and allow JDepend to enforce the levelization for you. I personally favor a LevelizedBuild over this approach since managing component relationships by defining package dependency constraints can become tedious, especially early in the lifecycle when the application structure is subject to wide refactorings that can significantly affect both logical and physical structure.

Same build script vs separate build scripts. Ability to build modules in parallel, at the same time, instead of a serialized build.

Consequences

Managing dependencies between components is important, but enforcing the dependencies can be a rather daunting task, especially for large development teams. An elegant conceptual design can quickly evolve into a tangled mess during implementation where not one developer understands how the initial high level vision has manifested itself in code. And though you may have made it clear what dependency relationships you expect, undesirable dependencies have a way of creeping into your application due to a variety of reasons. A LevelizedBuild, where you include only the required components in the classpath, helps enforce component relationships.

I’m too sure how many development teams design their physical relationships (ask on-line and put the stat here). But I suspect those that do are in the minority. Performing a LevelizedBuild forces you to think about component dependencies. Early in the development lifecycle, you have to define your component dependencies to create the build script. As development progresses, it can be tempting to introduce new dependencies on other components. However, when you perform a LevelizedBuild, you’ll also be forced to modify the build script each time a new component dependency is introduced. Since any new dependency requires a change to the build script, defining any new dependency must be a very conscious decision on the part of a developer. While a LevelizedBuild makes it a bit more effort to introduce new dependencies, I also feel that introducing heavyweight dependencies is an important enough design and architectural decision to warrant the extra effort.

Sample Code

Recall from Manage Relationships how we eliminated the dependencies between two modules by allocating an abstract class to it’s own module. Figure 1 illustrates this. Once we have a module structure that is acyclic as we have here, we can setup a levelized build.

Figure 1: Acyclic Module Relationships

(Use sample from Acyclic Relationships pattern.)Building on the billing sample discussed in previous sections, let’s examine how a LevelizedBuild can help enforce component relationships. The diagram in Figure7.1 shows the component relationships between the physical entities. Our initial build script performs a full classpath build. While the result is going to be the same as our final solution discussed later, there is no way to enforce the relationships in Listing 1 using a full classpath build. In the following snippet from our Ant build script, the path id specifies the location for all application and test source code, as well as the location of Struts and the Servlet API.

<project name="Comp" default="compile" basedir=".">

   <property name="src" location="${basedir}/src"/>
   <property name="build" location="${basedir}/build"/>
   <property name="lib" location="${basedir}/lib"/>
   <property name="buildstats" location="${basedir}/stats"/>
   <property name="dist" location="${basedir}/bin"/>
   <property name="version" value="1.0"/>

   <path id="project.class.path">
      <pathelement path="${src}"/>
      <pathelement path="${lib}/junit.jar"/>
   </path>

   <target name="clean">
      <delete dir="${build}"/>
      <delete dir="${buildstats}"/>
      <delete dir="${dist}"/>
   </target>

   <target name="init" depends="clean">
      <mkdir dir="${build}"/>
      <mkdir dir="${buildstats}"/>
      <mkdir dir="${dist}"/>
   </target>

   <target name="compile" depends="init">
      <javac srcdir="${src}" destdir="${build}">
         <classpath refid="project.class.path"/>
      </javac>

      <jar jarfile="${dist}/bill.jar" basedir="${build}" includes="com/kirkk/bill/**"/>
      <jar jarfile="${dist}/cust.jar" basedir="${build}" includes="com/kirkk/cust/**"/>
      <jar jarfile="${dist}/base.jar" basedir="${build}" includes="com/kirkk/base/**"/>
      <jar jarfile="${dist}/billtest.jar" basedir="${build}" includes="com/kirkk/test/**"/>

      <junit printsummary="yes" haltonfailure="yes">
         <classpath>
            <pathelement path="${dist}/bill.jar"/>
            <pathelement path="${dist}/cust.jar"/>
            <pathelement path="${dist}/base.jar"/>
            <pathelement path="${dist}/billtest.jar"/>
            <pathelement path="${lib}/junit.jar"/>
         </classpath>
         <test name="com.kirkk.test.AllTests" outfile="junitresults">
            <formatter type="plain"/>
         </test>
      </junit>
   </target>
</project>

Listing 1: Full Classpath Build

Below is the levelized version which does the exact same thing except it enforced module dependencies.

<project name="Comp" default="testcompile" basedir=".">

   <property name="src" location="${basedir}/src"/>
   <property name="build" location="${basedir}/build"/>
   <property name="buildsrc" location="${basedir}/buildsrc"/>
   <property name="lib" location="${basedir}/lib"/>
   <property name="buildstats" location="${basedir}/stats"/>
   <property name="dist" location="${basedir}/bin"/>
   <property name="version" value="1.0"/>

   <target name="clean">
      <delete dir="${build}"/>
      <delete dir="${buildsrc}"/>
      <delete dir="${buildstats}"/>
      <delete dir="${dist}"/>
   </target>

   <target name="init" depends="clean">
      <mkdir dir="${buildstats}"/>
      <mkdir dir="${dist}"/>
   </target>

   <target name="basecompile" depends="init">
      <mkdir dir="${build}"/>
      <mkdir dir="${buildsrc}"/>
      <copy todir="${buildsrc}">
         <fileset dir="${src}">
            <include name="com/kirkk/base/**"/>
         </fileset>
      </copy>

      <javac srcdir="${buildsrc}" destdir="${build}">
         <classpath>
            <pathelement path="${buildsrc}"/>
         </classpath>
      </javac>

      <jar jarfile="${dist}/base.jar" basedir="${build}" includes="com/kirkk/base/**"/>

      <delete dir="${buildsrc}"/>
      <delete dir="${build}"/>
   </target>

   <target name="billcompile" depends="basecompile">
      <mkdir dir="${build}"/>
      <mkdir dir="${buildsrc}"/>
      <copy todir="${buildsrc}">
         <fileset dir="${src}">
            <include name="com/kirkk/bill/**"/>
         </fileset>
      </copy>

   <javac srcdir="${buildsrc}" destdir="${build}">
      <classpath>
         <pathelement path="${buildsrc}"/>
         <pathelement path="${dist}/base.jar"/>
      </classpath>
   </javac>

   <jar jarfile="${dist}/bill.jar" basedir="${build}" includes="com/kirkk/bill/**"/>

      <delete dir="${buildsrc}"/>
      <delete dir="${build}"/>
   </target>

   <target name="custcompile" depends="billcompile">
      <mkdir dir="${build}"/>
      <mkdir dir="${buildsrc}"/>
      <copy todir="${buildsrc}">
         <fileset dir="${src}">
            <include name="com/kirkk/cust/**"/>
         </fileset>
      </copy>

      <javac srcdir="${buildsrc}" destdir="${build}">
         <classpath>
            <pathelement path="${buildsrc}"/>
            <pathelement path="${dist}/base.jar"/>
         </classpath>
      </javac>

      <jar jarfile="${dist}/cust.jar" basedir="${build}" includes="com/kirkk/cust/**"/>

      <delete dir="${buildsrc}"/>
      <delete dir="${build}"/>
  </target>

   <target name="testcompile" depends="custcompile">
      <mkdir dir="${build}"/>
      <mkdir dir="${buildsrc}"/>
      <copy todir="${buildsrc}">
         <fileset dir="${src}">
            <include name="com/kirkk/test/**"/>
         </fileset>
      </copy>

      <javac srcdir="${buildsrc}" destdir="${build}">
         <classpath>
            <pathelement path="${buildsrc}"/>
            <pathelement path="${dist}/base.jar"/>
            <pathelement path="${dist}/cust.jar"/>
            <pathelement path="${dist}/bill.jar"/>
            <pathelement path="${lib}/junit.jar"/>
         </classpath>
      </javac>

      <jar jarfile="${dist}/billtest.jar" basedir="${build}" includes="com/kirkk/test/**"/>

      <junit printsummary="yes" haltonfailure="yes">
         <classpath>
            <pathelement path="${dist}/bill.jar"/>
            <pathelement path="${dist}/cust.jar"/>
            <pathelement path="${dist}/base.jar"/>
            <pathelement path="${dist}/billtest.jar"/>
            <pathelement path="${lib}/junit.jar"/>
         </classpath>
         <test name="com.kirkk.test.AllTests" outfile="junitresults">
            <formatter type="plain"/>
         </test>
      </junit>
      <delete dir="${buildsrc}"/>
      <delete dir="${build}"/>
   </target>
</project>

Listing 2: Levelized Build

The build will fail if the build classpath isn’t correct, essentially enforcing module dependencies.

Wrapping Up

A levelized build will help enforce the module structure within an application. If undesirable dependencies are introduced between modules, the build will fail. Unfortunately, because the build has knowledge of module structure, if the structure changes, the build will have to change as well. This cost, however, is well worth the price of admission. That is, enforceable module dependencies.