Usage in a highly modular application

General Discussion
1

We are using Liquibase in a highly modular application. Until today we put all changelogs for the whole application into .xml files and structured them like it is described in the best practices.

In future we want to put the changelogs into the corresponding modules. So it would be possible to execute only those changes which exist in the deployment scenario. Is there a way to do that? We´ve tried a solution with the includeAll property, but its not possible to include multiple directories with the same name in different jars. Our idea was to use a structure like that:

  • liquibase module
    • org/example/liquibase/master.xml
      • <includeAll path="META-INF/liquibase" />
  • module 1
    • META-INF/liquibase/module1.xml
      •   < include file = "org/example/module1/liquibase/master.xml" />
    • org/example/module1/liquibase/master.xml
      •   < includeAll path = "org/example/module1/liquibase/r0.12" />
      •   < includeAll path = "org/example/module1/liquibase/r0.13" />
    • org/example/module1/liquibase/r0.12/2012-12-01-test.xml
    • org/example/module1/liquibase/r0.12/2012-12-01-test2.xml
    • org/example/module1/liquibase/r0.13/2012-12-09-test.xml
    • org/example/module1/liquibase/r0.13/2012-12-10-test.xml
    • ... further changes of this module
  • module 2
    • META-INF/liquibase/module2.xml
      •   < include file = "org/example/module2/liquibase/master.xml" />
    • org/example/module2/liquibase/master.xml
      •   < includeAll path = "org/example/module2/liquibase/r0.12" />
      •   < includeAll path = "org/example/module2/liquibase/r0.13" />
    • org/example/module2/liquibase/r0.12/2012-12-01-test.xml
    • org/example/module2/liquibase/r0.12/2012-12-03-test.xml
    • org/example/module2/liquibase/r0.13/2012-12-08-test.xml
    • org/example/module2/liquibase/r0.13/2012-12-10-test.xml
    • ... further changes of this module

The current implementation is not supporting a structure like that, because the classpath scanning and the sorting of the changelogs by naming conventions is not possible and its hard to extend the current implementation.

So it woud be interesting if there is a way to realize a modular changelog execution or if this is a completely wrong approach?

Thanks

Andreas

2

There is definitely a way to do it, but if you want to be truly modular you’re up against some interesting challenges.  I’ll describe how we do it at Jenzabar.


As background, we build higher education ERP software as Maven projects.


One of the nice things about Maven projects is that the dependency graph is already built in.


Next, Java is able to look up resources in a filesystem independent manner, and tell you where all resources with the same given name are.


If you put all this together, then you get something like the following.  I’ll describe it in terms of our integration testing harness, but we do this at .ear file assembly time too:


In each database-oriented Maven project, make sure it has a changelog.xml in src/main/resources/META-INF/liquibase/changelog.xml.  That way, just as you suggest, each module carries its own DDL and default data.  If you pack this module up in, say, an .ear file with other such modules, then taken together you have the whole database schema, and no more than you need.


At integration test time (or any other time where several modules are logically associated together), do a ClassLoader#getResources(“META-INF/liquibase/changelog.xml”) and then sort the resulting URLs in Maven dependency order (topological sort).


Throw together a temporary changelog somewhere that consists of statements (in Maven dependency order) that include the other changelogs.  In our case, those other changelogs are always resolved, so they’re file-based URLs to a developer’s local Maven repository.


I hope to be able to open source some of this someday; hope this sketchy overview helps.


Best,

Laird


–
http://about.me/lairdnelson