Multiple schemas: multiple LB sets?

General Discussion
1

Hi,

We are working on an application that is subdivided into several licensable sub-applications.
There is one central base application, that contains some metadata on all the sub-applications. Apart from that sub-applications are not allowed to refer any other sub-application but BASE.
Up until now we had a separate directory structure for each sub: BASE/install & BASE/update for the BASE app, FIN/install & FIN/update for the FIN sub, etc.
Working in dev and propagating changes to test when changes occurred went fine: we would run the updates from BASE, followed by the other subs.
However, when doing a fresh install after some time, we ran into the following:

01/01/2010: in FIN a table is added. Info about that table is inserted into BASE.TABLES
01/03/2010: in BASE.TABLES a column ind_backup was added. An sql is provided as part of this script that updates existing rows. Finally the column is made mandatory
01/04/2010: in FIN a table is added. Info about that table is inserted into BASE.TABLES, including a value for the new (mandatory) column

This scenario works out ok when run in this order (as we do on dev & test). However, when run as a fresh install, the order will be:

  • First run all BASE scripts
  • then run all other scripts (per sub-app)
    So, this means that the mandatory column is created before the first insert on BASE.TABLES takes place.
    The 01/01/2010 insert in turn fails, because at that time the mandatory column did not exist.

So, my questions are:
Is this a familiar scenario?
Is there a smart solution for this that we are overlooking?
Should we go into a single LB structure and explicitly define the schema in every changeset?

Thanks in advance for looking into this,
Frank

2

You can definitely run into problems like you mention if you have multiple changelogs that work on the same tables.  There isn’t a single, simple answer, however. 

A couple options you can look into:

  • Combine everything into a single liquibase changelog.  If you don’t want to always create all tables, you can use contexts on each changeSet to list the apps it should be run for.
  • Duplicate changeSet definitions across your apps for where the order can matter, but put preconditions in place to check if a similar changeSet had already ran from another changeLog
  • Break your BASE table changeSets into groups of changeLog files based on when the changes were made and the files in the sub-apps as necessary in the correct places.

Do any of those ideas help?

Nathan

3

Thanks for your answer. I really appreciate it.

If I understand your suggestions correctly, some presume that the changesets that already ran are changed in retrospect. (i.e. after we realize the 01/02 change, we should do something with the 01/01 change)
Since the dev and test environments are built using the original 01/01 file, all of the testing is done based on that.
Therefor I’d rather not change files once they are “installed” on the test environment.
Using preconditions would also add the obligation to make several passes:
first run - part a: BASE files - 01/02 has preCondition that 01/01 must be run first.
first run - part b: FIN (and other subs) files
second run - part a: BASE - now 01/02 can run
…
This could add up, which also makes it hard to create a generic installation procedure.

It could be that I either misinterpreted your suggestions or overlooked something, but considering the above mentioned, I am not sure if I can go with either of your suggestions.

Frank