I’m left a bit confused as to what this new config parameter will do… Maybe a more specific example would help here, showing why and how you would distribute files in different locations, and how the proposed config would allow you to point Liquibase at them.
(I’ll use the “search path” term here as a placeholder. Also maybe starting too basic but building it all up for whoever is following along)
Changelog files need to have paths coded into them. But we don’t want to be putting absolute paths in there, because the full paths are often different on different systems.
For example, if you had a root changelog like:
<databaseChangeLog>
<include file="/home/example/my-project/src/dbchangelog/changelog-1.0.xml"/>
<include file="/home/example/my-project/src/dbchangelog/changelog-1.1.xml"/>
<include file="/home/example/my-project/src/dbchangelog/changelog-1.2.xml"/>
</databaseChangeLog>
If you reorganize your machine so it’s not checked out to /home/example/my-project anymore those need to be changed. Or when another developer checks it out to a different location on their system. Or when you check them out to the build server. Or if you package them up into a spring application to run on startup. Etc.
So instead, you want those files to be the same for everyone like:
<databaseChangeLog>
<include file="dbchangelog/changelog-1.0.xml"/>
<include file="dbchangelog/changelog-1.1.xml"/>
<include file="dbchangelog/changelog-1.2.xml"/>
</databaseChangeLog>
and leave it up to a runtime configuration (searchPath) to specify where to look for those files. On your machine, you can set searchPath to /home/example/my-project/src/ On the build server, it might be /runner/checkout/src. When it runs packaged in an application, those paths are looked for in the application archive.
To make things easy, the CLI (and other Liquibase interfaces) will set up a default searchPath of “normal” places to look. The CLI for example, includes the current working directory in the search path, so as long as you run liquibase from /home/example/my-project/src/ you don’t have to set anything.
Besides include/includeAll, we also use the searchPath for finding data files in loadData/loadUpdateData as well as looking for liquibase.properties and other configuration files.
But… sometimes you aren’t running the CLI from /home/example/my-project/src/. What do you do then? You need to add that directory to your searchPath with --search-path=/home/example/my-project/src/. With that setting, it doesn’t matter where you run Liquibase, it will know where to look.
Or, maybe your changelogs get packaged into a changelogs.zip file on the build server. By setting --search-path=/path/to/changelogs.zip, Liquibase will find the files in that zip. Or your changelog files are stored remotely in an s3 bucket you can use --search-path=s3:/changelogs/latest.
The “searchPath” setting is what lets you specify those non-default places Liquibase should be looking for files in.
A bit more complex is a more componentized setup. For example, you may have an application which is divided up into a cart and an accounts sub-module, where each sub-module has it’s own changelog file. So your overall project structure looks like:
src/main/resources/dbchangelog/changelog-root.xml
accounts/src/main/resources/dbchangelog/accounts/changelog-root.xml
accounts/src/main/resources/dbchangelog/accounts/changelog-1.0.xml
cart/src/main/resources/dbchangelog/cart/changelog-root.xml
cart/src/main/resources/dbchangelog/cart/changelog-1.0.xml
cart/src/main/resources/dbchangelog/cart/changelog-1.1.xml
If you ran liquibase from the root of that directory structure, you’d run liquibase --changelog-file=src/main/resources/dbchangelog/changelog-root.xml update and your overall root changelog would look like:
<databaseChangeLog>
<include file="accounts/src/main/resources/dbchangelog/accounts/changelog-root.xml"/>
<include file="cart/src/main/resources/dbchangelog/cart/changelog-root.xml"/>
</databaseChangeLog>
which works, but is baking that current project structure into the changelog files. Since liquibase uses those paths for identifying changeSets, if you reorganize your project, want to reference the “compiled” versions in your project, or want to run from a built artifact you are going to hit problems.
Instead, you’d like to have your root changelog file look like:
<databaseChangeLog>
<include file="dbchangelog/accounts/changelog-root.xml"/>
<include file="dbchangelog/cart/changelog-root.xml"/>
</databaseChangeLog>
and set search-path=src/main/resources,accounts/src/main/resources,cart/src/main/resources
With that setting, when liquibase sees the reference to dbchangelog/accounts/changelog-root.xml, it searches for it in each of the specified searchPath locations. If you ever reorganize your project, you just update your searchPath and liquibase continues to identify all the files with a consistent path.
All that behavior is how Liquibase has always worked, but we used the “classpath” setting for it because early versions were more focused on running embedded in an application and the classpath is what provides that “search path” functionality in a java application. As we added a CLI etc., we just kept that “classpath” term (and originally implemented it relying on that java feature) but it continues to cause confusion so we’re looking to improve that terminology. And so we want to find the right word that describes that “list of base locations we look in for all the later specified paths in”. Ideally something that works for that overall concept like “Any time you are defining a path to a resource (like include’s ‘file’ or includeAll’s ‘path’ or the changelogFile setting) it is looked in the search path. Each interface will configure the search path with reasonable defaults, but they can be added to with the searchPath setting”
Does that help?
Nathan
I confess I’ve always kept things super simple with Liquibase, running it from the CLI with the changelog.xml and all changelog XML files in the current directory. Maybe a more specific example would help here, showing why and how you would distribute files in different locations, and how the proposed config would allow you to point Liquibase at them.