V4.0.0 upgrade help

General Discussion
1

Hi,

We’ve successfully been using Liquibase V3.6.3 for a while now for development of a new product that hasn’t gone into production yet and I’m currently investigating upgrading to V4.0.0 but I’m having a weird issue which is causing Liquibase runs to fail when trying to run with 4.0.0.

Our Liquibase scripts are all xml based and we have create table scripts, some sql scripts for triggers and some loadData scripts (using csv files) for loading test data for our dev and test environments. We’re currently using MySQL 5.7.

Everything worked fine for us in Liquibase 3.6.3 and I’ve also tried 3.10.2 and that worked fine as well, although I did have to tweak our csv files as we have columns in some tables that are nullable and when we were creating the csv files we just had no value at all for the nullable columns and that worked fine in 3.6.3, but 3.10.2 didn’t like it. So for example I changed from something like this:

ID,COLUMN_1,NULLABLE_COLUMN,COLUMN_2
1,Some text,,more text

to this:

ID,COLUMN_1,NULLABLE_COLUMN,COLUMN_2
1,Some text,NULL,more text

The problem that I’m having with 4.0.0 is that its complaining about one of our loadData scripts, its throwing this error:

Error setting up or running Liquibase:
[ERROR] liquibase.exception.MigrationFailedException: Migration failed for change set db/changelog/data/seed_data::filename::author:
[ERROR] Reason: liquibase.exception.DatabaseException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near csv data that its trying to insert

The weird thing is if I change the data that I have in the csv file (i.e. remove some rows of data) for the migration that its complaining about the error is still the same and shows that its still trying to insert the data that I actually removed from the csv file. So its almost like its trying to run a cached version of the file or its finding another version of the file from somewhere else.

I have tried setting relativeToChangelogFile=false and putting in the absolute file path and the script that it was complaining about then runs fine. However, this wouldn’t be any use for production, so its not a suitable solution.

So, I guess my question really is: have there been any changes in Liquibase 4.0.0 that affect how the relative paths work for accessing files for the loadData scripts?

I don’t really understand why Liquibase is finding a file with the same name but different file contents when I try to run with relativeToChangelogFile=true, but I can’t think of any other reason why this issue would be happening.

I saw this other post that seems to suggest a similar issue with relative paths, but there are no answers on that post: https://forum.liquibase.org/t/liquibase-v4-0-0-createprocedure-path-relativetochangelogfile/4627

I’d really appreciate any help anyone can give for this!

Cheers,
Dave

2

Thanks to another forum post (https://forum.liquibase.org/t/includeall-with-v4-0-0/4655 I’ve managed to figure out what’s going on with this issue! Thank you @Joachim!

In the post above Joachim mentioned that there seemed to be a change in 4.0.0 to the way that resources are looked up if they are in a jar file (their issue was with includeAll, which we don’t use in our liquibase setup, but I assumed that it would be a change that affects any resource that the scripts require, including csv files). So based on that I realised that maybe the issue was to do with the fact that changes to csv files don’t automatially trigger a rebuild in Intellij and therefore the csv files in the jar file that liquibase was trying to access were not always the latest versions. I normally run liquibase from within intellij while I’m developing and changes to csv files have always been included when running liquibase in the past.

So basically, to fix this issue all I had to do was to hit the Build button in intellij before running liquibase to make sure that the latest version of the csv files are included in the jar file!

So I can now confirm that liquibase 4.0.0 does actually work for us as long as we change our csv files to include NULL for nullable columns and we build the project in intellij before running in a dev environment.

It would be good if the liquibase developers could provide some details as to exactly how resources are accessed in 4.0.0 and why they made this change, but for now I’m just happy that I finally managed to get 4.0.0 working :smiley:

1 Like
3

Glad, that my post helped to solve your issue.
I´d also be happy if the Liquibase team would be more responsive to the forum posts and provide some insights; especially when a new major version was released…

1 Like
4

Hi @Joachim, thanks for helping our @devdave. We are trying to in the liquibase community to answer questions as quickly as possible, but there is just a few of us. And since you folks have been helping out too, you are a part of that! So many thanks from me (your liquibase advocate) and liquibase in general.

Please feel free to join us on Discord, if we are taking too long or you want to ping in another way, some of the team spends more time here:
https://discord.gg/9yBwMtj

Looking forward to “seeing” ya around.