Liquibase 4.22.0 - UpdateCommandStep not committing?

General Discussion
1

Hi, all -

I’m updating to Liquibase 4, and dropping use of the Liquibase class as recommended, and instead using the CommandStep mechanisms described here.

When I run my updates locally, all seems to work, but some of our tests in CI run with our application in a VM and the Postgres DB running in a Docker container hosted by the VM. In this CI environment, Liquibase update summaries appear to say that the updates have been applied, but once our migration is done, nothing has been changed.

Am I missing / misconfiguring an option?

Here’s my code (Kotlin):

class LiquibaseExecutor {
    @Throws(LiquibaseException::class)
    fun update(database: Database, accessor: ResourceAccessor, contexts: String, changeSetFile: String) {
        val scope = mapOf(
            Scope.Attr.database.name to database,
            Scope.Attr.resourceAccessor.name to accessor
        )
        try {
            Scope.child(scope) {
                CommandScope(*UpdateCommandStep.COMMAND_NAME)
                    .addArgumentValue(DbUrlConnectionCommandStep.DATABASE_ARG, database)
                    .addArgumentValue(UpdateCommandStep.CHANGELOG_FILE_ARG, changeSetFile)
                    .addArgumentValue(UpdateCommandStep.CONTEXTS_ARG, contexts)
                    .addArgumentValue(UpdateCommandStep.LABEL_FILTER_ARG, LabelExpression().originalString)
                    .addArgumentValue(ChangeExecListenerCommandStep.CHANGE_EXEC_LISTENER_ARG, null)
                    .addArgumentValue(
                        DatabaseChangelogCommandStep.CHANGELOG_PARAMETERS,
                        ChangeLogParameters(database)
                    )
                    // TODO - put this line back in -- temp testing...
//                    .addArgumentValue(ShowSummaryArgument.SHOW_SUMMARY, UpdateSummaryEnum.OFF)
                    .execute()
            }
        } catch (e: Exception) {
            if (e is LiquibaseException) {
                throw e
            } else {
                throw LiquibaseException(e)
            }
        }
    }
    :
}
2

Hi @scott.straw,

4.22 and 4.23 both contain checksum updates that are still in flux. I would suggest 4.21.1 and see if it works with that.

Do you mind trying 4.21.1?

-PJ

3

Tried that - it wasn’t successful.

Here are some things I discovered, debugging through the problem.

Here is what I know so far:

  • the new way to do things uses a new UpdateCommandStep object for every changelogFile (e.g, "001_.sql", "002_.sql, etc.)
  • However, AbstractUpdateCommandStep has a private static final Map<String, Boolean> upToDateFastCheck which uses to cache whether all the changesets are up-to-date. Note the STATIC in that statement - it lives from 001 to 002 to the end.
  • The key used for the map has only the contexts and the labels (which we don’t use) in it, so it is the same key for 001, 002, etc.
  • When listUnrunChangeSets() is called on AbstractUpdateCommandStep line 189 (with 001), it only checks … the changesets in file 001. And it determines that all of them have been applied.
  • When “update” is called on the rest of the files, upToDateFastCheck is already filled with a “true” for the key used for all of them, so all the rest are just skipped (AbstractUpdateCommandStep:77)

So the update command is working on data taken from the wrong sampling of changesets, and assuming nothing else needs to be applied.

4

Hello @scott.straw - isUpToDateFaskCheck was broken with command refactoring. PR Fix update fast check behavior by MalloD12 · Pull Request #4427 · liquibase/liquibase · GitHub that was merged to master but not released yet aims to fix it. would you be able to try it with master branch build?