Duplicate Records in DATABASECHANGELOG

1

I’m currently running a Jenkins Pipeline to run Liquibase updates specified in SQL format. After running the initial update, which consists of 39 changelog files, which each create schemas and tables/views in each, all entries are added into the DATABASECHANGELOG table correctly.

However, whenever I re-run my Jenkins Pipeline to run an update with additional changesets added to 2 of the changelogs, Liquibase creates new records in DATABASECHANGELOG table for every single changeset. Not just the 15 new ones, but also the 1556 ones that were already in place.

For example, one changeset already in place.
SELECT * FROM DATABASECHANGELOG WHERE ID = ‘4_102’;

ID AUTHOR FILENAME DATEEXECUTED ORDEREXECUTED EXECTYPE MD5SUM DESCRIPTION COMMENTS TAG LIQUIBASE CONTEXTS LABELS DEPLOYMENT_ID
4_102 M11X ./…/com/idrc/tst/CMS_DIM_BENE_CD_TST/CMS_DIM_BENE_CD_TST-changelog.sql 2023-04-24 13:42:38.718000 1686 EXECUTED 8:92e2619bc222e0e4730187a5cbee7f1b sql 4.21.1 2358012523
4_102 M11X ./…/com/idrc/tst/CMS_DIM_BENE_CD_TST/CMS_DIM_BENE_CD_TST-changelog.sql 2023-04-24 11:32:50.713000 130 EXECUTED 8:92e2619bc222e0e4730187a5cbee7f1b sql 4.21.1 2350193233

What is causing this? Shouldn’t the author and ID uniquely identify each changeset?

FYI this is the specific changeset from the SQL changelog file.
–changeset M11X:4_102
create or replace TABLE BENE_PTA_PRM_RATE_RDCTN_CD (
BENE_PTA_PRM_RATE_RDCTN_CD VARCHAR(1) NOT NULL,
BENE_PTA_PRM_RATE_RDCTN_DESC VARCHAR(50),
META_SK NUMBER(38,0) NOT NULL DEFAULT 0,
META_SRC_SK NUMBER(38,0) NOT NULL DEFAULT 0,
constraint PK_BENE_PTA_PRM_RATE_RDCTN_CD primary key (BENE_PTA_PRM_RATE_RDCTN_CD) rely
);

Any ideas for troubleshooting this will be helpful. I’ve already removed the DATABASECHANGELOG AND DATABASECHANGELOGLOCK tables, and dropped all schemas managed by Liquibase, and had Liquibase re-create them upon the initial update.

Thanks,
Braz

2

I can also confirm there’s only one Liquibase process running, and the database I’m deploying to is Snowflake.

3

Any ideas? This is preventing completion of a proof of concept for our team, and we would like to use this product for CI/CD integration with Snowflake.

I just ran a liquibase update to create the tables and views specified in the changelogs.

I just ran liquibase update again with modifications to the changelogs, and duplicate records are being created in the DATABASECHANGELOG table. Why is this? There is no lock on the lock table, it was FALSE whenever I started this second update with additional changelogs.

4

My database type is Snowflake version: 7.13.3

The Liquibase CLI I executed the first time:

liquibase update --username=IDRC_TST_LB_SVC --password=**** --url=jdbc:snowflake://cms-idritlab.privatelink.snowflakecomputing.com/?&warehouse=IDRC_NP_DEVOPS_WH_BATCH&db=IDRC_TST&schema=LB_CHANGELOG_TST --changeLogFile=db.change-master.xml

Starting Liquibase at 15:42:41 (version 4.21.1 #9070 built at 2023-04-13 20:56+0000)
Liquibase Version: 4.21.1
Liquibase Open Source 4.21.1 by Liquibase
WARNING:

Liquibase detected the following invalid LIQUIBASE_* environment variables:

  • LIQUIBASE_TST_PASS
  • LIQUIBASE_TST_USER

Find the list of valid environment variables at Liquibase Parameters

Running Changeset: ./…/com/idrc/tst/CMS_AGG_GEO_SPTL_TST/CMS_AGG_GEO_SPTL_TST-changelog.sql::1_1::M11X
Running Changeset: ./…/com/idrc/tst/CMS_AGG_GEO_SPTL_TST/CMS_AGG_GEO_SPTL_TST-changelog.sql::1_2::M11X
Running Changeset: ./…/com/idrc/tst/CMS_AGG_GEO_SPTL_TST/CMS_AGG_GEO_SPTL_TST-changelog.sql::1_3::M11X
Running Changeset: ./…/com/idrc/tst/CMS_AGG_GEO_SPTL_TST/CMS_AGG_GEO_SPTL_TST-changelog.sql::1_4::M11X
Running Changeset: ./…/com/idrc/tst/CMS_AGG_GEO_SPTL_TST/CMS_AGG_GEO_SPTL_TST-changelog.sql::1_5::M11X

Running Changeset: ./…/com/idrc/tst/CMS_VIEW_CLM_TST/CMS_VIEW_CLM_TST-changelog.sql::39_121::M11X
Running Changeset: ./…/com/idrc/tst/CMS_VIEW_CLM_TST/CMS_VIEW_CLM_TST-changelog.sql::39_122::M11X

UPDATE SUMMARY
Run: 1556
Previously run: 0
Filtered out: 0

Total change sets: 1556

Liquibase: Update has been successful.
Liquibase command ‘update’ was executed successfully.

I modified 2 changelogs, and re-ran the same command around 30 minutes later, and it’s executing the same exact changelogs again:

  • liquibase update --username=IDRC_TST_LB_SVC --password=**** --url=jdbc:snowflake://cms-idritlab.privatelink.snowflakecomputing.com/?&warehouse=IDRC_NP_DEVOPS_WH_BATCH&db=IDRC_TST&schema=LB_CHANGELOG_TST --changeLogFile=db.change-master.xml

Starting Liquibase at 16:12:17 (version 4.21.1 #9070 built at 2023-04-13 20:56+0000)
Liquibase Version: 4.21.1
Liquibase Open Source 4.21.1 by Liquibase
WARNING:

Liquibase detected the following invalid LIQUIBASE_* environment variables:

  • LIQUIBASE_TST_PASS
  • LIQUIBASE_TST_USER

Find the list of valid environment variables at Liquibase Parameters

Running Changeset: ./…/com/idrc/tst/CMS_AGG_GEO_SPTL_TST/CMS_AGG_GEO_SPTL_TST-changelog.sql::1_1::M11X
Running Changeset: ./…/com/idrc/tst/CMS_AGG_GEO_SPTL_TST/CMS_AGG_GEO_SPTL_TST-changelog.sql::1_2::M11X

SELECT * FROM DATABASECHANGELOG where id = ‘1_1’;

ID AUTHOR FILENAME DATEEXECUTED ORDEREXECUTED EXECTYPE MD5SUM DESCRIPTION COMMENTS TAG LIQUIBASE CONTEXTS LABELS DEPLOYMENT_ID
1_1 M11X ./…/com/idrc/tst/CMS_AGG_GEO_SPTL_TST/CMS_AGG_GEO_SPTL_TST-changelog.sql 2023-04-25 11:42:53.783000 1 EXECUTED 8:abe007ca95bdfa6527df243b7df21c42 sql 4.21.1 2437371791
1_1 M11X ./…/com/idrc/tst/CMS_AGG_GEO_SPTL_TST/CMS_AGG_GEO_SPTL_TST-changelog.sql 2023-04-25 12:12:38.592000 1557 EXECUTED 8:abe007ca95bdfa6527df243b7df21c42 sql 4.21.1 2439147615
5

hi, I’m getting the same warning “Liquibase detected the following invalid LIQUIBASE_* environment variables:” when I added env variable on our CI.

Would like to know if you’re able to get explanation on this?

Thanks,
Eugene