Schema migration and data migration

General Discussion
1

Hi everyone,
I have some questions regarding the intended usage of Liquibase for data migration alongside schema migration.
We have multiple services using Liquibase. They all work a bit different. Some just do schema migration, some do data migration or data generation and some do both.
If I look in the Liquibase documentation I can find the following:

Manage your reference data

  • Deploy test data to QA environments only
  • Deploy data fixes to pre-production and production environments
  • Manage application configuration data

Can schema and data migration changesets be used in a mix?
Are there any best practices regarding data handling with Liquibase?
Should data migration only be triggered after the schema migration is done?
If I have a changelog that contains both schema and data migration with huge amounts of data. is there a way I can ensure that the migration is done in a timely fashion? (background thread doing stuff while application is already operational)

Thanks a lot already for the help.

2

Hi @MartinKr,

Liquibase is designed to handle both schema and data migrations, and it’s common to use them together in your projects. Here are some best practices and answers to your questions:

  1. Mixing schema and data migration changesets: You can certainly mix schema and data migration changesets in a single changelog file. Just make sure to order them correctly, so the schema changes are applied before the data changes that depend on them. Interleaving the data migrations with their schema changes usually works best.

  2. Timely migration for large amounts of data: Liquibase executes changesets in a synchronous manner, meaning it will wait for a changeset to complete before moving on to the next one. Usually this works well. If you run into database timeouts you can consider breaking down large data migration tasks into smaller, more manageable changesets. If you are dealing with a huge amount of data you could use an ETL tool triggered from Liquibase by using executeCommand.

PJ

3

Good morning PJ,

thanks for the reply. That makes things clear.
When using the executeCommand with an external tool, is this also running synchronous (blocking the schema migration)?

Martin