I have a SQL format changelogs file with more than 1 million data records, the size of file is more than 200MB, close to 300MB.
I used Liquibase Maven plugin to load those data into one database, it will take more than half an hour, which is too long for me.
I set the verbose to false, and turn off the logging. It doesn’t improve much.
Is there anything else I can do to improve the performance of loading large amount of data using Liquibase Maven plugin?
I have a dump mysql file.
Will use Mysql function to restore the dumped file first (instead of using Liquibase to load the dumped file), and then use Liquibase just maintain the changes on top of those large amount of data.
This solution should work, thanks a lot!
The other method to load data is from a CSV file. But, all that will do is generate SQL like you probably already have. https://www.liquibase.org/documentation/changes/load_data.html. So, loading the data at every Maven runs is probably not the best way.
Maybe have a database dump with the data already in it. Liquibase will then just update the schema based on the version in DATABASECHANGELOG table. You can then restore the database from the dump prior to the Maven execution.
Let me know if that works.
