I’m currently working on a Liquibase-based CI/CD pipeline, and I have a question regarding rollback safety and best practices in real production environments.
Imagine the following situation:
A changeset creates a table and inserts a row.
A rollback is defined (e.g., DELETE and DROP TABLE).
The update runs successfully.
But when executing the rollback, it fails due to a logic or constraint error.
At this point, part of the change is already applied, and the rollback can’t clean it up automatically.
My questions for the community:
How do you normally deal with this in real life?
Do teams manually clean up the changes?
Are there structured rollback playbooks?
Is there any way to validate that a rollback actually works before applying the change in production?
I already use future-rollback-sql, but it only checks for syntax and presence—not actual execution success.
Have you found ways to automatically test update + rollback together safely as part of a pipeline?
I’ve tried wrapping Liquibase in a custom Java tool that uses conn.setAutoCommit(false) to simulate atomicity, but I know this isn’t Liquibase’s default behavior.
Is there a better pattern for testing rollback validity when the changeset contains both DDL and DML?
My goal is to make sure that if a rollback is defined, it truly works, and if it doesn’t, the changeset never gets applied in the first place.
Any guidance, real-world strategies, or patterns you’ve found helpful would be greatly appreciated!
If a database ends up in the state you describe, manual intervention is necessary. You’ll need to clean up the partially deployed or rolled back change. You might also need to update the DATABASECHANGELOG table that tracks changeset execution to remove the entry for the changeset that didn’t rollback cleanly.
Is there any way to validate that a rollback actually works before applying the change in production?
There are a couple of ways to do this and what’s right for you depends heavily on what’s available in your pipeline. A lot of users will test update and rollback in a pre-prod database that is frequently refreshed from production. Other users will use containerized databases in their workflows to test the update and rollback they are introducing. We recommend that update and rollback not be tested in prod because…well…nothing should really be tested in prod. If you have replicated databases in prod you could conceivably work out a blue/green deployment to test everything if you are able to remove a database from the replication cluster or temporarily pause replication to protect your data through isolation.
Have you found ways to automatically test update + rollback together safely as part of a pipeline?
There is the update-testing-rollback command that performs an update, performs the rollback for that update, and then performs the update again to make sure that the rollback sufficiently undoes the changes. I’ve also seen users build a CI workflow with individual commands to accomplish a more thorough version of this: taking a pre-update snapshot; run the update; run the rollback; compare the current state of the database to the snapshot taken in step 1. If there are no differences detected, you can be reasonably comfortable that the rollback did what it was supposed to.
Thanks for the insights! This is actually perfect timing — I’m in the process of setting up Liquibase for the first time in our environment, and we’re getting close to moving it into production.
I’m still evaluating whether Liquibase Pro would make a big difference for us. Since this is our first implementation, I’m trying to be cautious and anticipate potential issues, especially around rollbacks and how to integrate safely with our pipelines (we use Azure DevOps, PostgreSQL on AWS, and a multi-environment setup).
Given your experience, I’d love to hear:
Are there any lessons or “I wish I had known this before” moments you could share from when you first started using Liquibase in production? Any best practices or pitfalls to avoid?
Really appreciate your guidance — I want to build this right from the start.
In no particular order, these are the things that I consider when devising a rollback strategy:
If you are able to, fix forward instead of rolling back. When you adopt a fix forward mindset you are making changes based on the specifics of the unexpected scenario you find yourself in. With a rollback strategy, you have to guess or anticipate which problems you might have and craft a rollback that is general enough to address all of them. This is hard to do and not fool proof. Fix forward allows you to make the most minimal change possible to address the issue.
Figure out a way to use temporary or ephemeral databases for deploy testing. This is safe and usually not that hard to do if your org is cool with containerization or fleeting cloud resources in CICD.
If you have to roll back, consider writing your own instead of relying on automated rollbacks. The automated rollbacks are effective but can be brutish. Create a table or column? Automatic rollback is going to drop it along with any data that may have been written between the time the changeset was executed and the time of the rollback. Crafting rollbacks specific to the change you are making with the context you have enables you to do things like renaming tables and columns instead of dropping them to retain data.
You’re already thinking about this one, but test your rollbacks as early as possible and include peer review in your checklist for pull/merge requests. It’s easier to fix these up front than when your rollback fails in prod!
