Spring Autowiring in CustomChange classes

General Discussion
1

I was wondering if there were any plans to enable Spring support inside CustomChange classes?
Specifically the ability to use @Autowire

2

Hi @sergek ,

Welcome! Would love to help but I am not a Spring expert, could you walk me through what you specifically want Liquibase to do?

What is the SQL or changeset you wish to run?

Thanks,

Ronak

3

@ronak thanks for following up.

The majority of our migrations have some form of application logic being executed as part of a database update. For example, I need to add a column for a new entity attribute. Adding the column is one of many parts of the process. We also need to auto-populate it using some business rules and populate a number of other, associated tables with aggregated data that’s based on the new attribute.
Dependency injection of the rules engine and other components is currently done through Spring service “autowiring” in our in-house migrations framework.
I am in process of evaluating liquibase and flyway as potential replacements for our in-house system.
I’ve tried using @Autowired annotation in CustomChange classes and it doesn’t work, even when I used SpringLiquibase method of triggering update.
I was able to get this work in FlyWay. They also have CustomChange changetype (referred to as Java migrations) and from my digging through their source code, it looks like they instantiate customchange classes and wrap them in Spring context, thus enabling @Autowiring support.
I would like to know if there are any plans to introduce similar functionality in LB

4

Hey @sergek,

I am talking to the community support team. Hang tight, we’ll respond shortly.

5

We don’t currently have the internal spring integration we’d need to support the autowiring. Since you have found the spot in the flyway code that does that wrapping, feel free to send it along so we can look at adding that in as well.

In the mean time, even though Autowiring doesn’t work, you are able to access the spring context though static methods and lookup your objects on there. You could also write a single, generic CustomChange implementation that takes the bean name etc. as parameters and do the bean lookup in that generic CustomChange class. It’s a bit of a hack, but would allow you to keep your spring beans as they are and easily use them in liquibase, once you write that wrapper class.

1 Like
6

Thanks for the follow-up!
Since I’ve completed the initial review, I am done with first stage of the proof of concept.
As soon as I have resources for 2nd stage POC, I will try to create the wrapper class and send it over.
While on the topic of FW vs LB, are there any plans to make LB’s java interface to be as feature rich/dev-friendly as FW’s?

7

We are always looking to improve how people use Liquibase. Which parts of the java interface are you talking about in particular?

Nathan

8

@nvoxland let me share with you the details of my stage-1 proof of concept.
Requirements:

  1. Support for 2 datasources. We have core db and userdata db
  2. Support for customer-specific migrations
  3. Support for migration modularization: we have a dozen or so core maven modules plus customer modules. Migrations specific to a business module must reside in that module
  4. Support for java-based migrations. Currently we are at 450 active migrations (anything older than 2016 was deprecated) and at least 70% of these migrations are “java migrations”. Java components are either loaded using Spring Framework’s @Autowired annotation or wrapper for Spring Loader using .get() methods. Anything loaded using .get() is typically refactored to @Autowired whenever there’s free time.
  5. Migration execution order must be static. It should be easy to predict. It should try to minimize errors due to merge conflicts from long-running branches / commits. It should have an easy format and minimize chances of errors due to typos
  6. Allow to define migration version and description strings and have audit log persisted somewhere
  7. Be able to bootstrap schema for JUnit unit tests
  8. Support MS SQL Server
  9. Be able to generate schema diff based on JPA/Hibernate entity class changes
  10. Allow for ad-hoc migration triggering / be able to execute migrations while the application is running
  11. Have an easy way to switch existing customer environment from in-house migrator to 3rd-party migrator. Preferably to be able to it as an automatic migration that wouldn’t require additional work by dev when a new system is deployed to customer environment
  12. Try to minimize the duplication of configuration for datasources: either get DB connection info from already-defined Spring Beans or be able to extract it from application.yaml
  13. Have good documentation
  14. JDK 11 support
  15. Nice to have: support for other datasources we might want to explore, like H2, Mongo, ClickHouse, Redis
  16. Nice to have: easy way to migrate between datasources

Liquibase vs FlyWay pros:

  • Has generateChangeLog command. Almost works, but our solution contains both @Entity-annotated classes and legacy entities that just use .hbm.xml to define schema / attach to Hibernate. It seems that Liquibase Hibernate extension can either generate schema based on @Entity annotations or it expects pre-Spring Boot Hibernate and expects hibernate.cfg.xml to parse .hbm.xml entities. As such, we can’t rely on this until all entities are annotated with @Entity
  • Changeset DSL should help minimize bugs due to typos/sql syntax errors and should allow us to switch between datasources easily (I hope…)

FlyWay vs Liquibase pros:

  • Migrator has an easy-to-use Java API that supports (as far as I know) all migrator commands: API: Javadoc - Flyway by Redgate • Database Migrations Made Easy.
  • Java migrations support Spring context (autowiring)
  • Documentation seems to be a bit more straight-forward but that could be because there’s no FlyWay DSL, thus less functionality to document

Liquibase vs Flyway tie:

  • Liquibase out-of-the-box has a master changelog that can be populated with include references to changesets, thus ensuring a static migration order. FlyWay allows to overwrite MigrationResolver and I can write a small piece of logic that will let me support parsing the equivalent of LB’s master changelog with references to changesets

Everything else on my list is covered more-or-less equally by both libraries

POC Implementation in Liquibase: Liquibase POC - Pastebin.com
POC Implementation in FlyWay: FlyWay poc - Pastebin.com

After learning that Liquibase should be able to generate schema diffs, most of my time was spent on trying to cajole Liquibase Hibernate extension to create diffs and gave up after running out of time, so I didn’t implement a number of features in my FlyWay POC, like handling contexts and MigrationResolver override.
I’ve worked with liquibase in the past, I am quite fond of it and I really wanted to make it work for the current POC.
However, it seems to me that Spring/Spring Boot support is not very high on the priority list.
The LiquibaseHiberate extension is also disappointing by its lack of flexibility - I understand that we have a mix of legacy and modern entities, but if Hibernate can parse this soup and produce a valid schema, how come the extension (which relies on Hibernate schema parser) can’t?

1 Like
9

Thanks for the writeup, we always love hearing where we can improve. Your comparison is generally pretty fair and correct. We are working on improving our docs. We have an expanded doc team now, but (like you say) there is a lot to document so it takes a while to work through.

For the java API to run Liquibase, we have a similar facade to Flyway. While it supports the main operations (update, rollback, etc.) it doesn’t include everything Liquibase can do because the single facade class ended up too limiting as our functionality increased. Our active project right now is finishing up the new “Command” layer that will provide an easy and complete API to running Liquibase.

It’s been a bit since I’ve worked with the hibernate extension to know what it would take to support the mixed configuration options. I’ve tended to rely on the community for that extension since I don’t use hibernate a lot, so if you do figure it out feel free to send a PR. Since you are in spring boot with a running hibernate configuration, there may even be a way to pass the pre-created hibernate metadata into the HibernteDatabase class rather than relying on it to (re)configure itself?

The improved Spring support is on the list of things to improve, but there is higher priority items we are working on first (like the Command API).

Nathan

2 Likes
10

@nvoxland thanks for the follow-up!
I will keep an eye out for Command API and will revisit Spring’ing the LB customtask migrators the next time I have the opportunity to revisit this project.

As for Hibernate, as I mentioned before, I spent the majority of my available time fiddling with the extension’s source in hopes of having it parse all objects. At first I tried to fix the HibernateSpringPackageDatabase but then I felt that I might be better off create a brand new class that’s based on low-level HibernateDatabase class and starting from there, porting in features from both JpaPersistenceDatabase and HibernateClassicDatabase.
Since I didn’t have time to basically write a new custom component from scratch, I shelved the project.
I think in the future, when I’ll have more time to spend on this initiative, I might start with cleaning up our code and annotate all DB objects with @Entity annotation and go from there.

If you take a look at the attached snippets, you can see the difference between spinning up and using LB migrator vs FW migrator. I hope you can agree that LB is not particularly… clean? I am hoping that Command Api you are mentioning will make that stage cleaner

Thanks again for your time

11

Circling back to the original question of enabling Spring support in CustomChange classes. I do think there is a way to shim this into the current architecture and get @Autowired support as requested. I quickly whipped up a snippet of where I would put the support in. We can extend the CustomChangeWrapper and override the setClass(String) method. Then you can use the AutowireCapableBeanFactory (via the ApplicationContext) to create classes and it will inject anything set with @Autowired. I’ve used this pattern with great success at my workplace in other areas of code where injecting into dynamically created classes is needed. Here is the snippet:

public class BeanFactoryLoadingCustomChangeWrapper extends CustomChangeWrapper {
    private final ApplicationContext applicationContext;

    public BeanFactoryLoadingCustomChangeWrapper(ApplicationContext applicationContext) {
        this.applicationContext = applicationContext;
    }

    @Override
    public CustomChangeWrapper setClass(String className) throws CustomChangeException {
        try {
            AutowireCapableBeanFactory beanFactory = applicationContext.getAutowireCapableBeanFactory();
            Class<?> customChangeClass = ClassUtils.forName(className, Scope.getCurrentScope().getClassLoader());
            return (CustomChangeWrapper) beanFactory.createBean(customChangeClass);
        } catch (ClassNotFoundException e) {
            throw new CustomChangeException(e);
        }
    }
}

I’m happy to put in a pull request for this support but I’m still trying to figure out how to let Liquibase know about this new implementation of the CustomChangeWrapper and to make sure it is only available if the user is using Spring. I could use some help on that.

Let me know what you think @nvoxland and @ronak.

2 Likes
12

Hey @MattBertolini , thanks so much for responding. I really do think we should help on the developer side of the community. @MikeOlivas, would you mind advising @MattBertolini on:

13

@ronak What happened to @MattBertolini code? I didn’t find a pull request or any docs for this feature.

14

No code was submitted. I never figured out how to make sure it was only available if the user was using Spring. It may make sense to make it a separate jar instead. That would get rid of the issue I was seeing as you would only include the jar if you wanted to use it with Spring.

15

Hi there,
just faced the same issue as I was looking for a way to apply code-based migrations. And since we are using liquibase already it would be wasteful to integrate an additional mechanism…
I did try the solution proposed at java - Use other spring beans in liquibase CustomTaskChange class - Stack Overflow and can confirm it works.
Would be nice to have some clear way of overriding SpringLiquibase bean. For example, LiquibaseConfiguration.createSpringLiquibase might be protected so one who needs custom features such as spring bean awareness could just override this method instead copy-pasting almost the whole class.