Custom datatype with spring boot 2.5 and liquibase 4.3.5

roberthafner · July 11, 2022, 12:50pm

I’m in the process of upgrading one of our services from springboot 2.3.x using liquibase 3.8.9 to spring boot 2.5.x using liquibase 4.3.5. I’m running into an issue in which the custom data type is not being configured when running as a springboot thin jar (inside of a docker container in Kubernetes). However, the custom data type is configured properly when running the service directly from Intellij.

When upgrading to liquibase 4.3.5 I updated the configuration for the custom data type to include the following changes:

Added the following file:
META-INF/services/liquibase.datatype.LiquibaseDatatype

With the contents of the file being:
liquibase.change.FlowableBigIntType
Added the DataTypeInfo annotation to the custom type class. For example:

package liquibase.change;

…

@DataTypeInfo(name=“bigint”, aliases = {“java.sql.Types.BIGINT”, “java.math.BigInteger”, “java.lang.Long”, “integer8”, “bigserial”, “serial8”, “int8”}, minParameters = 0, maxParameters = 1, priority = LiquibaseDataType.PRIORITY_DEFAULT+1)
public class FlowableBigIntType extends BigIntType {

…

}

While troubleshooting, I noticed only the liquibase data types are being found on the following call in DataTypeFactory.java when running inside docker/kubernetes. However, when running from IntelliJ the custom data types are also found.

Scope.getCurrentScope().getServiceLocator().findInstances(LiquibaseDataType.class)

Is there additional configuration needed to have the service find the custom data type when running as a spring boot thin jar (inside docker/kubernetes)?

FWIW, I’ve confirmed that the TomcatEmbeddedWebappClassLoader is being used for the LiquibaseDataType annotation lookup. Which I believe should be the correct class loader.

nvoxland · July 11, 2022, 5:53pm

Adding it to liquibase.datatype.LiquibaseDataType should be all you need to do.

We end up relying on the regular java.util.ServiceLoader logic. Is tehre something different in how the classpath gets set up in docker? Maybe missing classes or some libraries not being bundled the same way?

Nathan

roberthafner · July 11, 2022, 10:09pm

I created a custom ServiceLocator and added a META-INF/services/liquibase.servicelocator.ServiceLocator file to help with troubleshooting. The custom locator prints out the class loader hiearchy as well as whether or not the custom data type can be loaded. The log output shows the custom data type class is loaded successfully by the LaunchedURLClassLoader.

public class WorkflowStandardServiceLocator implements ServiceLocator {
private static final Logger LOGGER = LoggerFactory.getLogger(WorkflowStandardServiceLocator.class);
private StandardServiceLocator standardServiceLocator = new StandardServiceLocator();

@Override
public int getPriority() {
    return PRIORITY_SPECIALIZED;
}

@Override
public <T> List<T> findInstances(Class<T> interfaceType) throws ServiceNotFoundException {
    if (interfaceType == LiquibaseDataType.class) {
        LOGGER.error("findInstances() >>");
        System.out.println("findInstances() >>");
        List<T> allInstances = new ArrayList<>();

        //final Logger log = Scope.getCurrentScope().getLog(getClass());
        ClassLoader classLoader = Scope.getCurrentScope().getClassLoader(true);
        ClassLoader cl = classLoader;
        StringBuilder sb = new StringBuilder();
        for (int i= 0; cl != null; i++) {
            sb.append(i + ":" + cl.toString() +"\n");
            cl = cl.getParent();
        }
        LOGGER.error(sb.toString());

        String className = "liquibase.change.FlowableBlobType";
        try {
            Class clz = classLoader.loadClass(className);
            LOGGER.error("Class " + className + " found on classloader " + clz.getClassLoader().toString());
        } catch(Exception cnfe) {
            LOGGER.error("Class " + className + "not found!");
        }

        final Iterator<T> services = ServiceLoader.load(interfaceType, Scope.getCurrentScope().getClassLoader(true)).iterator();
        while (services.hasNext()) {
            try {
                final T service = services.next();
                LOGGER.error("Loaded " + interfaceType.getName() + " instance " + service.getClass().getName());
                System.out.println("Loaded " + interfaceType.getName() + " instance " + service.getClass().getName());
                allInstances.add(service);
            } catch (Throwable e) {
                LOGGER.error("Cannot load service: " + e.getMessage());
                System.out.println("Cannot load service: " + e.getMessage());
            }
        }

        LOGGER.error("found " + allInstances.size() + " instances");
        System.out.println("found " + allInstances.size() + " instances");
        LOGGER.error("findInstances() <<");
        System.out.println("findInstances() <<");
        return Collections.unmodifiableList(allInstances);
    } else {
        return standardServiceLocator.findInstances(interfaceType);
    }
}

}

Log output:
ERROR 2022-07-11 21:49:19.897 +0000 [-workflow] - findInstances() >>
findInstances() >>
ERROR 2022-07-11 21:49:19.898 +0000 [-workflow] - 0:TomcatEmbeddedWebappClassLoader
context: workflow
delegate: true
----------> Parent Classloader:
org.springframework.boot.loader.LaunchedURLClassLoader@c39f790

1:org.springframework.boot.loader.LaunchedURLClassLoader@c39f790
2:jdk.internal.loader.ClassLoaders$AppClassLoader@49476842
3:jdk.internal.loader.ClassLoaders$PlatformClassLoader@56620197

ERROR 2022-07-11 21:49:19.900 +0000 [-workflow] - Class liquibase.change.FlowableBlobType found on classloader org.springframework.boot.loader.LaunchedURLClassLoader@c39f790
ERROR 2022-07-11 21:49:19.905 +0000 [-workflow] - Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.BigIntType
Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.BigIntType
ERROR 2022-07-11 21:49:19.906 +0000 [-workflow] - Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.BlobType
Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.BlobType
ERROR 2022-07-11 21:49:19.908 +0000 [-workflow] - Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.BooleanType
Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.BooleanType
ERROR 2022-07-11 21:49:19.909 +0000 [-workflow] - Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.CharType
Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.CharType
ERROR 2022-07-11 21:49:19.910 +0000 [-workflow] - Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.ClobType
Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.ClobType
ERROR 2022-07-11 21:49:19.919 +0000 [-workflow] - Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.CurrencyType
Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.CurrencyType
ERROR 2022-07-11 21:49:19.921 +0000 [-workflow] - Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.DatabaseFunctionType
Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.DatabaseFunctionType
ERROR 2022-07-11 21:49:19.923 +0000 [-workflow] - Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.DateTimeType
Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.DateTimeType
ERROR 2022-07-11 21:49:19.924 +0000 [-workflow] - Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.DateType
Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.DateType
ERROR 2022-07-11 21:49:19.925 +0000 [-workflow] - Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.DecimalType
Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.DecimalType
ERROR 2022-07-11 21:49:19.927 +0000 [-workflow] - Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.DoubleType
Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.DoubleType
ERROR 2022-07-11 21:49:19.928 +0000 [-workflow] - Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.FloatType
Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.FloatType
ERROR 2022-07-11 21:49:19.929 +0000 [-workflow] - Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.IntType
Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.IntType
ERROR 2022-07-11 21:49:19.930 +0000 [-workflow] - Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.MediumIntType
Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.MediumIntType
ERROR 2022-07-11 21:49:19.931 +0000 [-workflow] - Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.NCharType
Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.NCharType
ERROR 2022-07-11 21:49:19.933 +0000 [-workflow] - Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.NVarcharType
Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.NVarcharType
ERROR 2022-07-11 21:49:19.934 +0000 [-workflow] - Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.NumberType
Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.NumberType
ERROR 2022-07-11 21:49:19.935 +0000 [-workflow] - Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.SmallIntType
Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.SmallIntType
ERROR 2022-07-11 21:49:19.937 +0000 [-workflow] - Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.TimeType
Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.TimeType
ERROR 2022-07-11 21:49:19.941 +0000 [-workflow] - Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.TimestampType
Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.TimestampType
ERROR 2022-07-11 21:49:19.942 +0000 [-workflow] - Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.TinyIntType
Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.TinyIntType
ERROR 2022-07-11 21:49:19.943 +0000 [-workflow] - Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.UUIDType
Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.UUIDType
ERROR 2022-07-11 21:49:19.944 +0000 [-workflow] - Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.UnknownType
Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.UnknownType
ERROR 2022-07-11 21:49:19.945 +0000 [-workflow] - Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.VarcharType
Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.VarcharType
ERROR 2022-07-11 21:49:19.946 +0000 [-workflow] - Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.XMLType
Loaded liquibase.datatype.LiquibaseDataType instance liquibase.datatype.core.XMLType
ERROR 2022-07-11 21:49:19.947 +0000 [-workflow] - found 25 instances
found 25 instances
ERROR 2022-07-11 21:49:19.947 +0000 [sas-workflow] - findInstances() <<
findInstances() <<

roberthafner · July 11, 2022, 10:24pm

As another point of reference, I updated the custom locator to log the class loader that a core liquibase data type is loaded from and it is using the same class loader.

ERROR 2022-07-11 22:20:40.475 +0000 [-workflow] - Class liquibase.datatype.core.BlobType found on classloader org.springframework.boot.loader.LaunchedURLClassLoader@c39f790

roberthafner · July 12, 2022, 3:46pm

This issue appears to be spring-boot related as I can reproduce the issue outside of docker/kubernetes by running the service as (fat) jar via the following command.

java -jar ./build/libs/workflow-service.jar

roberthafner · July 12, 2022, 8:12pm

Finally tracked the root cause of this issue down. It turns out I had a typo in the name of the META-INF/services file. Once I corrected that everything worked as expected.

Had:
META-INF/services/liquibase.datatype.LiquibaseDatatype

Should be:
META-INF/services/liquibase.datatype.LiquibaseDataType

What’s interesting though is the failure doesn’t occur (with the typo included) when running the service from the IDE but does fail when running the serving as a spring boot fat jar. I suspect the difference may have to do with the file system on a MacBook being case-insensitive and files being case sensitive inside spring boot’s fat jar.

Topic		Replies	Views
Custom LiquibaseDataTypes not found in docker image classpath Liquibase Development	1	654	June 13, 2022
Adding new datatype via the ServiceLocator Liquibase Development	3	383	September 22, 2009
Extends NumberType not working with version 4.4.0 Liquibase Development	3	511	July 1, 2021
Custom Type Converter? General Discussion	18	2466	June 23, 2010
Liquibase-Package based classes not recognized with 3.5.0-SNAPSHOT jar Liquibase Development	3	386	April 7, 2016

Custom datatype with spring boot 2.5 and liquibase 4.3.5

Related topics