Custom Type Converter?

General Discussion
1

I am working with 2.0-rc2-SNAPSHOT (the code, I did a maven install locally to work with the libraries).  I checked the code out of trunk today, June 23rd.

I would like to be able to do custom type conversion on column types.

We support multiple databases, and each database (as you know) has different ways of dealing with Unicode.  An nvarchar column in SQL Server is a varchar(3x precision) in an Oracle database with Unicode enabled.

I want to be able to specify java.sql.Types.NVARCHAR and, if the dbms=oracle, make the column’s data type VARCHAR2(3xprecision).

I tried creating a custom OracleTypeConverter type, but it wasn’t picked up.  Is there a better way to do this?

2

Couldn’t you just use a different context for different rdbms?

3

Yes, in theory I could (if I understand what you are saying), but given the sheer size of our schemas, and that we support many different dbms, I wouldn’t want developers to track schema changes for every dbms we support, or else, what’s the point of using liquibase?

4

You should be able to provide your own type converter. You should be able to create your class in a sub-package of liquibase.ext and override the getPriority() method to return a value higher than 5 (PRIORITY_DEFAULT). 

Have you tried that?

Nathan

5

Nathan,
I just tried this and it still is not working.

I have a database project (I am using maven and eclipse), and created a CustomOracleTypeConverter class that extends OracleTypeConverter.  I put it in liquibase.ext pacakage.

It is pretty straightforward, the code is below.  Is there another step I need to take to register the Type Converter?  It seems to be hitting the OracleTypeConverter, but not my custom type at all.

package liquibase.ext;

import liquibase.database.Database;
import liquibase.database.structure.type.DataType;
import liquibase.database.structure.type.VarcharType;
import liquibase.database.typeconversion.core.OracleTypeConverter;

public class CustomOracleTypeConverter extends OracleTypeConverter {
 

  @Override
  public int getPriority() {
      return 4;
  }

  @Override
  protected DataType getDataType(String columnTypeString, Boolean autoIncrement, String dataTypeName, String precision) {
 

      System.err.println( “\r\n\r\n\r\n!!!\r\n\r\n\r\n\r\n”);
      DataType returnType = null;

      // Convert NVARCHAR to VARCHAR
      if( columnTypeString.startsWith( “java.sql.Types” ) && precision != null ) {
        if( dataTypeName.equalsIgnoreCase( “NVARCHAR” ) ) {
            returnType = new VarcharType( “VARCHAR2” );
           
            // Do a 3x precision
            Integer precisionValue = Integer.parseInt( precision ) * 3;
            precision = precisionValue.toString();
            addPrecisionToType( precision, returnType );
            return returnType;
        }
      }
      return super.getDataType( columnTypeString, autoIncrement, dataTypeName, precision );
  }
 

  @Override
  public boolean supports( Database database ) {
 

      System.err.println( “\r\n\r\n\r\n!!!\r\n\r\n\r\n\r\n”);
      return super.supports( database );
  }
}

6

Your priority should be higher than liquibase.servicelocator.PrioritizedService.PRIORITY_DATABASE. This is not the case, so your implementation will always ne skipped.

Try this:

    public int getPriority() {   return PRIORITY_DATABASE + 1; }
7

Thanks for the quick response.  This still does not work.

Keep in mind, I am running this through the liquibase maven plugin.

I added a static initializer to my type converter, and it never gets fired.

Here is the exception:
Failed to execute goal org.liquibase:liquibase-maven:2.0-rc2-SNAPSHOT:update (default) on project pas.database: Execution default of goal org.liquibase:liquibase-maven:2.0-rc2-SNAPSHOT:update failed: Could not determine NVARCHAR for liquibase.database.typeconversion.core.OracleTypeConverter -> [Help 1]

Here is my updated class…
public class CustomOracleTypeConverter extends OracleTypeConverter {
 

  static {
     System.err.println( “\r\n\r\n\r\n!!!\r\n\r\n\r\n\r\n”);      
  }
 

  @Override
  public int getPriority() {
     return PRIORITY_DATABASE + 1;
  }

  @Override
  protected DataType getDataType(String columnTypeString, Boolean autoIncrement, String dataTypeName, String precision) {
 

     System.err.println( “\r\n\r\n\r\n!!!\r\n\r\n\r\n\r\n”);
     DataType returnType = null;

     // Convert NVARCHAR to VARCHAR
     if( columnTypeString.startsWith( “java.sql.Types” ) && precision != null ) {
        if( dataTypeName.equalsIgnoreCase( “NVARCHAR” ) ) {
           returnType = new VarcharType( “VARCHAR2” );
           
           // Do a 3x precision
           Integer precisionValue = Integer.parseInt( precision ) * 3;
           precision = precisionValue.toString();
           addPrecisionToType( precision, returnType );
           return returnType;
        }
     }
     return super.getDataType( columnTypeString, autoIncrement, dataTypeName, precision );
  }
 

  @Override
  public boolean supports( Database database ) {
 

     System.err.println( “\r\n\r\n\r\n!!!\r\n\r\n\r\n\r\n”);
     return super.supports( database );
  }
}

HERE is the POM snippet that I am using:

org.liquibase
liquibase-maven
2.0-rc2-SNAPSHOT


package

sqlservertest.xml
oracle.jdbc.OracleDriver
${local.database.url}
lb_test
lb_test
false
true
true
debug
true


update





com.oracle
ojdbc14
10.2.0.4


8

Gotta bump this up.

Does anyone have an example about how to introduce a custom type converter?  I still can’t get this to work, and need some advice as to where I should look.

Thanks

9

I was able to test it out with a Type converer that looked like this:

    package liquibase.ext.customtype;

    import liquibase.database.Database;
    import liquibase.database.core.MySQLDatabase;
    import liquibase.database.structure.type.DataType;
    import liquibase.database.structure.type.TextType;
    import liquibase.database.typeconversion.core.MySQLTypeConverter;

    public class MyTypeConverter extends MySQLTypeConverter {
        @Override
        public int getPriority() {
            return 100;
        }

        @Override
        public boolean supports(Database database) {
            return database instanceof MySQLDatabase;
        }

        @Override
        public DataType getDataType(String columnTypeString, Boolean autoIncrement) {
            if (columnTypeString.toLowerCase().startsWith(“faketype(”)) {
                return new TextType(columnTypeString.replaceFirst(“faketype”, “varchar”), 1,1);
            }
           
            return super.getDataType(columnTypeString, autoIncrement);
        }
    }

It was picked up correctly by liquibase without any additional configuration.  I tested it out by creating a changeset like:

                                       

which fails unless the type converter handles it. 

how recent is your RC2 snapshot?

Nathan

10

Thanks Nathan for the sample.  I am going to update my snapshot and try again.  I have been sidetracked with some other priorities.

Keep in mind, I am using maven for my build, and my custom type resides in the same maven module as the changesets, not sure if that is different or not.

I’ll post a reply once I try this out.

11

Strike that, I still cannot get my converter to get hit, it is just “printing out” the type that is specified, that is why the SQL is mangled in the response below.

I also want to stress that I am trying to run this through MAVEN

Here is my original response…

I believe I found the issue, I cannot start my custom type with java.sql.Types and OVERRIDE the existing functionality.  I changed my prefix to custom.sql.Types and it appears as though my Converter was hit.

However, with the sample and my customer converter, it doesn’t completely work.

The CREATE TABLE statement that is generated is mangled, look at the create table SQL, it is trying to create a column of faketype(10)…

    Reason: liquibase.exception.DatabaseException: Error executing SQL CREATE TABLE test6 (id faketype(10)):
         Caused By: Error executing SQL CREATE TABLE test6 (id faketype(10)):
         Caused By: ORA-00907: missing right parenthesis

12

I believe this is a class loading issue.  How can I get my custom type converter to be recognized by liquibase?  I managed to turn on debug output, and the ServiceLocator never “finds” my class.

13

I think I found a bug with the class loading that takes place.

When liquibase maven plugin first fires, it appears to load my jar into the class loader, exhibited by the following output which says it has my.database jar file:
[INFO]  artifact: file:/C:/Documents%20and%20Settings/paulc/.m2/repository/junit/junit/4.4/junit-4.4.jar
[INFO]  artifact: file:/C:/Documents%20and%20Settings/paulc/.m2/repository/com/oracle/ojdbc14/10.2.0.4/ojdbc14-10.2.0.4.jar
[INFO]  artifact: file:/C:/Documents%20and%20Settings/paulc/.m2/repository/org/liquibase/liquibase/2.0-rc2-SNAPSHOT/liquibase-2.0-rc2-SNAPSHOT.jar
[INFO]  artifact: file:/C:/development/workspaces/subversion/liquibase-demo/my.database/target/my.database-9.4.0.0.0-SNAPSHOT.jar
[INFO]  artifact: file:/C:/development/workspaces/subversion/liquibase-demo/my.database/target/test-classes

However, I modified the ServiceLocator to print out the resources in the ClassLoaderResourceAccessor, and it spit out the following:
liquibase.resource.ClassLoaderResourceAccessor(file:/C:/Documents%20and%20Settings/paulc/.m2/repository/org/liquibase/liquibase-maven/2.0-rc2-SNAPSHOT/liquibase-maven-2.0-rc2-SNAPSHOT.jar,file:/C:/Documents%20and%20Settings/paulc/.m2/repository/com/oracle/ojdbc14/10.2.0.4/ojdbc14-10.2.0.4.jar,file:/C:/Documents%20and%20Settings/paulc/.m2/repository/org/codehaus/plexus/plexus-utils/1.0.4/plexus-utils-1.0.4.jar,file:/C:/Documents%20and%20Settings/paulc/.m2/repository/junit/junit/3.8.1/junit-3.8.1.jar,file:/C:/Documents%20and%20Settings/paulc/.m2/repository/org/liquibase/liquibase/2.0-rc2-SNAPSHOT/liquibase-2.0-rc2-SNAPSHOT.jar)

As you can tell, my jar is not on there!  I don’t know why.

However, I added a dependency in my pom to my customer jar like so:

com.sample
my.database
9.4.0.0.0-SNAPSHOT

And, BINGO, all of a sudden it was found.  Of course, it assumes that you have run a mvn clean install on the jar to have it available first, but this appears to work.

I’ll re-iterate, it appears to be a bug, as I am not sure why it says it has it at first, but then it is lost.

14

Glad you found a way to get it to work.  I’m not sure how classpaths end up working in maven.  I’ll have to look into it more (http://liquibase.jira.com/browse/CORE-651). 

Nathan

15
Originally posted by: pcleary00
I believe this is a class loading issue.  How can I get my custom type converter to be recognized by liquibase?  I managed to turn on debug output, and the ServiceLocator never "finds" my class.

I can’t really tell from your pom file, but bear in mind the jar file that contains your type converter needs to be placed in the section of your .  I hope this helps.

Best,
Laird

16

I haven’t time to test.
But I think that it should work also without the if the plugin is run with the project that has the custom typeconverter code (this is the way that the maven integration test works).
If the code is in another module then dependency is absolutely needed.

17

Yes, you are correct, the class files that are generated by maven should be available on the classpath by default, I should not have to add the dependency manually to the plugin since the files are in the same project that maven is building.  While not a huge issue, they should be able to be found.

18

Hi I would like to write a custom Type Converter as well. It would be helpful to add a register method to TypeConverterFactory so I do not have to  use a liquibase.ext subpackage.

19

Yes, it should, I added the method.  Thanks for the tip.

Nathan