LiquiBase does not work in non-english default locale

Liquibase Development
1

LiquiBase fails to correctly parse the changelog.xml file on a host with a non-english locale.
Our test host (Windows) is configured with Turkish as the primary language and LiquiBase fails with the following stack trace:

java.lang.RuntimeException: Property not found: initiallyDeferred
        at liquibase.util.ObjectUtil.setProperty(ObjectUtil.java:38)
        at liquibase.parser.xml.XMLChangeLogHandler.setProperty(XMLChangeLogHandler.java:354)
        at liquibase.parser.xml.XMLChangeLogHandler.startElement(XMLChangeLogHandler.java:228)
        at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:501)
        at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:179)
        at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.emptyElement(XMLSchemaValidator.java:719)
        at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:377)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2740)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:645)
        at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:508)
        at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
        at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
        at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
        at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
        at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:522)
        at liquibase.parser.xml.XMLChangeLogParser.parse(XMLChangeLogParser.java:70)
        at liquibase.parser.ChangeLogParser.parse(ChangeLogParser.java:28)
        at liquibase.parser.xml.XMLChangeLogHandler.handleIncludedChangeLog(XMLChangeLogHandler.java:319)
        at liquibase.parser.xml.XMLChangeLogHandler.startElement(XMLChangeLogHandler.java:94)
        at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:501)
        at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:179)
        at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.emptyElement(XMLSchemaValidator.java:719)
        at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:377)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2740)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:645)
        at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:508)
        at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
        at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
        at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
        at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
        at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:522)
        at liquibase.parser.xml.XMLChangeLogParser.parse(XMLChangeLogParser.java:70)
        at liquibase.parser.ChangeLogParser.parse(ChangeLogParser.java:28)
        at liquibase.Liquibase.validate(Liquibase.java:624)

Debugging shows that the methodName calculated in ObjectUtil is “set\u0130nitiallyDeferred”.
This occurs because the String.toUpper() method used to capitalize the “i” of the initiallyDeferred propertyName uses the default locale.
On the test system, the default locale is “tr_TR”, and converting the lowercase “i” to uppercase results in the Unicode character \u0130  (LATIN CAPITAL LETTER I WITH DOT ABOVE)

I was able to work around by setting the default locale to Locale.ENGLISH in the main() of the database migration program.
This was an acceptable work-around in this case, since this program has no user interface.
It would be nice however if the LiquiBase core was not influenced by the default locale.

2

Thanks for finding the problem with the code.  I forced the uppercase method in ObjectUtil to use english locale and that seems to clear up the problem you reported.  My precondition tests are failing, but that may be due to creating non-turkish table names.  Would you be able to test out the latest 2.0 code from http://liquibase.org/ci/latest and let me know if it works for you? If you are normally using 1.9, make sure you run the test against a backup database since the checksums are incompatible.

Nathan

3
Originally posted by: Nathan
Thanks for finding the problem with the code.  I forced the uppercase method in ObjectUtil to use english locale and that seems to clear up the problem you reported.  My precondition tests are failing, but that may be due to creating non-turkish table names.  Would you be able to test out the latest 2.0 code from http://liquibase.org/ci/latest and let me know if it works for you? If you are normally using 1.9, make sure you run the test against a backup database since the checksums are incompatible.

Nathan

Your precondition tests are probably failing for similar reasons.
If you convert English table names from lower to uppercase using the default (Turkish) locale, the table names will not match, since the conversion is not the same as English.    (esp. if you have table names that contain the “i” character.)

It is obvious that ObjectUtil should ALWAYS use Locale.ENGLISH, since the JavaBean methods that match the attribute names are all in English.
It is less obvious what should happen to table names when converting them from lower to upper case or the reverse.

I would suggest that the best way to go would be to add a new optional attribute to the element to allow specification of the locale.
If not specified, the “default” locale could be used, which maintains the current functionality.

For example:
<databaseChangeLog
  xmlns=“http://www.liquibase.org/xml/ns/dbchangelog/1.9”
  xmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance”
  xsi:schemaLocation=“http://www.liquibase.org/xml/ns/dbchangelog/1.9
         http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-1.9.xsd”
  locale=“en_US”

  …

If the locale attribute is not specified, just get the default using Locale.getDefault().
If the locale attribute is specified in the , parse the value (one, two or three arguments, separated by ‘_’ (underscore)) to construct a Locale and push it on a stack structure.  (Since changelogs can be nested.)

While processing the changelog, change all of the calls to String classes that don’t currently use the optional Locale object, [ toUpperCase(), toLowerCase(), etc.] to use the method signature that includes the locale.  When finished processing the changelog, pop the locale off the stack.

Is specifying the locale in the element granular enough?
Does it need to be allowed on the element?

The above method would allow me to specify “en_US” as the locale to use when processing the changelog, w/o forcing me to change the default Locale before invoking liquibase.

I will try to test out the latest 2.0 build sometime today.
I actually use Liquibase to initially populate the schema, as well as maintin it, so no issues with checksums if I start with an empty DB.   ;)

4

The latest build (LQB-DEF-286) does get past the XML parsing.

However, the FirebirdDatabase class changed between version 1.9.5 and 2.0 RC2.
The supportsDDLInTransaction() method in 1.9.5 returned false, and the 2.0 RC2 version returns true.

If looks like the FirebirdDatabase class was modified ın revısıon 1380, with just that single return value changed from false to true.
You may want to reconsider that change.

While it is true that Firebird does support transaction start/commit/rollback of DDL statements, it does not allow an uncommited database object to be used.

This means that on an empty database, the “CREATE TABLE DATABASECHANGELOGLOCK …” must be committed before the “INSERT INTO DATABASECHANGELOGLOCK…” will succeed.
It looks like a commit should occur immediately after the DDL create table databasechangeloglock, but it doesn’t appear to be happening.

I created a MyFirebirdDatabase class that extends FirebirdDatabase, and only overrides supportsDDLInTransaction(), so that it again returns “false”.
Using that class as my Database class allows the changelog to be processed as it was previously in 1.9.5 version.

5

I changed the firebird setting.  I had apparently been given bad info in a bug that firebird supports it.

Nathan