Fails to create DB objects having specific symbols like Yen, plus sign and Japanese character's on the table

Liquibase Development
1

Fails to create DB objects having specific symbols like Yen, plus sign and Japanese character’s on the table.

“\uffe5” (65509) (Yen symbol)
“\uff0b” (65291) (Plus sign)
“\uff1c” (65308) (Less than symbol)
“\uff1e” (65310) (Greater than symbol)
“\uff5e” (65374) (Tilda)

We also have Japanese Kanji and Characters on the Table data, need to deal on this too.
Do Liquibase has some limitation or we are doing anything wrong in the process.

your inputs will be highly appreciated. Thank you.

2

There are possibly several places where this kind of thing can go wrong:

  • the database table or column (depending on your database and its configuration, mysql, mssql, mongodb etc) needs to have the character set that supports such characters. I presume you want UTF-8 but there are various flavours even of “UTF-8” that you can use.
  • the connection that liquibase makes to it must support transmitting those characters (e.g. the url connection string in the properties file)
  • the data source that writes the characters must support the characters (you do not specify if your characters are stored in the changeset XML file, or are read from a file and written to the database, etc)
  • when testing if the characters were written to the database, the connection string must again support the right character set (e.g. if you use a SQL tool to read from the database and display contents from a SELECT statement it must be able to read the characters, I’ve seen squirrel-sql reading valid UTF-8 emoji from a database but displaying them as boxes because its connection string didn’t translate them properly)
  • when displaying them on screen via something like Java or PHP the display encoding must be correct, e.g. if you read from a valid connection string but display via a PHP system that doesn’t support UTF-8 it may not render the characters on your output web page correctly.
    (probably more that I haven’t thought of, but I’ve seen all these go wrong at various times)

So… at which point do you think you’ve done things right, and which point might it be failing? :slight_smile:

3

Hi @Sangramsk ,

You will need to introduce the following environment variables:

For Windows:
set JAVA_OPTS=-Dfile.encoding=UTF-8 && liquibase update

For Linux:
JAVA_OPTS=-Dfile.encoding=UTF-8 liquibase update

1 Like
4

DB2LUW>set JAVA_OPTS=-Dfile.encoding=UTF-8 && liquibase update
Starting Liquibase at 18:57:19 (version 4.6.1 #98 built at 2021-11-04 20:16+0000)
Liquibase Version: 4.6.1
Liquibase Community 4.6.1 by Liquibase

Unexpected error running Liquibase: Lexical error at line 51, column 47. Encountered: “\ufffd” (65533), after : “”

For more information, please use the --log-level flag
Logs saved to C:\Apps\Databases\liquibase-4.6.1\LiquibaseDB2LUW\liquibase.log

We still have the error even setting the UTF-8

5

Thanks Aditi , not much changes after setting the below env value.
set JAVA_OPTS=-Dfile.encoding=UTF-8 && liquibase update

6

Its resolved now.

The issue is little different what we have anticipated.

We have a “” character within the query which skewed the query, and it was not parsing the query, Could you please help how we can handle the “” characters.

7

I haven’t really tried this, but you may try enclosing it in single quotes or by escaping the "".