Diacritics Handling in LoadData from CSV to Postgres

General Discussion
1

Hi Team,

I am trying to load a reference data codeset into my table which has standard datatypes i.e. Character Varying in PostgreSQL.

The input is a comma seperated CSV file which has matching columns and matching data types with the table it is loading into.

I am using the loadData tag to load the data from CSV.

The CSV contains some diacritics, data like this:

,Åland Islands,Åland Islands
Côte d’Ivoire,Côte d’Ivoire
The CSV gets loaded into the database without any errors but when we query the data inside the database it has “?” against the diacritics.

I want to load this diacritics as how they appear in the CSV(above).

can anyone guide me which option to use in loadData for this?
Encoding is UTF-8

TIA,
Jigar

2

Hi @jpota ,

After doing some investigation I found that by adding the following argument below your issue might be fixed.

For Windows:
set JAVA_OPTS=-Dfile.encoding=UTF-8 && liquibase update

For Linux:
JAVA_OPTS=-Dfile.encoding=UTF-8 liquibase update

You can also refer to loadData documentation. You’ll see that the “encoding” attribute in loadData change type defaults to UTF-8.

3

@amalik
Thanks for responding, I will check with this settings today and let you know the result.