Liquibase 2.1 planning

General Discussion
1

Now that 2.0.2 is out, I’d like to start in earnest on 2.1, which will focus on diff support and cross-database logic. If there are major bugs in 2.0.2 there may be a 2.0.3 release, but the bulk of my time will be on 2.1 now.


You can see a list of all the bugs I have in 2.0.3 and 2.1 at:

http://liquibase.jira.com/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+CORE+AND+resolution+%3D+Unresolved+AND+fixVersion+in+%2810060%2C+10020%29+ORDER+BY+priority+DESC  


Looking at the Jira issues, I see the following major areas that need to be addressed:


Improving Database Snapshots


Currently we rely on JDBC metadata for fetching tables, columns, indexes, etc. but there are many cases in which the data returned is incomplete and/or incorrect. We may need to convert to more database-specific metadata reading in many cases, such as reading from the information_schema schema


Snapshots currently only take basic object types into consideration, and even on those only understands standard attributes. For example, we don’t snapshot check constraints at all, and even though we snapshot indexes we don’t track if they are clustered or nonclustered.


Standardizing Snapshot Comparisons


Comparing two snapshots of the same database type runs problems with the definition of “equal”. For example, if two foreign keys are on the same table and columns, are they the same even though their names are different?


Besides comparing between the same database type, diff comparisons are done between different databases as part of the generateChangeLog (where source database is an empty database), hibernate integration, and by end users. With cross-database type comparisons, we run into issues like the equality comparison above, but with extra nuances like “does an oracle clob equal a sqlserver varchar”.


Standardizing and Documenting Standard Data Types 


What are the available standard datatypes in the changelog file? What do those map to in each database type, and how can we auto-generate documentation based on that? 


What if a database-specific type matches but conflicts with a standard datatype? We have TIMESTAMP as a data type that means date+time, but it means something different in mysql.


Extension Support of Diff


No matter how robust the diff tool is, it will still not meet everyone’s needs. We need to make sure there is extensiblity built into the framework.


Remember that Diff is a Side Project of Liquibase


While there are times that database diffs are helpful, at its core, liquibase is not a diff tool and diff should not be a normal part of the liquibase workflow. There needs to be a point were we say “the diff is good enough for what it should be used for”. 


Prior Discussions


There has been some discussion before on 2.1 features on the forum, including:

Are there any other threads that should be included in the design process?


Tracking Design


If you have comments on the general "what should be included in 2.1 from a “database diff” standpoint, add it to this thread. If you one to comment more in depth on a particular feature, start a dedicated thread on the Development forum. I think that will help keep things organized.


Thoughts?

Nathan



2

I am still learning the product and terminology but one of the first issues I ran into was when diffing my MySQL vs  my SQLServer database.  The diff show a difference but the report indicated both columns were DATETIME and when I found out the final problem is was the COLUMN_SIZE was also being included in the comparison.

This brought two ideas:

  1. If the diff reports columns as different, it would be good to include ALL fields that were part of the comparison, so in my case, it could have been DATETIME COLUMN_SIZE=19 vs DATETIME COLUMN_SIZE=23.
  2. I think it may be what you are referring to as the STANDARD DATA TYPES, but maybe in this case, if the diff comparison has some flag to indicate strict comparison vs lenient comparison.  In the case above, a lenient comparison would show no difference but the strict comparison would show the difference that it currently finds.

3

Discussion on how the liquibase type->database type conversion started at http://forum.liquibase.org/#Topic/49382000000025389


Nathan