Hello All,
I happened across LiquiBase as I was recently doing some Google searching. It looks like an interesting project and I would like to get involved at the user level or at the developer level.
In any case, I recently created an opensource project that has some similarities to LiquiBase. I developed my solution before I learned about LiquiBase and it takes a different approach.
http://code.google.com/p/garindriver/
My project is similar to LiquiBase in some respects but a took a different approach. GarinDriver is a much smaller project (dev team of 1). Most of the code was a weekend project (last weekend). The approach, I took was to integrate the system for running the deployment scripts into the JDBC driver layer. By integrating the deployment of scripts into the JDBC driver my system ensures that client’s can never access the database until it is properly initialized (i.e. the JDBC url used by the client describes a minimal database level and internally the GarinDriver can promote the database to the proper level by running required members from a stack of scripts using a databased tracking table to coordinate script running). The GarinDriver contract with scripts is basically run once and supports many clients with contention rules to prevent conflict. When a client connects through the driver it checks it’s stack of scripts vs. the database tracking table and runs only the scripts needed to move the database forward. The GarinDriver is really a wrapper around a backend jdbc driver and as such the GarinDriver focus is on the script stack with all real work handed off to the backend jdbc driver. It seems like a different approach then LiquiBase and there is a possibility that the concept could be merged into LiquiBase.
From my understanding the LiquiBase concept is similar but also includes a set of tools to create a stack of deployment scripts and a way to organize running such scripts. What I really like is that LiquiBase seems to have a database vendor neutral way of describing DDL changes (and possibly a vendor neutral way of describing data loads).
The LiquiBase servlet listener approach is somewhat similar to my approach but I think integration at the JDBC driver level is a better approach vs. servlet listener since it prevents clients from getting to the database before the servlet listener has a chance to do it’s work (i.e. other servlet listers and/or objects that kick off from web context initialization). In addition the JDBC driver approach works for applications that are not running in the servlet model.
Does LiquiBase provide a JDBC driver wrapper concept similar to GarinDriver?
Do members of the LiquiBase community think that JDBC driver integration concept provides value?
Is LiquiBase interested in the idea of integration with the GarinDriver concept?
-Mark
http://markfarnsworth.wordpress.com
I had not thought of implementing a hook into liquibase at the JDBC driver layer before. I think that is a very good idea, it abstracts away the fact that liquibase runs and will automatically integrate into any database connection. It is not something I thought of before, but would certainly simplify things.
What sort of integration are you thinking of? If you replace your update code with liquibase api calls into your project but want to remain a separate project, we could list you at http://www.liquibase.org/extensions. If you would like to to simply submit your code to us and have it included directly in the liquibase codebase, I think it would be a great addition, and may be what we want to look at as the recommended way to execute liquibase.
Not having looked at your code, my only concern may be classpath issues. Often times the JDBC driver is loaded from a higher classloader than liquibase, your changelog file, and any liquibase extensions/custom change classes. For example, the jdbc driver is in tomcat’s lib directory, but liquibase etc. is in your WEB-INF/lib directory. If you put liquibase into the app server’s lib directory to act as a JDBC driver, it will not have access to the changelog files etc. via the classloader like it usually expects. How have you been loading your update scripts? Have you ran into and solved similar issues, or is it not really a problem?
Nathan
Nathen,
Thanks for the response.
My script engine is simple but it runs standard mysql dumps and meets my needs so my general plan of integration would be to allow the GarinDriver to accept a URL format that would allow the user to select the schema management engine. This way the user can plug in the LiquiBase schema engine if needed. LiquiBase would be then be in good position to bundle my driver with the distribution.
My equivalent of change log is my script stack of numbered sql files.
My script engine currently expects to find the scripts on the file system. I don’t currently support distribution of schema scripts as resources within the war file. As you point out, accessing resources of a client’s class loader from the driver is is a bit tricky butwith a little bit of JVM trickery I think I could handle it.
For our tomcat solution we have the drivers in $CATALINA_HOME/common/lib so they can be shared with all webapps running on the server. The scripts are all placed in the $CATALINA_BASE/deploy folder. This deploy folder is specific for our application (The actual folder name used for deploy can be passed in as part of the GarinDriver connect string).
Right now GarinDriver is limited to use with MySql but this is only because my schema engine is limited to mysql (i run native mysql multi statement DDL/SQL scripts). My script parser is mysql specific ’ vs. ‘’ for quote treatment. I also detect some mysql error conditions to control running the scripts in my engine.
After I get comfortable with LiquiBase as a schema engine I might choose to abandon my own little script stack schema engine. My script stack is very simple but at the same time it works well and meets my needs so for the short term allowing GarinDriver to support a parameter for schema engine makes sense (GarinDriver already uses a backend JDBC driver so it would be a similar approach). I like the idea of allowing a LiquiBase option as part of the GarinDriver JDBC URL.
With LiquiBase it should be simple because from what I understand you support all databases. What I need is an API where I can call a prepare method. I would pass a “real” connection reference to some sort of prepare method and expect success or an exception thrown to indicate failure.
I tried the link (http://www.liquibase.org/extensions) and was not able to login – what user do I need?
I will look at the code for your servlet listener and see if I can derive what would be needed to make integration work.
Is it currently possible to load the change log from the file system or does it HAVE to be loaded from the class loader?
I will get back to you about the class loader issue. The ability to load from the client’s class loader would be nice and I have some ideas that I want to try.
If you have time and interest please look at my existing code. Review is always helpful.
http://code.google.com/p/garindriver/source/checkout
I fixed up the /extensions url so it is publicly accessible now. That is the site that we are building up with information on embedding and extending liquibase. So it should be the best place for the info you are looking for. http://liquibase.jira.com/wiki/display/CONTRIB/Liquibase+Facade in particular has the code you are looking for.
It is still new and in progress documentation and should continue to improve over the next couple weeks.
There is the ability to directly reference changelog files by filesystem rather than classpath, when you create the liquibase.Liquibase object you are able to pass a liquibase.resource.ResourceAccessor, one of which is FileSystemResourceAccessor. The classpath route is generally best because then the name and path to the changelog is not installation-dependent and will be tracked better as our databasechangelog table contains the changelog file/path as well.
If it helps give you incentive to build in liquibase as an option, there is the soon-to-be-officially-announced LiquiBase extension contest
Nathan
Support for LiquiBase as an optional schema manager is now available in GarinDriver!!!
http://markfarnsworth.wordpress.com/2009/06/27/garin/
I did some testing and it seems to work quite well although really I am not an expert in designing LiquiBase change sets.
The driver allows users to configure a change set and optionally a context string. When the system establishes connections it hands the connection to LiquiBase prior to returning the connection to the client. This ensures 100% that LiquiBase is invoked prior to any application code. For teams that keep their changeset file in a code repository (i.e. like SVN) it team members need only configure their local environments to use the garin driver and in doing so they can ensure that the schema manager will run when they start up their applications (i.e. no ant task, manual running, etc). This is in essence similar to the servlet listener path but IMHO easier to setup and more reliable. For Tomcat you simply replace your current JDBC url (often in servlet.xml) with the GarinDriver url.
In any case please give it a try and let me know what you think.
The documentation is non-existent at this point but it is also very easy to use so not much documentation is required.
To give it a try I recommend downloading the current-release JAR from my google code site and then configuring your JDBC URL based as follows:
garin:m=liquibase,s=myapp/liquidbase/changelog.xml;demo:jdbc:mysql://localhost/myDB
The following link takes you to the google code site.
http://code.google.com/p/garindriver/
I’ll take a look and let you know if I have any suggestions.
If you’d like, please add it to the extension portal (http://www.liquibase.org/extensions). I’m hoping the documentation on there tells you how to add it, if it is unclear let me know so it can be improved.
Nathan
Great, thanks! Do you have any suggestions on how to improve the adding process?
