I have been working on extensions to add support for Cassandra. While Cassandra is not a RDBMS the work around CQL (cassandra query language) has produced tools like cassandra-jdbc, a JDBC driver for Cassandra. The DML, e.g., select, update, etc. functionality is rather limited though. Much of this is due to the fact that Cassandra is not relational database. One place where I ran into trouble is with LockDatabaseChangeLogGenerator. I cannot run that update statement against Cassandra.
My preference, at least at this stage, would be to somehow disable or turn off the locking. I have tried simulating that with a few extensions, notablly LockDatabaseChangeLogGeneratorCassandra and UnlockDatabaseChangeLogGeneratorCassandra. LockDatabaseChangeLogGeneratorCassandra applies a simple update that always sets the locked column to true, and UnlockDatabaseChangeLogGeneratorCassandra also sets the column to false. I ran into problems though in the LockService.acquireLock method at line 104ish which is,
int rowsUpdated = executor.update(new LockDatabaseChangeLogStatement());
My problem is that rowsUpdated is always 0 even though the update happens. The reason being is that the update operation translates to an operation on the server with a return type of void; consequently, the cassandra jdbc driver always returns 0 when its implementation of java.sql.Statement.executeUpdate is invoked. LockService is never able to get the lock. The return value of zero makes sense because there really isn’t a good way to indicate the number of rows updated. The update might not happen immediately as it has to propagate across nodes in the cluster. Furthermore, one of the target nodes that should receive the update could be down, and when it comes back up, it will receive and apply that update.
At this point, I do not see how to work around this using the existing extension points. I want to know how feasible it would be to add an extension point either for LockService or for Executor. Extending LockService would allow me to more or less turn off locking, but maybe extending Executor would be a safer, more narrowly scoped change. I basically want to wrap JDBCExecutor and intercept the call to update(new LockDatabaseChangeLogStatement()).
I am currently blocked on this. If this is doable, I am more than happy to work on and provide a patch.
Thanks
John
I think an extendable LockService would be the best approach, because it would allow you to include a no-locking.jar file in the classpath and maven, ant, grails, etc. would just work without needing to modify them. I’d also see an extendable LockService helpful for many other people as well with different needs for locking.
Nathan
I think making LockService be extendable would be the best option because it is the the logical unit you want to replace. Executor is the class that all statements go though, which may be a larger change to deal with and seems like not the right place to change how the locking works. If you wanted to look at providing a patch for LockService to work like the other extendable classes it would be nice to include. I know there are other times people have asked for making the lock service a no-op, so it would be nice to have an extension to point them too as well.
That being said, I think you could get it working as it is by overriding the Executor now. If you call ExecutorService.getInstance().setExecutor(cassandraDatabase, new NonLockingExecutor()) you can have the custom executor.update() method check if the passed Statement is a LockDatabaseChangeLogStatement or not.
Nathan
The createLiquibase() method is protected, so you could create your own maven plugin that just has classes extending the regular liquibase maven classes with:
- protected Liquibase createLiquibase(ResourceAccessor fo, Database db) throws MojoExecutionException {
- executorService.setExecutor(database, executor);
- return super.createLiquibase(fo,db);
- }
You’d have to subclass any maven tasks you are using, but you wouldn’t be needing a custom build at least. If only java allowed monkey-patching…
Nathan
Hi Nathan,
Thanks for the response and suggestions. I have done some additional prototyping and have had success with overriding Executor as you suggested. Here is another question. I plan on using the maven plugin during development. I am not sure I see a way use NonLockingExecutor without modifying the maven plugin code. In my prototype I have modified AbstractLiquibaseMojo.execute. I have done the following,
…
database = CommandLineUtils.createDatabaseObject(…)
This works great, but I am not sure if/how I can do this without modifying either the maven plugin or core code. Any suggestions?
With Clojure, you wouldn’t even need monkey patching 
If I go this route, there are minimal changes to make, but I have to fork the maven plugins. Now if I am running Liquibase in another context, like embedded in application code, then this would not be an issue. I am not sure however, about the implications for running from Ant or Grails. If I go the route of making LockService extendable, there is some more work, but I can run Liquibase from Maven without having to introduce new plugins. Any preference for one approach or the other?
It should be at https://github.com/liquibase/liquibase-cassandra/releases but it looks like there have been no builds yet. It’s a community-contributed extension so I haven’t really looked at it much.
A quick look at the current state of the code appears that I have a few changes to make it work with 3.3.2. I’ll look at getting that done quickly and and put a snapshot build up for testing.
Nathan
I am also looking for same. From where i get this modified jar file.
Hi all,
I have also been looking at extending liquibase to support cassandra
migrations. It looks like the existing liquibase-cassandra [1] extension
relies on using some sort of cassandra-jdbc driver, such as
org.apache-extras.cassandra-jdbc:cassandra-jdbc:1.2.5 [2] which has not
seen an updated release since 2013.
I started looking into implementing a liquibase-cassandra extension with
a DatabaseConnection implementation that uses the Datastax Java driver
under the hood, which is actively developed and uses CQL & the
cassandra binary protocol [3]. The first impediment towards this (or in
general extending liquibase to support any non-JDBC system) seems to be
the fact that DatabaseFactory is responsible to locate the JDBC driver
and setup the JDBC connection. Should this perhaps be the responsibility
of the DatabaseConnection implementation? eg. add a method to
DatabaseConnection interface like:
void
openConnection(Map<String, String> connectionProperties,
ResourceAccessor resourceAccessor);
and delegate the responsibility to manage everything DB-specific to the
implementation of DatabaseConnection? Does this direction look good or
should I pursue a different path? I am willing to work on this and would
like to contribute back something useful, so feedback is greatly appreciated.
Thanks,
Vassilis
[1] https://github.com/liquibase/liquibase-cassandra
[2] http://search.maven.org/#artifactdetails|org.apache-extras.cassandra-jdbc|cassandra-jdbc|1.2.5|jar
[3] http://wiki.apache.org/cassandra/ClientOptions
