I have a question as to the best use of LoadData. I’m on LQB 1.9.5 and Postgres 8.3/4. It works great for an initial data load, however since the CSV is calculated into the md5 sum, modifying the CSV later and using it in a new changeset that refreshes the tables doesn’t work because of a checksum violation. I’ve tried adding to the changesets that load the data, but it still fails checksum validation. Here’s what that changeset looks like:
delete from errors;
I tried taking advantage of the fact that the CSV is part of the md5 sum and setting runOnChange to true. That worked, but not in all cases since runOnChange changesets run after everything else, and sometime migrations rely on data previously loaded.
So I’m still kinda stuck. Is there a good workaround for this short of clearing the checksum?
Ah, I see. This works: 5d89f4ba80b20fdb69da68b4a487a68
It’s not quite as easy as I’d like though; if I change my CSV regularly I have to keep updating all the changesets that reference it with this. It would be nice to tell the LoadData task not to include the CSV in the md5 calculation.
Glad you found the solution, sorry I hadn’t gotten back to you.
Not including a way to say “take whatever checksum you get” is a design decision I’ve been trying to hold to. My concern is that it would be abused much more than there are valid uses for it. The checksums are there to prevent surprises when people think they can change changeSets and expect that the database will match the current changeSet. There are times you want to make changes, which is why I put in the validCheckSum field, but a blanket “take whatever” attribute would probably be used by default on all changesets by too many people.
CSVs are a bit different, perhaps, because there are times when you want to allow changes to the csv file to simply be ignored by anyone that has already ran it. Test data in particular comes to mind. Adding an “includeInChecksum” flag on loadData may work, but I’m still concerned that it would cause more pain than it would solve. I’ll think about it a bit.
Just wanted to say, I’m having the same problem on the checksum for CSVs. I wanted to use them to load test data as well. Not having a way to be able to change the CSV file after initial creation to add/remove test data is going to be an issue. I hope that you do add something to ignore or update the checksum for loadData at some point soon.
