smartcat-labs cassandra-migration-tool-java
License: Apache License 2.0
Language: Java
Cassandra migration tool for java is a lightweight tool used to execute schema and data migration on Cassandra database. Schema versioning state is saved in schema_version
table which stores name and description of migration along with type and timestamp.
There are two types of migrations:
The idea behind this project was born while working on a project with live data. Since the development required changes to the data model we had to figure out a way we can update the database schema while keeping the data. We ended up with a lightweight tool that versions database and enables us to change schema and transform data when and how we want (as a part of a build and deploy cycle or application itself). Since these changes are part of the codebase we could also test them before executing on a live cluster.
We are going to follow the official DataStax Java driver versions and build our driver for each major, minor and patch version of the driver. We chose this path because of the simplicity to develop, test and deploy for different versions of the driver. This means that if you are using the driver version 2.1.9 you can add this dependency to your project:
<dependency>
<groupId>io.smartcat</groupId>
<artifactId>cassandra-migration-tool</artifactId>
<version>2.1.9.0</version>
</dependency>
or whatever the latest build version of the migration tool is. Check the version at the Maven repository.
We have two test cases which explain common problems which migration tool can solve. This is only subset of use cases but we think these are most frequent once:
First use case touches problem of adding new field and populating historic data with value. Cassandra does not have DDL default
attribute, so you must populate data on application level. MigrationEngineBooksTest can serve as an example of that use case which this tool can cover.
The initial table is simple (can be found in books.cql file) and we use migration classes to do the following:
data
migration InsertBooksMigrationgenre
column with schema
migration AddBookGenreFieldMigrationgenre
column with second data
migration AddGenreMigrationSecond use case touches problem of query based modeling. Cassandra has good performance because you model your data as you will query it. Often after initial modeling you have request to read it based on different criteria. In Cassandra you do this with another table which is optimized for new requirements. You need to populate this new table with existing data and you can solve this with migration tool. MigrationEngineItemsTest can serve as an example of that use case which this tool can cover.
The initial table is simple (can be found in items.cql file) and we use migration classes to do the following:
data
migration InsertInitialItemsMigrationitems_by_number_external_id
table with schema
migration CreateItemByNumberAndExternalIdMigrationitems_by_number_external_id
table with second data
migration PopulateItemByNumberAndExternalIdMigrationWhen executing schema migrations it is necessary to wait for cluster to propagate schema on all nodes. Schema agreement is implemented based on this fix and is exposed through Migration abstract class.
To execute a statement with schema agreement you can use executeWithSchemaAgreement
method.
Join Our Newsletter!
Sign up below to receive email updates and see what's going on with our company.