Cassandra.Tools

A Curated List of Apache Cassandra Dev Tools

About

Heroic

https://spotify.github.io/heroic/ spotify (3454939) spotify/heroic (767)
Languages:Java
Templates:Cassandra
License:Apache 2.0

Heroic Heroic

Build Status Codecov License

A scalable time series database based on Bigtable, Cassandra, and Elasticsearch. Go to https://spotify.github.io/heroic/ for documentation.

This project adheres to the Open Code of Conduct. By participating, you are expected to honor this code.

Install

Docker

Docker images are available on Docker Hub.

$ docker run -p 8080:8080 -p 9091:9091 spotify/heroic

Heroic will now be reachable at http://localhost:8080/status.

In production it's advised to use a tagged version.

Configuration

For help on how to write a configuration file, see the Configuration Section of the official documentation.

Heroic has been tested with the following services:

Developing

Building from source

In order to compile Heroic, you'll need:

  • A Java 11 JDK
  • Maven 3
  • Gradle

The project is built using Gradle:

# full build, runs all tests and builds the shaded jar
./gradlew build

# only compile
./gradlew assemble

# build a single module
./gradlew heroic-metric-bigtable:build

The heroic-dist module can be used to produce a shaded jar that contains all required dependencies:

./gradlew heroic-dist:shadowJar

After building, the entry point of the service is com.spotify.heroic.HeroicService. The following is an example of how this can be run:

./gradlew heroic-dist:runShadow <config>

which is the equivalent of doing:

java -jar $PWD/heroic-dist/build/libs/heroic-dist-0.0.1-SNAPSHOT-shaded.jar <config>

Building with Docker

$ docker build -t heroic:latest .

This is a multi-stage build and will first build Heroic via a ./gradlew clean build and then copy the resulting shaded jar into the runtime container.

Running heroic via docker can be done:

$ docker run -d -p 8080:8080 -p 9091:9091 -v /path/to/config.yml:/heroic.yml spotify/heroic:latest

Logging

Logging is captured using SLF4J, and forwarded to Log4j.

To configure logging, define the -Dlog4j.configurationFile=<path> parameter. You can use docs/log4j2-file.xml as a base.

Testing

We run tests with Gradle:

# Start pubsub emulator
gcloud beta emulators pubsub start

# run unit tests
./gradlew test

# run integration tests
./gradlew integrationTest

or to run a more comprehensive set of checks:

./gradlew check

This will run:

It is strongly recommended that you run the full test suite before setting up a pull request, otherwise it will be rejected by Travis.

Remote Integration Tests

Integration tests are configured to run remotely depending on a set of system properties.

Elasticsearch
Property Description
-D elasticsearch.version=<version> Use the given client version when building the project
-D it.elasticsearch.remote=true Run Elasticsearch tests against a remote database
-D it.elasticsearch.seed=<seed> Use the given seed (default: localhost)
-D it.elasticsearch.clusterName=<clusterName> Use the given cluster name (default: elasticsearch)
Datastax
Property Description
-D datastax.version=<version> Use the given client version when building the project
-D it.datastax.remote=true Run Datastax tests against a remote database
-D it.datastax.seed=<seed> Use the given seed (default: localhost)
Bigtable
Property Description
-D bigtable.version=<version> Use the given client version when building the project
-D it.bigtable.remote=true Run Bigtable tests against a remote database
-D it.bigtable.project=<project> Use the given project
-D it.bigtable.zone=<zone> Use the given zone
-D it.bigtable.instance=<instance> Use the given instance
-D it.bigtable.credentials=<credentials> Use the given credentials file

The following is an example Elasticsearch remote integration test:

$> mvn -P integration-tests \
    -D elasticsearch.version=5.6.0 \
    -D it.elasticsearch.remote=true \
    clean verify
PubSub

PubSub relies on having the PUBSUB_EMULATOR_HOST environment variable set instead of a system property. Detailed instructions are available in the Google PubSub emulator docs.

Full Cluster Tests

Full cluster tests are defined in heroic-dist/src/test/java.

This way, they have access to all the modules and parts of Heroic.

The JVM RPC module is specifically designed to allow for rapid execution of integration tests. It allows multiple cores to be defined and communicate with each other in the same JVM instance.

Code Coverage

Coverage

There's an ongoing project to improve test coverage. Clicking the above graph will bring you to codecov.io, where you can find areas to focus on.

Bypassing Validation

To bypass automatic formatting and checkstyle validation you can use the following stanza:

// @formatter:off
final List<String> list = ImmutableList.of(
   "Welcome to...",
   "... The Wild West"
);
// @formatter:on

To bypass a FindBugs error, you should use the @SupressFBWarnings annotation.

@SupressFBWarnings(value="FINDBUGS_ERROR_CODE", justification="I Know Better Than FindBugs")
public class IKnowBetterThanFindbugs() {
    // ...
}

© 2020 Anant Corporation, All Rights Reserved. All logos, trademarks and registered trademarks are the property of their respective owners. .

© Netlify 2020