CassKop deals with Cassandra clusters on one datacenter. For multi-dacenters deployments, please use MultiCassKop in addition to CassKop. This second operator is part of this same repository.
The following features are supported by CassKop:
Rolling update of a Cassandra cluster
Stopping a Kubernetes node for maintenance
CassKop doesn't use nodetool but invokes operations through authenticated JMX/Jolokia call
Operator SDK is part of the operator framework provided by RedHat & CoreOS. The goal is to provide high-level abstractions that simplifies creating Kubernetes operators.
The quick start guide walks through the process of building the Cassandra operator using the SDK CLI, setting up the RBAC, deploying the operator and creating a Cassandra cluster.
You can find this in the Developer section
Users should only need Kubectl & helm cli
You can install the plugin by copying the file into your PATH.
We uses CircleCI as our CI tool to build and test the operator.
To accelerate build phases we have created a custom build-image used by the CircleCI pipeline:
https://cloud.docker.com/u/orangeopensource/repository/docker/orangeopensource/casskop-build
You can find more info in the developer Section
The Cassandra operator image is automatically built and stored on Docker Hub
The operator works with specific Docker Cassandra image which is build in the repository: https://github.com/Orange-OpenSource/cassandra-image
This table shows compatibility between CassKop and associated Cassandra image
Operator | Cassandra-k8s |
---|---|
0.3.1-release | >= 3.11.4-8u212-0.3.1 |
0.3.2-release | >= 3.11.4-8u212-0.3.2-release |
0.3.3-release | >= 3.11.4-8u212-0.3.3-release |
0.4.0-release | >= 3.11.4-8u212-0.4.0-release |
docker image: orangeopensource/cassandra-image:3.11.4-8u212-0.4.0-release or 3.11.4-8u212-0.4.0-release-cqlsh if you need cqlsh
First, we need to create a Kubernetes namespace
in order to host our operator & cluster
kubectl create namespace cassandra
To ease the use of the Cassandra operator, a Helm chart has been created
We are looking where to store our helm in the future
Helm 2 users
If it is the 1rst time you deploy a chart with Helm, you may need to install the helm client and tiller, and create the service account for tiller (more informations about how to create a tiller service account here.
helm init --history-max 200 kubectl create serviceaccount tiller --namespace kube-system kubectl create -f tiller-clusterrolebinding.yaml helm init --service-account tiller --upgrade
Helm 3 users
You will need to manually install the crds beforehand
kubectl apply -f https://raw.githubusercontent.com/Orange-OpenSource/casskop/master/deploy/crds/db.orange.com_cassandraclusters_crd.yaml
Add the Helm incubator repo if you do not already have it:
helm repo add orange-incubator https://orange-kubernetes-charts-incubator.storage.googleapis.com
Get the latest information about charts from the chart repositories.
helm repo update
helm install --name casskop orange-incubator/cassandra-operator
Helm 3 users
Remove --name from the command
Deploy CassKop:
$ helm install --name casskop orange-incubator/cassandra-operator
NAME: casskop
LAST DEPLOYED: Thu May 23 15:34:27 2019
NAMESPACE: cassandra-demo
STATUS: DEPLOYED
RESOURCES:
==> v1/ServiceAccount
NAME SECRETS AGE
casskop 1 0s
==> v1beta1/Role
NAME AGE
casskop 0s
==> v1/RoleBinding
NAME AGE
casskop 0s
==> v1/Deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
casskop-casskop 1 1 1 0 0s
==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
casskop-casskop-78786b9bf-cjggg 0/1 ContainerCreating 0
0s
You can find more information in the Cassandra operator Helm readme
If you have problem you can see troubleshooting section
This creates a Kubernetes Deployment for the operator, with RBAC settings.
Once deployed, you may find the Pods created by the Charts. If you deploy a release
named casskop
, then your pod will have a name similar to :
$ kubectl get pod
NAME READY STATUS RESTARTS AGE
casskop-cassandra-operator-78786b9bf-cjggg 1/1 Running 0 1h
You can view the CassKop logs using
$ kubectl logs -f cassandra-cassandra-operator-78786b9bf-cjggg
The charts also deploy the cassandracluster CRD we can check that it is deployed:
$ kubectl get crd
NAME AGE
...
cassandraclusters.db.orange.com 1h
...
Once the operator is deployed inside a Kubernetes cluster, a new API will be accessible, so you'll be able to create, update and delete cassandraclusters.
In order to deploy a new cassandra cluster a specification has to be created. As an example :
kubectl apply -f samples/cassandracluster.yaml
See pods coming to life :
kubectl get pods -w
You can watch the status updates in real time on your CassandraCluster object :
watch 'kubectl describe cassandracluster cassandra-demo | tail -20'
You can find mode information on the CassandraCluster.status
in this section
You can do a lot of operations on your Cassandra cluster.
If the Cassandra operator restarts, it can recover its previous state thanks to the CRD objects
CassandraClusters
which stored directly in Kubernetes, description and state of the Cassandra cluster.
Upgrading the operator consists in uninstalling the current version and installing the new version :
helm uninstall casskop
helm repo update
helm install --name casskop casskop/cassandra-operator
It's also possible to decide to temporarily install a developement release by specifying the image tag to use :
helm install --name casskop casskop/cassandra-operator --set debug.enabled=true --no-hooks \
--set image.tag=v0.5.0b-branch1
:triangularflagon_post: We made a few breaking changes in that image:
Because of it upgrading casskop from 0.5.0 to 0.5.1+ requires to the following steps:
helm delete casskop
)Edit all statefulsets to manually make those few changes using kubectl edit statefulsets $name-of-your-statefulset
Add a new emptyDir in the volumes section
volumes:
- emptyDir: {}
name: tools
....
Mount that new volume in all initcontainers and containers (:bulb: init-config does not really need it, but we need to change it first in the operator #189)
name: init-config
....
volumeMounts:
- mountPath: /opt/bin
name: tools
....
name: bootstrap
....
volumeMounts:
- mountPath: /opt/bin
name: tools
....
name: cassandra
...
volumeMounts:
- mountPath: /opt/bin
name: tools
....
If you want to delete the operator from your Kubernetes cluster, the operator deployment should be deleted.
$ helm delete casskop
The command removes all the Kubernetes components associated with the chart and deletes the helm release.
The CRD created by the chart are not removed by default and should be manually cleaned up (if required)
Manually delete the CRD:
kubectl delete crd cassandraclusters.dfy.orange.com
:triangularflagon_post: If you delete the CRD then it will delete ALL Clusters that has been created using this CRD!!! Please never delete a CRD without very very good care
CassKop is build using operator SDK:
We can quickly setup monitoring for our deployed Cassandra nodes using Prometheus operator.
You can deploy the CoreOs Prometheus operator on your cluster: You can find example helm value.yaml to configure the Prometheus operator:
$ kubectl create namespace monitoring
$ helm install --namespace monitoring --name prometheus stable/prometheus-operator
Then you have to define a ServiceMonitor object to monitor cluster deployed by your cassandra operator (one time), update this to specify which namespace to monitor.
cassandra-service-monitor.yml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: kube-prometheus-cassandra-k8s-jmx
labels:
k8s-apps: cassandra-k8s-jmx
prometheus: kube-prometheus
component: cassandra
release: prometheus
spec:
jobLabel: kube-prometheus-cassandra-k8s-jmx
selector:
matchLabels:
k8s-app: exporter-cassandra-jmx
namespaceSelector:
matchNames:
- cassandra
- cassandra-demo
- default
endpoints:
- port: promjmx
interval: 15s
Your namespace need to be listed in the namespaceSelector
section.
You can import this dashboard to retrieve metrics about your Cassandra cluster.
© 2020 Anant Corporation, All Rights Reserved. All logos, trademarks and registered trademarks are the property of their respective owners.