I’ve been working on getting Elasticsearch working on Kubernetes-On-ARM. The biggest problem has been storage. I’m using Elasticserach for storing logs and the cluster generates 1.4 million entries per day (i know, need to do something about it). If you want to deploy your applications to Kubernetes, sooner or later you need to think about how to solve storage and persistence.
Kubernetes is a distributed cluster where nodes and pods comes and goes. We don’t want to solve our persistence problem for a single node, we want to solve it for the whole cluster. That’s where Kubernetes Volumes comes in. A quick look at the different Volume Plugins that are available gives us the following alternatives:
I wrote an earlier post about
GlusterFS On Kubernetes-ARM.
I still use
glustefs and it’s running on my
boards, but the I/O performance isn’t enough for handling the logs from
Fluentd that are stored in Elasticsearch. We need a bigger machinery.
I’ve been looking for a board that have Gigabit Ethernet and SATA or
USB 3, that can be used for handling persistence. But buying a board and some
disks and configuring services, that sounds a lot like building your own NAS,
and there are plenty of cheap NAS products out there that supports at least two
of the alternatives on our list,
I already own a home
NAS and it supports both
I created two different
PersistentVolume:s, one for each Volume Plugin, and
mounted them in two different Elasticsearch data node pods, and suddenly, all the
problems with Elasticsearch and Kibana is gone.
iSCSI Initiator in Arch Linux is rather straight forward:
$ pacman -S --noconfirm open-iscsi $ systemctl enable open-iscsi.service $ systemctl restart open-iscsi.service $ iscsiadm -m discovery -t sendtargets -p <ip of your iscsi target>
You need to repeat the steps above for each node in your cluster and that’s it,
if your not going to use
Now we can create a
apiVersion: v1 kind: PersistentVolume metadata: name: lun0 spec: capacity: storage: 10Gi accessModes: - ReadWriteOnce persistentVolumeReclaimPolicy: Retain iscsi: targetPortal: xxx.xxx.xxx.xxx:yyyy iqn: <iqn-target> lun: 0 fsType: ext4 readOnly: false
I leave it up to you to create a
PersistentVolumeClaim and mount it in to your
If you want to use your Kubernetes-On-ARM cluster for more advanced applications then you need to think about how to solve persistence.
I’ve found a solution that suits my needs using
nfs, the only
problem is that my NAS is almost full, and not so portable.
Maybe it’s time to build
Kodbasen cluster version 2?