Installing the Prometheus-Operator with Helm v3

I’ve recently had to install Prometheus and Grafana on a brand new AKS cluster. Having done this before with Helm v2 I wanted to see how different (stream lined?) the process has become.

For this exercise I’m using the latest version of Helm (v.3.2.4) as of today:

~ $ helm version
version.BuildInfo{Version:"v3.2.4", GitCommit:"0ad800ef43d3b826f31a5ad8dfbb4fe05d143688", GitTreeState:"clean", GoVersion:"go1.13.12"}

Create a namespace for Prometheus and Grafana to live:

~ $ kubectl create ns monitor
namespace/monitor created

NOTE: Make sure you add a repo to Helm. This will save you the frustration of asking: What is going on here ? Something like this is a good indication that you haven’t initialize Helm with a repo.

~ $ helm install stable/prometheus-operator --generate-name 
Error: failed to download "stable/prometheus-operator" (hint: running `helm repo update` may help)

Adding a repo:

~ $ helm repo add stable https://kubernetes-charts.storage.googleapis.com
"stable" has been added to your repositories

Installing the prometheus-operator chart:

~ $ helm install stable/prometheus-operator --generate-name --namespace monitor
manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
NAME: prometheus-operator-1594052836
LAST DEPLOYED: Mon Jul  6 10:27:24 2020
NAMESPACE: monitor
STATUS: deployed
REVISION: 1
NOTES:
The Prometheus Operator has been installed. Check its status by running:
  kubectl --namespace monitor get pods -l "release=prometheus-operator-1594052836"

Visit https://github.com/coreos/prometheus-operator for instructions on how
to create & configure Alertmanager and Prometheus instances using the Operator.

Argh! What’s up with those pesky crd-install warnings? Let me clear that in a second. As per this thread on Github this chart will work for both versions of Helm: v2 and v3. The comments there also suggest that this is harmless in Helm v3 - with that I’m removing them by adding the following parameter to our chart config (inline): prometheusOperator.createCustomResource=false

This removes those warnings:

~ $ helm install stable/prometheus-operator --generate-name --set prometheusOperator.createCustomResource=false --namespace monitor
NAME: prometheus-operator-1594053273
LAST DEPLOYED: Mon Jul  6 10:34:37 2020
NAMESPACE: monitor
STATUS: deployed
REVISION: 1
NOTES:
The Prometheus Operator has been installed. Check its status by running:
  kubectl --namespace monitor get pods -l "release=prometheus-operator-1594053273"

Visit https://github.com/coreos/prometheus-operator for instructions on how
to create & configure Alertmanager and Prometheus instances using the Operator.

Finally, let’s check if the deployment worked out as expected:

~ $   kubectl --namespace monitor get pods -l "release=prometheus-operator-1594053273"
NAME                                                            READY   STATUS    RESTARTS   AGE
prometheus-operator-159405-operator-6bcff87479-hsz27            2/2     Running   0          9m5s
prometheus-operator-1594053273-prometheus-node-exporter-c6bvr   1/1     Running   0          9m5s
prometheus-operator-1594053273-prometheus-node-exporter-jpzdk   1/1     Running   0          9m5s
prometheus-operator-1594053273-prometheus-node-exporter-v7nrd   1/1     Running   0          9m5s
~ $ 

Accessiong Prometheus

To access the Prometheus dashboard, we will port-forward to the pod running the operator:

kubectl port-forward  $(kubectl get pod -n monitor --selector app=prometheus --output=jsonpath="{.items..metadata.name}"
) 9090

Connecting to Grafana

~ $ kubectl port-forward $(kubectl get pods -n monitor --selector app.kubernetes.io/name=grafana --output=jsonpath="{.items..metadata.name}") 3000
Forwarding from 127.0.0.1:3000 -> 3000
Forwarding from [::1]:3000 -> 3000

NOTE: To login you will be using admin as your user and prom-operator as the password. This information can be retrieved from the Kubernetes secrets created when the chart was installed - In my case since I did not specify a name the secrets were under prometheus-operator-1594053273-grafana:

grafana

kubectl get secret --namespace monitor prometheus-operator-1594053273-grafana  -o jsonpath="{.data.admin-password}" | base64 --decode ; echo
prom-operator